An intelligent cloud video analysis method and system based on artificial intelligence

Abstract

The application discloses an intelligent cloud video analysis method and system based on artificial intelligence, relates to the technical field of video analysis, and deploys an edge computing device at a video acquisition end, which is used for running an artificial intelligence large model to analyze original video streams in real time, obtaining structured data and processing the same based on a preset first trigger strategy, uploading data packets containing feature vectors and key video frames after processing to a cloud intelligent platform for graph analysis, obtaining analysis results and combining feedback information of relevant edge computing devices to make decisions, generating corresponding second trigger strategies and issuing the same to all edge computing devices, selecting execution by all edge computing devices, integrating analysis information of original video streams of several edge computing devices, generating corresponding global information sets, processing global information sets to obtain global event graphs and global event chains, and finally deciding on a video acquisition end with an anomaly, so that efficient and accurate video analysis is achieved.

Description

Technical Field

[0001] This invention relates to the field of video analytics technology, specifically to an artificial intelligence-based smart cloud video analytics method and system. Background Technology

[0002] With the increasingly widespread application of video surveillance systems and the continuous expansion of video acquisition equipment deployment, massive amounts of low-value, low-density raw video stream data are generated, placing higher demands on the real-time performance, accuracy, and global coordination of analysis technologies. Traditional analysis methods have significant shortcomings: Pure cloud-based centralized analysis requires uploading all raw video streams, leading to a surge in network bandwidth and cloud storage pressure, high transmission latency, cloud overload, and an inability to achieve real-time response. Pure edge-based independent analysis is limited by device computing power, making it impossible to efficiently run large AI models for in-depth analysis. Furthermore, isolated operation of each device lacks collaboration, making it difficult to integrate global information for judgment, resulting in a high false positive and false negative rate. At the same time, the lack of an effective two-way collaboration and closed-loop optimization mechanism between the cloud and the edge makes it impossible to dynamically adjust analysis strategies, which restricts the intelligent upgrade and large-scale multi-scenario application of smart cloud video analysis technology. Summary of the Invention

[0003] The purpose of this invention is to provide an intelligent cloud video analysis method and system based on artificial intelligence to address the shortcomings in the prior art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a smart cloud video analysis method based on artificial intelligence, comprising the following steps: Step S1: Deploy edge computing devices at the video acquisition end, and run an artificial intelligence model on the edge computing devices to analyze the original video stream in real time to obtain the structured data corresponding to the original video stream; Step S2: Based on the preset first triggering strategy, the structured data is processed, and the processed data packet containing feature vectors and key video frames is uploaded to the cloud intelligent platform, which then performs a graph analysis on the data packet. Step S3: Obtain the analysis results corresponding to the map analysis, and make a decision based on the feedback information of the relevant edge computing devices. Generate the corresponding second trigger strategy and send it to all edge computing devices, and let all edge computing devices choose to execute it. Step S4: Integrate the analysis information of the original video streams corresponding to several edge computing devices to generate a corresponding global information set. Based on the global information set, process to obtain a global event map and a global event chain, and finally decide on the video acquisition terminal with anomalies.

[0005] In a preferred embodiment, an edge computing device is deployed at the video acquisition end. The edge computing device runs a large artificial intelligence model to analyze the raw video stream in real time, and the process of obtaining the structured data corresponding to the raw video stream includes: The video acquisition terminal is equipped with a number of video devices, each video device corresponding to a capture coordinate. A number of edge computing devices are deployed at the video acquisition terminal to obtain the model scripts corresponding to the configuration of the large artificial intelligence model. The execution steps related to video parsing recorded in the model script are processed in the edge computing device to generate a parsing node for each execution step. The parsing node is used to parse the raw video stream of the edge computing device to obtain the structured data of the corresponding entries. By integrating the structured data of all entries in each edge computing device, a device-specific structured data is generated for that edge computing device. By integrating the structured data of all devices, the final structured data of the raw video stream being processed is obtained.

[0006] In a preferred embodiment, the process of processing structured data based on a preset first triggering strategy and uploading the processed data packet containing feature vectors and key video frames to a cloud-based intelligent platform includes: The first triggering strategy is used to extract data segments corresponding to the main information representing the original video stream from the structured data. The first triggering strategy is predefined based on the service type required by the cloud intelligent platform. The first triggering strategy is stored in the edge computing device in the form of a rule engine or configuration file. Set the packet size and obtain the data size of the currently processed feature vectors and key video frames in real time. Whenever the data size reaches the packet size, the corresponding feature vectors and key video frames are processed into a data packet and uploaded to the cloud intelligent platform. The feature vectors and key video frames included in the data packet are filtered by the first trigger strategy.

[0007] In a preferred embodiment, the process of performing spectral analysis of data packets by a cloud-based intelligent platform includes: The cloud-based intelligent platform has several domain knowledge graphs that represent different video semantic knowledge. Based on each domain knowledge graph, an inference point is created on the cloud-based intelligent platform. Each inference point creates a migration point at the location of the data packet, and a knowledge migration path is established between the inference point and the migration point. For each knowledge migration path, set corresponding knowledge matching information. The knowledge matching information is used to obtain the structured data in the data package that is adapted to the current knowledge migration path and to serve as graph retrieval information. The migration point will transfer the graph retrieval information to the inference point on the same knowledge migration path, and the inference point will adapt the corresponding domain knowledge graph based on the graph retrieval information. When all the structured data corresponding to a data packet has been adapted to the corresponding domain knowledge graph, it means that the cloud-based intelligent platform has completed the graph analysis of the current data packet, updated all the domain knowledge graphs adapted to the same data packet to a preset blank graph template, and then obtained a real-time event graph that integrates video semantic knowledge from different domains. The real-time event graph is updated based on each newly adapted domain knowledge graph, and the real-time event graph is used as the final analysis result.

[0008] In a preferred embodiment, the process of obtaining the analysis results corresponding to the map analysis, making a decision based on the feedback information from relevant edge computing devices, generating a corresponding second triggering strategy and distributing it to all edge computing devices, and having all edge computing devices select and execute the strategy includes: A timeline is established, which consists of several timeline nodes. The real-time event graph includes several timeline sub-event graphs. All behavioral events of several timeline nodes on the timeline are arranged based on the timeline correspondence of the timeline nodes to construct analytical information representing the location of a collection coordinate. All the analysis information is packaged into corresponding real-time data packets. Based on the acquisition coordinates corresponding to each analysis information, a separate data transmission channel is established for each real-time data packet. The analysis information is then transmitted to the respective affiliated edge computing devices through the respective data transmission channels. The feedback information cached in each edge computing device is obtained synchronously. The feedback information is compared with the analysis information to determine the device resolution status of each edge computing device. The generated second triggering strategy is sequentially sent to all edge computing devices through the data delivery channel and cached on the edge computing devices. Edge computing devices whose status is resolved to be untrusted are selected as the target devices for executing the second triggering strategy to update the devices. After the devices are updated, the edge computing devices generate their latest analysis information based on their original video streams.

[0009] In a preferred embodiment, the process of integrating the analysis information of the original video streams corresponding to several edge computing devices to generate a corresponding global information set includes: Initialize a blank layer covering the locations of all edge computing devices. On the blank layer, based on the coordinates of each edge computing device, create an information display point at the coordinates of each device. The information display point is used to store the analysis information of each edge computing device. A summary display point is constructed, and the analysis information of all information display points is uploaded to the summary display point. The summary display point integrates all the analysis information to generate the corresponding global information set. The global information set is used to record different node types and edge types.

[0010] In a preferred embodiment, the process of obtaining a global event graph and a global event chain based on global information set processing, and then ultimately determining the video acquisition terminal with an anomaly, includes: Treat each edge computing device as a node object, construct the mapping points of each edge computing device on the blank layer, connect all the mapping points on the blank layer based on the attributes of the node type and the functional relationship of the edge type, and construct the global event graph. Several global event chains are obtained by filtering based on the global event graph. Each global event chain is used to trace the abnormal behavior details of a video capture terminal at a certain capture coordinate. Based on all the abnormal behavior details, the capture coordinate location corresponding to the video capture terminal with the abnormality is marked.

[0011] This invention also provides an artificial intelligence-based intelligent cloud video analysis system, which includes: The edge computing module is used to deploy edge computing devices at the video acquisition end. The edge computing devices run a large artificial intelligence model to analyze the raw video stream in real time and obtain the structured data corresponding to the raw video stream. The first processing module processes the structured data based on a preset first triggering strategy, and uploads the processed data packet containing feature vectors and key video frames to the cloud intelligent platform, whereby the cloud intelligent platform performs a graph analysis of the data packet. The second processing module is used to obtain the analysis results corresponding to the map analysis, and make decisions based on the feedback information of the relevant edge computing devices, generate the corresponding second trigger strategy and send it to all edge computing devices, and all edge computing devices select to execute it. The analysis and decision-making module is used to integrate the analysis information of the original video streams corresponding to several edge computing devices, generate the corresponding global information set, process the global information set to obtain the global event map and global event chain, and finally decide on the video acquisition terminal with anomalies.

[0012] The technical effects and advantages provided by the present invention in the above technical solution are as follows: 1. By running a large AI model locally on edge computing devices to analyze the raw video stream, only core data containing feature vectors and key video frames are uploaded to the cloud, reducing network transmission pressure and cloud storage costs, improving the real-time response speed of video analysis, and effectively making up for the computing power shortcomings of edge computing devices by constructing a two-way collaborative mode of edge real-time analysis + cloud deep graph analysis, reducing the probability of false alarms and false negatives, and improving the accuracy of video analysis; 2. By integrating the analysis information from multiple edge computing devices, a global event graph and global event chain are constructed, breaking down information barriers between different devices. This enables rapid and accurate location of abnormal video acquisition terminals, achieving global collaborative analysis. Through a closed-loop mechanism of parsing, filtering, analysis, feedback, and optimization, the triggering strategies at edge computing devices are dynamically adjusted to enhance the system's adaptability and flexibly adapt to video analysis needs in multiple fields and scenarios. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0014] Figure 1 This is a flowchart of the method of the present invention.

[0015] Figure 2 This is a system block diagram of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Example 1, please refer to Figure 1 As shown in the figure, the intelligent cloud video analysis method based on artificial intelligence described in this embodiment includes the following steps: Step S1: Deploy edge computing devices at the video acquisition end, and run an artificial intelligence model on the edge computing devices to analyze the original video stream in real time to obtain the structured data corresponding to the original video stream; Step S2: Based on the preset first triggering strategy, the structured data is processed, and the processed data packet containing feature vectors and key video frames is uploaded to the cloud intelligent platform, which then performs a graph analysis on the data packet. Step S3: Obtain the analysis results corresponding to the map analysis, and make a decision based on the feedback information of the relevant edge computing devices. Generate the corresponding second trigger strategy and send it to all edge computing devices, and let all edge computing devices choose to execute it. Step S4: Integrate the analysis information of the original video streams corresponding to several edge computing devices to generate a corresponding global information set. Based on the global information set, process to obtain a global event map and a global event chain, and finally decide on the video acquisition terminal with anomalies.

[0018] It should be further explained that, in the specific implementation process, the process of deploying edge computing devices at the video acquisition end, and having these edge computing devices run large-scale artificial intelligence models to analyze the raw video stream in real time and obtain the structured data corresponding to the raw video stream includes: The video acquisition terminal is equipped with a number of video devices. The number of video devices is labeled and denoted as i, then i = 1, 2, 3, ..., n, where n is a natural number greater than 0. Each video device corresponds to a capture coordinate of the video acquisition terminal. A number of edge computing devices are deployed at the video acquisition end. The edge computing devices are used to obtain the model scripts corresponding to the configuration of the large artificial intelligence model. The execution steps related to video parsing recorded in the model scripts are processed in the edge computing devices in sequence to generate parsing nodes corresponding to each execution step. The parsing nodes are used to parse the raw video stream obtained by the edge computing device. Each parsing node is set with a corresponding parsing domain radius in the edge computing device. The parsing range corresponding to each parsing node is set by the parsing domain radius. Within the parsing scope, the parsing node performs real-time parsing of the corresponding part of the original video stream, thereby obtaining the structured data of the corresponding part of the entries. The retrieval index of each entry is set to the device serial number corresponding to the edge computing device. The structured data of all entries in each edge computing device is integrated to generate the structured data of a device corresponding to the edge computing device. The structured data of all devices in all edge computing devices are integrated to obtain the final structured data corresponding to the original video stream being processed.

[0019] It should be further explained that, in the specific implementation process, the process of processing structured data based on the preset first trigger strategy and uploading the processed data packet containing feature vectors and key video frames to the cloud intelligent platform includes: The first triggering strategy is used to extract data segments corresponding to the main information representing the original video stream from the structured data. The first triggering strategy is predefined based on the service type required by the cloud intelligent platform. The first triggering strategy is stored in the edge computing device in the form of a rule engine or configuration file. The first trigger strategy includes several categories of decision factors, including event type, confidence threshold, degree of scene change, periodic messages, and device resource status. The relevant explanations for different decision factors are as follows: Event Type: Such as detecting a preset sensitive target (person, vehicle), specific behavior (loitering, running), or area intrusion; Confidence Threshold: When the confidence of the detected target is lower than the lower limit of the preset threshold (e.g., <0.6), it is considered a "hard sample" and all structured data must be uploaded to the cloud intelligent platform for review. When the confidence is higher than the upper limit of the preset threshold (e.g., >0.95), only the metadata corresponding to the structured data is uploaded; Scene Change Degree: The original video stream is processed into several video frames, and several reference frames are set. The cosine distances corresponding to the feature vectors of the current video frame and the reference frames are compared. If the cosine distance exceeds a predefined distance threshold, it indicates that a significant change has occurred in the scene, and the corresponding video frame is used as the key. If the video frame is not displayed, it indicates that the scene has not changed and no operation is performed. Periodic messages: In the absence of event triggering, feature vectors obtained from structured data are uploaded once every preset time window. The uploaded feature vectors are used to perform global situational awareness at the location of the cloud intelligent platform. Device resource status: Edge computing devices are set with fixed upload capacity and corresponding maximum upload capacity. The CPU / bandwidth usage of each edge computing device is obtained. When it is determined that the device resources are idle, the upload capacity of the edge computing device is increased to the maximum upload capacity. This is used to actively upload more feature vectors and key video frames obtained from structured data processing in an edge computing device. Set the packet size and obtain the data size of the currently processed feature vectors and key video frames in real time. Whenever the data size reaches the packet size, the corresponding feature vectors and key video frames are processed into a data packet and uploaded to the cloud intelligent platform.

[0020] The data packet includes feature vectors and key video frames, which are filtered by a first triggering strategy. The specific filtering process is as follows: the edge computing device evaluates the structured data frame by frame based on the first triggering strategy, parses the structured data of the selected frame in the current data packet, extracts the detection results, feature vectors, and timestamps, and uses them as information digests. The information digests are matched with the rules of the first triggering strategy. If the match is successful, the currently selected frame is marked as "to be uploaded", and the data type combination to be uploaded is determined (e.g., only feature vectors + detection result metadata are uploaded, or key video frames are uploaded at the same time). The key video frames are compressed and encoded to reduce their size.

[0021] It should be further explained that, in the specific implementation process, the process of performing graph analysis on data packets by the cloud-based intelligent platform includes: The cloud-based intelligent platform has several domain knowledge graphs that represent different video semantic knowledge. Based on each domain knowledge graph, an inference point is created on the cloud-based intelligent platform. Each inference point creates a migration point at the location of the data packet, and a knowledge migration path is established between the inference point and the migration point. For each knowledge migration path, set corresponding knowledge matching information. The knowledge matching information is used to obtain the structured data in the data package that is adapted to the current knowledge migration path and to serve as graph retrieval information. The migration point will transfer the graph retrieval information to the inference point on the same knowledge migration path, and the inference point will adapt the corresponding domain knowledge graph based on the graph retrieval information. When all the structured data corresponding to a data packet has been adapted to the corresponding domain knowledge graph, it means that the cloud-based intelligent platform has completed the graph analysis of the current data packet, updated all the domain knowledge graphs adapted to the same data packet to a preset blank graph template, and then obtained a real-time event graph that integrates video semantic knowledge from different domains. The real-time event graph is updated based on each newly adapted domain knowledge graph, and the real-time event graph is used as the final analysis result.

[0022] The specific adaptation is as follows: The graph retrieval information includes device ID, timestamp, feature vector, detection result list, and key video frames. For each target (person, vehicle, object) in the detection result list, a high-precision model in the cloud (such as a Transformer-based object detector) is used for secondary confirmation, and corresponding attribute tags (such as vehicle color, model, license plate, clothing and items carried by the person) are extracted. The identified visual entities are then mapped to predefined entity types in the domain knowledge graph. For example, a visual "box" is mapped to the "suspicious item" category in the knowledge graph, and "person in uniform" is mapped to "security personnel" or "staff". The information obtained from the current data packet is associated with the static and dynamic information stored in the knowledge graph. Specifically, spatial association involves querying the geographical location, area (e.g., "East Gate of the Factory Area", "Warehouse A"), and area attributes (e.g., "Core Restricted Area", "Public Passage") of the camera in the knowledge graph based on the device ID. Temporal association involves combining timestamps to associate historical event records of the area (e.g., whether similar events occurred in the past hour) and the security level of the current time period (e.g., high vigilance period at night). Static attributes and dynamic states related to the current visual entity are retrieved. For example, the handling plans and historical cases associated with "suspicious items" in the knowledge graph of all domains are retrieved; vehicle registration information and blacklist status associated with "license plate number" are retrieved and used as contextual information for association. Based on the associated context information, the cloud-based intelligent platform calls a pre-deployed inference engine (built based on rules or graph neural networks) to perform semantic reasoning, transforming the original visual detection results into high-level semantic events.

[0023] Examples of inference rules include: **Intrusion into a Restricted Area:** If the visual entity is a "person," and its location is within a "restricted area" marked in the domain knowledge graph, and the person is not associated with "staff," then the inference is "illegal intrusion event." **Item Leaving:** If a "box" type target is detected in multiple consecutive frames at the same location, and there are no associated "personnel" nearby, and the area is marked as a "sensitive area" in the knowledge graph, then the inference is "suspicious item leaving event." **Abnormal Behavior:** If a "person's" trajectory frequently repeats itself within a short period, or remains stationary in a non-rest area for a long time, combined with the "normal behavior pattern" of the area in the domain knowledge graph, it can be inferred as "loitering" or "abnormal lingering." The inference process outputs an event triple: (entity, relation, attribute), for example: (target person A, located in, restricted area), (restricted area, associated risk level, high).

[0024] It should be further explained that, in the specific implementation process, the process of obtaining the analysis results corresponding to the graph analysis, making decisions based on the feedback information from relevant edge computing devices, generating the corresponding second trigger strategy and distributing it to all edge computing devices, and then having all edge computing devices select and execute the strategy includes: A timeline is established, which consists of several timeline nodes. The analysis result corresponding to the graph analysis is a real-time event graph, which includes several sub-event graphs corresponding to the timeline. Each sub-event graph is used to represent the behavioral event corresponding to a video frame. Among them, all the behavioral events corresponding to several time nodes on the time axis are arranged according to the time sequence of the time nodes, and then the analysis information representing the location of a video acquisition coordinate is constructed. The video details at the corresponding location are recorded through the analysis information. All the analysis information is packaged into corresponding real-time data packets. Based on the acquisition coordinates corresponding to each analysis information, a separate data transmission channel is established for each real-time data packet. The two ends of the data transmission channel are connected to edge computing devices and cloud intelligent platforms, respectively. The analysis information from the location of the cloud-based intelligent platform is distributed to the corresponding edge computing devices through their respective data distribution channels. The feedback information cached in each edge computing device is obtained synchronously. The feedback information at each edge computing device is compared with the analysis information to determine the device resolution status at the location of each edge computing device. The device resolution status includes a trusted status and an untrusted status; When the edge computing device is in a trusted state, it means that the large artificial intelligence model loaded and running on the edge computing device has achieved the expected results, and it can accurately analyze the raw video stream recorded by the video acquisition terminal at a certain acquisition coordinate. Conversely, when the edge computing device is in an untrusted state, the AI ​​model of the edge computing device does not achieve the expected results and cannot accurately complete the analysis of the corresponding original video stream. At this time, due to various internal or external factors, the edge computing device is unable to accurately use the AI ​​model to analyze the original video stream to the expected degree. All live data packets are used to represent all types of video behavior scenarios. All live data packets are processed on a cloud-based intelligent platform using an artificial intelligence model built on historical data to generate corresponding second trigger strategies. The second triggering strategy is a set of executable instructions that the cloud-based intelligent platform dynamically generates and distributes to the edge computing device based on in-depth analysis results and global status. It is a dynamic optimization and adjustment of the first triggering strategy inherent in the edge computing device, with the aim of achieving on-demand resource allocation and adaptive performance optimization. The second triggering strategy includes acquisition control parameters, trigger condition parameters, upload scheduling parameters, and model scheduling parameters. Among them, the acquisition control parameters are used to regulate the acquisition behavior of the video acquisition end corresponding to the edge computing device on the original video stream; specifically, they include: frame rate adjustment: specifying the target acquisition frame rate (e.g., increasing from 15fps to 30fps) to capture details of high-speed moving targets; resolution adjustment: specifying the resolution of the uploaded video (e.g., decreasing from 720p to 480p) to save bandwidth when the network is congested; encoding quality: specifying the quality parameters of video compression encoding (e.g., adjusting the CRF value from 23 to 28) to balance image quality and bandwidth; ROI region: specifying the region of interest, and the device only performs high-precision analysis on this region (e.g., performing face recognition only on a specific door in the image). Triggering condition parameters are used to dynamically adjust the rule thresholds in the first triggering strategy, including: event type priority: temporarily adding or adjusting the event types that need to be triggered for uploading, such as adding the "suspicious persons at night" type; confidence threshold: adjusting the confidence threshold for target detection or event recognition, such as lowering the trigger threshold for "illegally parked vehicles" from 0.8 to 0.6 to capture more suspected events; scene change sensitivity: adjusting the sensitivity parameter for scene change detection to determine when to upload keyframes, such as lowering the sensitivity for "light changes" to avoid a large number of invalid uploads caused by flickering streetlights at night; upload scheduling parameters are used to control the content, timing, and priority of data uploads, including: upload content type: specifying the data combination to be uploaded, such as "upload metadata only", "upload metadata + keyframes", "upload metadata + short video clips"; upload frequency: adjusting the interval of periodic reporting, such as from 5 minutes / The frequency is adjusted to once every 1 minute. Priority marking: Marks the upload priority for specific event types, such as marking "fire alarm" events as the highest priority, and uploading immediately when bandwidth is seized. Local caching strategy: Specifies the retention time or space limit of the local cache, such as "retaining event videos from the last 24 hours". Model scheduling parameters are used to control the large AI model running locally on the edge computing device, including: Model switching: Instructs the edge computing device to switch to a specially trained specific scene model, such as switching to the low-light enhancement model in the sub-scene of the large AI model during nighttime, and switching to the rain-free model in the sub-scene of the large AI model during rainy days; Model update: Carries new model parameters or differential update packages, instructing the edge computing device to load updates when idle; Inference precision mode: Adjusts the precision of model inference (such as switching from FP16 to INT8) to reduce power consumption when resources are scarce. The generated second triggering policy is distributed sequentially to all edge computing devices through the data distribution channel and cached on the edge computing devices. Edge computing devices whose device status is untrusted are selected as the target devices for executing the second triggering policy to update the devices. Edge computing devices in the trusted state continue to cache the second triggering policy, and after the trusted state changes to the untrusted state, the cached second triggering policy is used to execute the device update for the corresponding edge computing device. The updated edge computing devices generate their own latest analysis information based on their respective original video streams.

[0025] It should be further explained that, in the specific implementation process, the process of integrating the analysis information of the original video streams corresponding to several edge computing devices to generate a corresponding global information set, processing based on the global information set to obtain a global event graph and a global event chain, and finally deciding on the video acquisition terminal with anomalies includes: Initialize a blank layer covering the locations of all edge computing devices. On the blank layer, based on the coordinates of each edge computing device, create an information display point at the coordinates of each device. The information display point is used to store the analysis information corresponding to each edge computing device. Construct a summary display point, upload the analysis information corresponding to each of the information display points to the summary display point for storage, and the summary display point integrates all the analysis information to generate the corresponding global information set. The global information set is used to record different node types and edge types. Each edge computing device is treated as a corresponding node object. The mapping points of each edge computing device covered by the current global information set on the blank layer are constructed. Based on the attributes of the node type and the functional relationship of the edge type, all mapping points on the blank layer are connected to construct the corresponding global event graph. Several global event chains are obtained by filtering based on the global event graph. Each global event chain is used to trace the details of abnormal behavior at a certain acquisition coordinate of a video acquisition terminal. The global event graph serves as a summary of the details of abnormal behavior of all video acquisition terminals corresponding to all acquisition coordinates. The node types include target nodes, event nodes, device nodes, and region nodes. The attributes corresponding to each node type are described as follows: Target Node: Each tracked physical entity (person, vehicle, object) is a node, with attributes including target ID, category, global trajectory, and confidence level; Event Nodes: Each video semantic event, such as "intrusion" or "left behind," is treated as a node. Attributes include event type, occurrence time, confidence level, and associated target ID. Device Node: Each edge device is a node, and its attributes include device ID, geographic location, and current status, which includes online, offline, and load. Region Node: A predefined logical region is treated as a node, such as a gate or a warehouse. Attributes include region type and risk level.

[0026] The functional relationships corresponding to the edge types include: Occurrence, Trigger, Temporal Follow, and Causal Association. The detailed descriptions of each functional relationship are as follows: Occurrence: used to connect the target node and the device / region node, indicating that the target appears in the field of view or region of a certain device at a certain time; Trigger: used to connect the target node and the event node, indicating that the target triggered the event; Temporal Follow: used to connect event nodes under different time sequences, indicating the chronological order of events; Causal Association: connects different event nodes based on the graph rules of the knowledge graph, representing the logical causal relationship between different times.

[0027] Based on all the details of abnormal behavior, the location coordinates of all video capture terminals with abnormal behavior are marked, and then a decision is made to locate the specific location area of ​​the video capture terminal with abnormal behavior.

[0028] Example 2, please refer to Figure 2 As shown in the figure, this embodiment also provides an artificial intelligence-based smart cloud video analysis system, which includes: The edge computing module is used to deploy edge computing devices at the video acquisition end. The edge computing devices run a large artificial intelligence model to analyze the raw video stream in real time and obtain the structured data corresponding to the raw video stream. The first processing module processes the structured data based on a preset first triggering strategy, and uploads the processed data packet containing feature vectors and key video frames to the cloud intelligent platform, whereby the cloud intelligent platform performs a graph analysis of the data packet. The second processing module is used to obtain the analysis results corresponding to the map analysis, and make decisions based on the feedback information of the relevant edge computing devices, generate the corresponding second trigger strategy and send it to all edge computing devices, and all edge computing devices select to execute it. The analysis and decision-making module is used to integrate the analysis information of the original video streams corresponding to several edge computing devices, generate the corresponding global information set, process the global information set to obtain the global event map and global event chain, and finally decide on the video acquisition terminal with anomalies.

[0029] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A smart cloud video analysis method based on artificial intelligence, characterized in that, Includes the following steps: Step S1: Deploy edge computing devices at the video acquisition end, and run an artificial intelligence model on the edge computing devices to analyze the original video stream in real time to obtain the structured data corresponding to the original video stream; Step S2: Based on the preset first triggering strategy, the structured data is processed, and the processed data packet containing feature vectors and key video frames is uploaded to the cloud intelligent platform, which then performs a graph analysis on the data packet. Step S3: Obtain the analysis results corresponding to the map analysis, and make a decision based on the feedback information of the relevant edge computing devices. Generate the corresponding second trigger strategy and send it to all edge computing devices, and let all edge computing devices choose to execute it. Step S4: Integrate the analysis information of the original video streams corresponding to several edge computing devices to generate a corresponding global information set. Based on the global information set, process to obtain a global event map and a global event chain, and finally decide on the video acquisition terminal with anomalies.

2. The intelligent cloud video analysis method based on artificial intelligence according to claim 1, characterized in that, The process of deploying edge computing devices at the video acquisition end, and having these devices run large-scale artificial intelligence models to analyze the raw video stream in real time to obtain the structured data corresponding to the raw video stream includes: The video acquisition terminal is equipped with a number of video devices, each video device corresponding to a capture coordinate. A number of edge computing devices are deployed at the video acquisition terminal to obtain the model scripts corresponding to the configuration of the large artificial intelligence model. The execution steps related to video parsing recorded in the model script are processed in the edge computing device to generate a parsing node for each execution step. The parsing node is used to parse the raw video stream of the edge computing device to obtain the structured data of the corresponding entries. By integrating the structured data of all entries in each edge computing device, a device-specific structured data is generated for that edge computing device. By integrating the structured data of all devices, the final structured data of the raw video stream being processed is obtained.

3. The intelligent cloud video analysis method based on artificial intelligence according to claim 2, characterized in that, The process of processing structured data based on a preset first trigger strategy and uploading the processed data packet containing feature vectors and key video frames to the cloud intelligent platform includes: The first triggering strategy is used to extract data segments corresponding to the main information representing the original video stream from the structured data. The first triggering strategy is predefined based on the service type required by the cloud intelligent platform. The first triggering strategy is stored in the edge computing device in the form of a rule engine or configuration file. Set the packet size and obtain the data size of the currently processed feature vectors and key video frames in real time. Whenever the data size reaches the packet size, the corresponding feature vectors and key video frames are processed into a data packet and uploaded to the cloud intelligent platform. The feature vectors and key video frames included in the data packet are filtered by the first trigger strategy.

4. The intelligent cloud video analysis method based on artificial intelligence according to claim 3, characterized in that, The process of performing graph analysis on data packets by a cloud-based intelligent platform includes: The cloud-based intelligent platform has several domain knowledge graphs that represent different video semantic knowledge. Based on each domain knowledge graph, an inference point is created on the cloud-based intelligent platform. Each inference point creates a migration point at the location of the data packet, and a knowledge migration path is established between the inference point and the migration point. For each knowledge migration path, set corresponding knowledge matching information. The knowledge matching information is used to obtain the structured data in the data package that is adapted to the current knowledge migration path and to serve as graph retrieval information. The migration point will transfer the graph retrieval information to the inference point on the same knowledge migration path, and the inference point will adapt the corresponding domain knowledge graph based on the graph retrieval information. When all the structured data corresponding to a data packet has been adapted to the corresponding domain knowledge graph, it means that the cloud-based intelligent platform has completed the graph analysis of the current data packet, updated all the domain knowledge graphs adapted to the same data packet to a preset blank graph template, and then obtained a real-time event graph that integrates video semantic knowledge from different domains. The real-time event graph is updated based on each newly adapted domain knowledge graph, and the real-time event graph is used as the final analysis result.

5. The intelligent cloud video analysis method based on artificial intelligence according to claim 4, characterized in that, The process of obtaining the analysis results corresponding to the map analysis, making decisions based on feedback information from relevant edge computing devices, generating a corresponding second trigger strategy, distributing it to all edge computing devices, and having all edge computing devices select and execute the strategy includes: A timeline is established, which consists of several timeline nodes. The real-time event graph includes several timeline sub-event graphs. All behavioral events of several timeline nodes on the timeline are arranged based on the timeline correspondence of the timeline nodes to construct analytical information representing the location of a collection coordinate. All the analysis information is packaged into corresponding real-time data packets. Based on the acquisition coordinates corresponding to each analysis information, a separate data transmission channel is established for each real-time data packet. The analysis information is then transmitted to the respective affiliated edge computing devices through the respective data transmission channels. The feedback information cached in each edge computing device is obtained synchronously. The feedback information is compared with the analysis information to determine the device resolution status of each edge computing device. The generated second triggering strategy is sequentially sent to all edge computing devices through the data delivery channel and cached on the edge computing devices. Edge computing devices whose status is resolved to be untrusted are selected as the target devices for executing the second triggering strategy to update the devices. After the devices are updated, the edge computing devices generate their latest analysis information based on their original video streams.

6. The intelligent cloud video analysis method based on artificial intelligence according to claim 5, characterized in that, The process of integrating the analysis information from the raw video streams corresponding to several edge computing devices to generate a corresponding global information set includes: Initialize a blank layer covering the locations of all edge computing devices. On the blank layer, based on the coordinates of each edge computing device, create an information display point at the coordinates of each device. The information display point is used to store the analysis information of each edge computing device. A summary display point is constructed, and the analysis information of all information display points is uploaded to the summary display point. The summary display point integrates all the analysis information to generate the corresponding global information set. The global information set is used to record different node types and edge types.

7. The intelligent cloud video analysis method based on artificial intelligence according to claim 6, characterized in that, The process of obtaining a global event graph and global event chain based on global information set processing, and then ultimately determining the video capture terminal with anomalies, includes: Treat each edge computing device as a node object, construct the mapping points of each edge computing device on the blank layer, connect all the mapping points on the blank layer based on the attributes of the node type and the functional relationship of the edge type, and construct the global event graph. Several global event chains are obtained by filtering based on the global event graph. Each global event chain is used to trace the abnormal behavior details of a video capture terminal at a certain capture coordinate. Based on all the abnormal behavior details, the capture coordinate location corresponding to the video capture terminal with the abnormality is marked.

8. An artificial intelligence-based intelligent cloud video analysis system, used to implement the intelligent cloud video analysis method according to any one of claims 1 to 7, characterized in that, The system includes: The edge computing module is used to deploy edge computing devices at the video acquisition end. The edge computing devices run a large artificial intelligence model to analyze the raw video stream in real time and obtain the structured data corresponding to the raw video stream. The first processing module processes the structured data based on a preset first triggering strategy, and uploads the processed data packet containing feature vectors and key video frames to the cloud intelligent platform, whereby the cloud intelligent platform performs a graph analysis of the data packet. The second processing module is used to obtain the analysis results corresponding to the map analysis, and make decisions based on the feedback information of the relevant edge computing devices, generate the corresponding second trigger strategy and send it to all edge computing devices, and all edge computing devices select to execute it. The analysis and decision-making module is used to integrate the analysis information of the original video streams corresponding to several edge computing devices, generate the corresponding global information set, process the global information set to obtain the global event map and global event chain, and finally decide on the video acquisition terminal with anomalies.

Images