Offline video analysis method and system

Abstract

The invention discloses an off-line video analysis method and system, and the method comprises the steps: detecting whether an off-line video has a scene abrupt change point, if yes, segmenting the off-line video according to the scene abrupt change point, obtaining a plurality of video segments, and if not, taking the off-line video as a video segment; performing scene matching on each video clip based on a preset scene library, and determining a target scene to which each video clip belongs; based on the accuracy rate, the processing speed and the resource occupancy rate, calculating the adaptive score of each detection algorithm in a preset algorithm set corresponding to the target scene, and selecting the detection algorithm with the highest adaptive score as the optimal detection algorithm of the target scene to which each video clip belongs; and detecting the corresponding video clip based on the optimal detection algorithm of each video clip, and outputting a detection result of the offline video. According to the method and the device, automatic analysis of the offline video is realized, manual intervention is reduced, and the offline video analysis efficiency is improved.

Description

An offline video parsing method and system Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an offline video parsing method and system. Background Technology

[0002] With the increasing application of AI and image processing technologies in public safety, urban governance, and other fields, the demand for video detection has expanded from real-time monitoring to offline recording and playback. However, existing technical solutions have the following shortcomings:

[0003] 1. Reliance on fixed hardware and network: Existing systems require binding to fixed-location cameras, access to dedicated networks, and pre-configuration of platforms and algorithm deployment. This makes it impossible to cover the parsing needs of scenarios without fixed equipment (such as temporary emergency scenes or remote areas without network) and mobile recordings (such as videos recorded by law enforcement officers' mobile phones).

[0004] 2. Cumbersome and inefficient operation: Real-time video analysis requires equipment procurement, network deployment, algorithm application, task control and other steps, which takes several days to several weeks; for video clips that are reviewed afterward, algorithms need to be selected manually and processed in segments, which cannot quickly output detection results and is difficult to meet the needs of emergency decision-making.

[0005] 3. Poor scene and algorithm adaptability: Existing systems cannot automatically match the optimal detection algorithm based on video content (e.g., road scenes require vehicle detection algorithms, and public places require crowd gathering algorithms). It is necessary to manually determine the scene type and switch algorithms, which can easily lead to a decrease in detection accuracy due to adaptation errors.

[0006] In summary, existing technologies cannot meet the demands for offline, efficient, and automatically adaptable video parsing. There is an urgent need for an AI parsing solution that is compatible with multiple devices, supports offline processing, and can automatically match algorithms. Summary of the Invention

[0007] This application provides an offline video parsing method and system for automatically parsing offline videos, reducing manual intervention and improving the efficiency of offline video parsing.

[0008] In view of this, the first aspect of this application provides an offline video parsing method, including:

[0009] If there are scene change points in the offline video, the offline video is segmented according to the scene change points to obtain several video segments; if there are no scene change points, the offline video is treated as a single video segment.

[0010] Based on a pre-built scene library, scene matching is performed on each video segment to determine the target scene to which each video segment belongs;

[0011] Based on accuracy, processing speed and resource utilization, the adaptation score of each detection algorithm in the preset algorithm set corresponding to the target scene is calculated, and the detection algorithm with the highest adaptation score is selected as the optimal detection algorithm for the target scene to which each video segment belongs.

[0012] The corresponding video segments are detected using the optimal detection algorithm for each video segment, and the detection results of the offline video are output.

[0013] Optionally, detecting whether there are scene abrupt changes in the offline video includes:

[0014] Calculate the similarity between adjacent frames in the offline video. If there is a similarity lower than the first similarity threshold, then determine the next frame in the adjacent frames with a similarity lower than the first similarity threshold as a scene change point.

[0015] Optionally, before detecting whether there are scene abrupt changes in the offline video, the method further includes:

[0016] The frame extraction frequency is calculated based on the duration of the offline video, the scene complexity coefficient of the first frame, and the target detection accuracy threshold.

[0017] The offline video is frame-sampling according to the stated frame-sampling frequency.

[0018] Optionally, the step of performing scene matching on each video segment based on a pre-set scene library to determine the target scene to which each video segment belongs includes:

[0019] For each video segment, calculate the similarity between the video segment and each scene in the pre-set scene library;

[0020] If the maximum similarity exceeds the second similarity threshold, the scene corresponding to the maximum similarity is selected as the target scene to which the video segment belongs.

[0021] If the maximum similarity score does not exceed the second similarity threshold, the user is prompted to manually confirm the target scene to which the video clip belongs.

[0022] Optionally, the step of calculating the adaptation score of each detection algorithm in the preset algorithm set corresponding to the target scene based on accuracy, processing speed, and resource utilization includes:

[0023] By using preset weighting coefficients, the accuracy, processing speed, and resource utilization of each detection algorithm in the preset algorithm set corresponding to the target scene are weighted and summed in the target scene to obtain the adaptation score of each detection algorithm in the preset algorithm set corresponding to the target scene.

[0024] Optionally, the optimal detection algorithm based on each video segment detects the corresponding video segment and outputs the detection results of the offline video, including:

[0025] The video segments are split into segments based on the optimal processing segment length of each detection algorithm to obtain a segment set;

[0026] The corresponding optimal detection algorithm is used to detect targets in each video segment in the segment set, and the detection results of the offline video are output.

[0027] Optionally, the method further includes:

[0028] If an abnormal target is found in the detection results, an alarm will be generated.

[0029] The second aspect of this application provides an offline video parsing method, including:

[0030] Offline videos are matched to scenes based on a pre-built scene library to determine the target scene to which the offline video belongs.

[0031] Based on accuracy, processing speed and resource utilization, the adaptation score of each detection algorithm in the preset algorithm set corresponding to the target scene is calculated, and the detection algorithm with the highest adaptation score is selected as the optimal detection algorithm for the target scene to which the offline video belongs.

[0032] The offline video is checked for scene abrupt changes. If such changes exist, the offline video is segmented into several video segments based on the scene abrupt changes. The optimal detection algorithm for each video segment is determined. The corresponding video segments are then detected based on the optimal detection algorithm for each video segment, and the detection result of the offline video is output. If no such changes exist, the offline video is then detected based on the optimal detection algorithm corresponding to the offline video, and the detection result of the offline video is output.

[0033] A third aspect of this application provides an offline video parsing system, comprising:

[0034] The customer display module is used to upload offline videos and display test results;

[0035] The analysis algorithm module is used to perform scene matching, algorithm matching, scene change point detection, and target detection on the offline video.

[0036] Optionally, the system may also include:

[0037] The offline video processing module is used to analyze the duration and first frame scene features of the offline video, calculate the optimal frame extraction frequency, and extract frames from the offline video based on the optimal frame extraction frequency.

[0038] The platform service module is used for distributed scheduling of algorithm resources and structured storage of detection results and alarm data, supporting historical queries and secondary analysis.

[0039] As can be seen from the above technical solutions, this application has the following advantages:

[0040] This application provides an offline video parsing method that automates the entire process of scene matching and algorithm adaptation, enabling offline video detection without manual intervention and solving the problems of existing technologies that rely on fixed equipment and are cumbersome to operate.

[0041] Furthermore, the frame sampling frequency is dynamically calculated based on video duration and content complexity, and multi-dimensional scores are combined to achieve accurate matching between the scene and the algorithm, balancing detection accuracy and resource efficiency. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 is a flowchart illustrating an offline video parsing method provided in an embodiment of this application;

[0044] Figure 2 is another flowchart illustrating an offline video parsing method provided in an embodiment of this application;

[0045] Figure 3 is a schematic diagram of an offline video parsing system provided in an embodiment of this application;

[0046] Figure 4 is a flowchart of an offline video parsing system provided in an embodiment of this application. Detailed Implementation

[0047] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0048] For ease of understanding, please refer to Figure 1. This embodiment of the application provides an offline video parsing method, including:

[0049] Step 110: Detect whether there are scene change points in the offline video. If there are, cut the offline video according to the scene change points to obtain several video segments. If there are no, treat the offline video as a single video segment.

[0050] After a user uploads an offline video file (such as MP4) via the web / H5 client, the system automatically verifies the format and stores it on a distributed storage node, generating a unique Video ID (VID) for subsequent tracking. The similarity between adjacent frames in the offline video is calculated. If a similarity score is lower than a first similarity threshold (e.g., 0.5), the next frame in that adjacent frame is identified as a scene abrupt change point. If a scene abrupt change point is detected, the consecutive video frames between the first frame and the first scene abrupt change point are segmented into the first video segment; the consecutive video frames between the first and second scene abrupt change points are segmented into the second video segment, and so on, until the last scene abrupt change point and the last frame of the offline video are segmented into the last video segment. If no scene abrupt change point is detected, it indicates that the offline video represents the same scene, and the entire offline video is treated as a single video segment without further segmentation.

[0051] Furthermore, in order to improve the efficiency of video data processing without affecting the accuracy of detection results, this embodiment of the application preprocesses and extracts frames from the offline video before detecting whether there are scene abrupt changes after the user uploads the offline video. First, the core parameters of the offline video are analyzed: let the video duration be T (unit: seconds), the scene complexity coefficient of the first frame be C (C∈[1,5], dynamic scene C∈[3,5], static scene C∈[1,3], which can be identified by the model as a dynamic or static scene), and the target detection accuracy threshold be P (P∈[0.8,1.0], which can be customized by the user or the system default value of 0.9). Then, the frame extraction frequency is calculated based on the duration of the offline video, the scene complexity coefficient of the first frame, and the target detection accuracy threshold. The frame extraction frequency f (unit: frames / second) needs to balance detection accuracy and resource consumption. The formula for calculating the frame extraction frequency is as follows:

[0052]

[0053] Where α is the correction coefficient (α∈[0.5,2.0], based on historical data calibration, default value 1.2); when T≤60s (short video), f≥2 (to ensure frame density); when T>300s (long video), f≤1 (to reduce resource consumption).

[0054] Frames are extracted from the offline video based on the frame extraction frequency f to generate a set of scene graphs. The total number of scene graphs N = round(f × T) + 2. Where s1 is the first frame (time point 0), s... N For the last frame (time point T), s2~s N-1The intermediate frames are extracted at a frequency f (e.g., 1 frame is extracted every 1 second when f=1). The scene graphs are uniformly converted to 1080P resolution and RGB format to facilitate subsequent feature extraction. After obtaining the scene graph set S, the similarity between adjacent frames in the scene graph set S is calculated to detect scene abrupt changes. Then, the scene graph set S is segmented to obtain several video segments S', each of which consists of several image frames s.

[0055] Step 120: Based on a pre-set scene library, perform scene matching on each video segment to determine the target scene to which each video segment belongs;

[0056] For each video segment, calculate the similarity between the video segment and each scene in the pre-set scene library; if the maximum similarity exceeds the second similarity threshold (e.g., 0.6), select the scene corresponding to the maximum similarity as the target scene to which the video segment belongs; if the maximum similarity does not exceed the second similarity threshold, prompt the user to manually confirm the target scene to which the video segment belongs.

[0057] Pre-built scene library Sc = {Sc1, Sc2, ..., Sc M (M≥10, e.g., Sc1 is a road scene, Sc2 is a public place scene), the similarity between each video segment and each scene is calculated using a feature matching algorithm. Since the aforementioned process detects scene abrupt changes by calculating the similarity between adjacent frames, meaning that image frames within the same video segment represent the same scene, the similarity between the first frame of each video segment and each scene (Sc1) can be used. j The similarity is used as the similarity between each video segment and each scene, such as the first frame s1 of the first video segment and each scene Sc. j The similarity is Sim(s1, Sc). j (Using the cosine similarity algorithm, Sim∈[0,1]), the optimal scene (the target scene to which the video clip belongs) is the scene with the highest similarity. If the maximum similarity is less than 0.6 (low scene matching degree), the system automatically calls the general scene algorithm and prompts the user to manually confirm the scene type.

[0058] Step 130: Calculate the adaptation score of each detection algorithm in the preset algorithm set corresponding to the target scene based on accuracy, processing speed and resource utilization, and select the detection algorithm with the highest adaptation score as the optimal detection algorithm for the target scene to which each video segment belongs.

[0059] Each scenario corresponds to a set of pre-defined algorithms, such as scenario Sc. j Corresponding pre-set algorithm set A1 j = {Al j1}, Al j2 ,..., Al jK (K≥2, such as Sc1 corresponding to Al) 11=Vehicle detection algorithm, AI 12 =License plate recognition algorithm), which selects the optimal detection algorithm through multi-dimensional scoring. Detection algorithm A1 jk The fit score (AI) jk Taking into account accuracy, processing speed, and resource utilization:

[0060]

[0061] Among them, Acc(Al) jk )∈[0,1] represents the algorithm's detection accuracy (e.g., Acc=0.95 for vehicle detection algorithm), Speed(Al jk )∈[0,1] represents the processing speed (e.g., 10 frames / second corresponds to Speed=0.8), Rate(Al) jk The range [0,1] represents resource utilization (e.g., 20% CPU usage corresponds to Rate=0.2); weight coefficients w1=0.5, w2=0.3, w3=0.2 (can be adjusted according to user needs). Finally, the detection algorithm with the highest adaptation score from the preset algorithm set corresponding to the target scene of each video segment is selected as the optimal detection algorithm Al for the target scene of each video segment. opt .

[0062] Step 140: Detect the corresponding video segments based on the optimal detection algorithm for each video segment, and output the detection results of the offline video.

[0063] Based on the optimal processing segment length of each detection algorithm, the video segments are split by length to obtain a segment set. Let a certain video segment S... r The duration of the video clip is t, and the video clip S is... r The corresponding optimal detection algorithm is A1. opt,r If the optimal processing segment length is L (e.g., L=30s, which can be adjusted according to algorithm characteristics), then the number of segments G r =ceil(t / L), fragment set V r ={v r1 ,v r2 ,...,v rG},v G =[(G-1)L,min(GL,t)]. The entire offline video is split into a set of segments {V1,V2,...,V...}. r ,...,V R}, where R is the number of scene mutation points plus one.

[0064] Each video clip V r The corresponding optimal detection algorithm A1 opt,rThe system parses and outputs structured detection results; if an abnormal target is detected (e.g., Sim≥P), an alarm is generated. (Video clip V) r The test results Res r Res contains key information about all objectives r ={Res r1 Res r2 ,...,Res rQ}, where Q is the video segment V r Number of targets detected in the middle, Res of a single target ri The structure is as follows:

[0065] Res ri ={Type ri ,Loc ri Sim ri Time ri}

[0066] Among them, Type ri For target type (e.g., "vehicles", "gathering of people"); Loc ri For the target location (e.g., pixel coordinates (x1, y1, x2, y2)); Sim ri For target similarity (∈[0,1], Sim≥P is considered a valid target); Time ri For detecting time (relative time within a video segment, such as V) r (Time=15s corresponds to a total video duration of 15s).

[0067] The system aggregates the full video detection results (Res) according to event type (such as "vehicle violation" or "fire hazard") and generates an alarm list (Alarm={Alarm1, ..., Alarm...)). E (E represents the number of alarms, each alarm...) e (Associate the detection results (Res) with similar targets); push the Alarm and Res to the platform service module storage via subscription address callback.

[0068] The offline video parsing method provided in this application solves the problems of existing technologies relying on fixed equipment and networks and being unable to process offline videos. It realizes automatic parsing of offline videos, covering detection needs in scenarios without fixed hardware / network and mobile recording; it achieves automatic scene-algorithm matching without manual intervention, reducing the user's operating threshold and shortening the offline video parsing cycle from several hours to minutes; it builds a multi-terminal compatible parsing ecosystem, supporting seamless access from web and H5 mobile terminals, meeting users' needs to upload, parse, and view results anytime, anywhere; and it realizes intelligent collection of detection results and alarm output, improving the readability of results and providing accurate data support for subsequent decision-making.

[0069] Please refer to Figure 2. This application also provides an offline video parsing method, including:

[0070] Step 210: Perform scene matching on the offline video based on the pre-set scene library to determine the target scene to which the offline video belongs;

[0071] After a user uploads an offline video file (such as MP4) via the web / H5 client, the system automatically verifies the format and stores it on a distributed storage node, generating a unique Video ID (VID) for subsequent tracking. Then, it calculates the average similarity between the offline video and each scene in a pre-built scene library. If the maximum average similarity exceeds a second similarity threshold (e.g., 0.6), the scene corresponding to the maximum average similarity is selected as the target scene to which the offline video belongs. If the maximum average similarity does not exceed the second similarity threshold, the user is prompted to manually confirm the target scene to which the video segment belongs.

[0072] Furthermore, in order to improve the efficiency of video data processing without affecting the accuracy of detection results, this embodiment of the application performs frame extraction processing on the offline video after the user uploads it, before scene matching. First, the core parameters of the offline video are analyzed: let the video duration be T (unit: seconds), the scene complexity coefficient of the first frame be C (C∈[1,5], dynamic scene C∈[3,5], static scene C∈[1,3], which can be identified by the model as a dynamic or static scene), and the target detection accuracy threshold be P (P∈[0.8,1.0], which can be customized by the user or the system default value of 0.9). Then, the frame extraction frequency is calculated based on the duration of the offline video, the scene complexity coefficient of the first frame, and the target detection accuracy threshold. The frame extraction frequency f (unit: frames / second) needs to balance detection accuracy and resource consumption. The formula for calculating the frame extraction frequency is as follows:

[0073]

[0074] Where α is the correction coefficient (α∈[0.5,2.0], based on historical data calibration, default value 1.2); when T≤60s (short video), f≥2 (to ensure frame density); when T>300s (long video), f≤1 (to reduce resource consumption).

[0075] Frames are extracted from the offline video based on the frame extraction frequency to generate a set of scene graphs. The total number of scene graphs N = round(f × T) + 2. Where s1 is the first frame (time point 0), s... N For the last frame (time point T), s2~s N-1 The intermediate frames are extracted at a frequency f (e.g., when f=1, 1 frame is extracted every 1 second); the scene images are uniformly converted to 1080P resolution and RGB format to facilitate subsequent feature extraction.

[0076] Pre-built scene library Sc = {Sc1, Sc2, ..., Sc M (M≥10, e.g., Sc1 is a road scene, Sc2 is a public place scene), the average similarity between the offline video and each scene is calculated using a feature matching algorithm. Let s be a single frame. i With Scene Sc j The similarity is Sim(s) i , Sc j (Using the cosine similarity algorithm, Sim∈[0,1]), then the scene graph set S and the scene Sc j The average similarity is:

[0077]

[0078] The target scene Sc to which the scene graph set S belongs opt The scene with the highest average similarity. If (If the scene matching degree is low), the system will automatically call the general scene algorithm and prompt the user to manually confirm the scene type.

[0079] Step 220: Calculate the adaptation score of each detection algorithm in the preset algorithm set corresponding to the target scene based on accuracy, processing speed and resource utilization, and select the detection algorithm with the highest adaptation score as the optimal detection algorithm for the target scene to which the offline video belongs;

[0080] Each scenario corresponds to a set of pre-defined algorithms, such as scenario Sc. j Corresponding pre-set algorithm set A1 j = {Al j1}, Al j2 ,..., Al jK (K≥2, such as Sc1 corresponding to Al) 11 =Vehicle detection algorithm, AI 12 =License plate recognition algorithm), which selects the optimal detection algorithm through multi-dimensional scoring. Detection algorithm A1 jk The fit score (AI) jk Taking into account accuracy, processing speed, and resource utilization:

[0081]

[0082] Among them, Acc(Al) jk )∈[0,1] represents the algorithm's detection accuracy (e.g., Acc=0.95 for vehicle detection algorithm), Speed(Al jk )∈[0,1] represents the processing speed (e.g., 10 frames / second corresponds to Speed=0.8), Rate(Al)jk The range [0,1] represents resource utilization (e.g., 20% CPU usage corresponds to Rate=0.2); weighting coefficients w1=0.5, w2=0.3, w3=0.2 (can be adjusted according to user needs). The final target scenario Sc is selected. opt The detection algorithm with the highest adaptation score in the corresponding pre-set algorithm set is used as the target scene Sc opt The optimal detection algorithm A1 opt .

[0083] Step 230: Detect whether there are scene change points in the offline video. If so, cut the offline video according to the scene change points to obtain several video segments, and determine the optimal detection algorithm for each video segment. Detect the corresponding video segments based on the optimal detection algorithm for each video segment, and output the detection results of the offline video. If not, detect the offline video based on the optimal detection algorithm corresponding to the offline video, and output the detection results of the offline video.

[0084] If the scene changes in the offline video (e.g., the first 30 seconds are a road, the next 20 seconds are a public place), the system detects the scene change point t through frame similarity. s The video is segmented, generating a set of segments corresponding to multiple algorithms. After the system segments the video based on scene abrupt changes, the scene matching-algorithm selection process needs to be re-executed for the new segments after the abrupt changes to ensure that each segment is adapted to the optimal detection algorithm. The specific process is as follows:

[0085] (1) Detecting scene abrupt changes: This can be achieved by calculating the relationship between consecutive frames in the scene graph set S and the current optimal scene Sc. opt Similarity to identify the first scene mutation point t s (Sim(t) s , Sc opt (The first time point when <0.5)

[0086] (2) Taking the scene change point t as an example s Using [0, t] as the boundary, divide the scene graph set S into two video segments V1=[0, t] s ] and V2=[t s ,T];

[0087] (3) For the current optimal scenario Sc opt For video clips V1 with high similarity, the original optimal detection algorithm Al is used. opt For video segment V2 after the scene change, extract its first frame (the first frame after the scene abrupt change point), recalculate the average similarity between this first frame and the pre-set scene library (since a single frame can represent the initial scene of the new segment, it is simplified to single-frame similarity), and select the new scene Sc. opt′ And based on the new scenario Sc opt′The optimal detection algorithm A1 for re-matching this segment opt ′.

[0088] (4) Repeat the above process until all frames in the scene graph set S are traversed, generating a set of segments corresponding to multiple algorithms. Each video segment is parsed independently by the corresponding algorithm to ensure that the detection accuracy does not decrease after scene switching.

[0089] Let the video duration be t, and the optimal detection algorithm be Al. opt If the optimal processing fragment length is L, then the number of fragments G = ceil(t / L) and the fragment set V = {V1, V2, ..., V} G},v G =[(G-1)L,min(GL,t)]. If there exists a scene mutation point t. s Then the set of fragments is V={V1([0,t s ]),V2([t s ,T])}, where V1 corresponds to the optimal detection algorithm Al of Sc1. opt V2 corresponds to the optimal detection algorithm Al for Sc2. opt For segments V1 and V2, length optimization is performed based on the optimal processing segment length of their respective optimal detection algorithms, and the segments are then merged to generate a segment set V.

[0090] If a video contains multiple scene abrupt changes, it needs to be split into multiple consecutive segments based on these abrupt changes. Each segment is then bound to its optimal detection algorithm and parsed independently. Segment division must adhere to the aforementioned principle of "abrupt changes as boundaries, segment length adapted to algorithm processing characteristics," generating a final set of segments. Each video segment is parsed by its corresponding optimal detection algorithm; the specific parsing process is consistent with step 140 above and will not be elaborated upon here.

[0091] The offline video parsing method provided in this application solves the problems of existing technologies relying on fixed equipment and networks and being unable to process offline videos. It realizes automatic parsing of offline videos, covering detection needs in scenarios without fixed hardware / network and mobile recording; it achieves automatic scene-algorithm matching without manual intervention, reducing the user's operating threshold and shortening the offline video parsing cycle from several hours to minutes; it builds a multi-terminal compatible parsing ecosystem, supporting seamless access from web and H5 mobile terminals, meeting users' needs to upload, parse, and view results anytime, anywhere; and it realizes intelligent collection of detection results and alarm output, improving the readability of results and providing accurate data support for subsequent decision-making.

[0092] This application also provides an offline video parsing system, including:

[0093] The customer display module is used to upload offline videos and display test results;

[0094] The analysis algorithm module is used to perform scene matching, algorithm matching, scene change point detection, and target detection on the offline video.

[0095] As a further improvement, the system also includes:

[0096] The offline video processing module is used to analyze the duration and first frame scene features of the offline video, calculate the optimal frame extraction frequency, and extract frames from the offline video based on the optimal frame extraction frequency.

[0097] The platform service module is used for distributed scheduling of algorithm resources and structured storage of detection results and alarm data, supporting historical queries and secondary analysis.

[0098] Please refer to Figures 3 and 4. In this embodiment, the customer display module provides a web / H5 visual interface and supports video uploading (including progress display and resume upload). The customer display module displays detection results (target type, location, similarity), historical records, and alarm lists. The customer display module supports result export (such as Excel / PDF format) and alarm message push (such as SMS / internal message).

[0099] The offline video processing module is compatible with mainstream video formats (MP4 / AVI / MOV, etc.), supports format verification and anomaly repair (such as frame break repair), and is also used for video preprocessing, including analyzing duration, first frame scene features, calculating the optimal frame extraction frequency, and extracting scene graphs according to the frame extraction frequency to generate a structured frame set.

[0100] The analysis algorithm module stores a pre-set scene library (10+ types of scenes such as roads, public places, fire hazards, and violations) and an algorithm set (20+ types of algorithms such as object detection, behavior analysis, and anomaly recognition), enabling automatic matching of "scene-algorithm", segmenting and parsing video segments according to algorithm characteristics; generating alarms according to event type and pushing them to the platform service module.

[0101] The platform service module uses a distributed scheduling algorithm to handle high-concurrency requests; it connects the entire process of "upload-preprocessing-parsing-display" and supports custom process nodes; it stores parsing results (JSON format) and alarm data in a structured manner, supporting historical queries and secondary analysis.

[0102] This application automates the entire process of "preprocessing - frame extraction - scene matching - algorithm adaptation," enabling offline video detection without manual intervention. This solves the problems of existing technologies relying on fixed equipment and cumbersome operations. It dynamically calculates the frame extraction frequency based on video duration and content complexity, combining multi-dimensional scores to achieve accurate matching between scenes and algorithms, balancing detection accuracy and resource efficiency. Adopting a modular design of "customer display - offline processing - algorithm analysis - platform services," it supports seamless access to web and H5 mobile terminals, and distributed resource scheduling to handle high-concurrency requests. Detection results are integrated according to event type to generate structured alarms, improving readability and decision-making efficiency.

[0103] The offline video parsing system provided in this application has the following advantages:

[0104] 1. Offline processing capability: Supports multi-format offline video parsing, requiring no fixed device / network, covering 100% of offline scenarios, with a video format compatibility rate of ≥98%;

[0105] 2. Operational efficiency: Fully automated process, shortening cycle and improving efficiency;

[0106] 3. Detection accuracy: Scene-algorithm automatic matching and targeted analysis;

[0107] 4. Multi-platform compatibility: Supports web and H5 mobile platforms, no app installation required, and fast response time;

[0108] 5. Resource utilization: Distributed dynamic scheduling, allocating resources on demand.

[0109] The parsing system provided in this application can be applied to the following scenarios:

[0110] In police enforcement scenarios: law enforcement officers can record videos on-site and upload and analyze them via mobile phone (H5 terminal) to quickly identify suspects and vehicles violating regulations, thus shortening the case investigation cycle;

[0111] Emergency rescue scenarios: Rescue videos from areas without network access can be uploaded afterward, automatically detecting scenarios such as fires and people being trapped, to help review the rescue process;

[0112] In urban management scenarios, urban management personnel upload street-side video footage, and the system automatically detects violations such as street vending and illegal parking, improving law enforcement efficiency.

[0113] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0114] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or apparatus.

[0115] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0116] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0117] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0118] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0119] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for executing all or part of the steps of the methods described in the various embodiments of this application through a computer device (which may be a personal computer, server, or network device, etc.). The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0120] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An offline video parsing method, characterized in that, include: If there are scene change points in the offline video, the offline video is segmented according to the scene change points to obtain several video segments; if there are no scene change points, the offline video is treated as a single video segment. Based on a pre-built scene library, scene matching is performed on each video segment to determine the target scene to which each video segment belongs; The adaptation score of each detection algorithm in the preset algorithm set corresponding to the target scene is calculated based on accuracy, processing speed and resource utilization. The detection algorithm with the highest adaptation score is selected as the optimal detection algorithm for the target scene to which each video segment belongs. Based on the optimal detection algorithm for each video segment, the corresponding video segment is detected, and the detection result of the offline video is output.

2. The offline video parsing method according to claim 1, characterized in that, The method of detecting whether there is a scene change point in the offline video includes: calculating the similarity between adjacent frames in the offline video; if there is a similarity lower than a first similarity threshold, then the next frame in the adjacent frames with a similarity lower than the first similarity threshold is determined to be the scene change point.

3. The offline video parsing method according to claim 1, characterized in that, Before detecting whether there are scene abrupt changes in the offline video, the method further includes: calculating the frame extraction frequency based on the duration of the offline video, the scene complexity coefficient of the first frame, and the target detection accuracy threshold; and performing frame extraction on the offline video according to the frame extraction frequency.

4. The offline video parsing method according to claim 1, characterized in that, The step of matching each video segment with a pre-set scene library to determine the target scene to which each video segment belongs includes: for each video segment, calculating the similarity between the video segment and each scene in the pre-set scene library; if the maximum similarity exceeds a second similarity threshold, then selecting the scene corresponding to the maximum similarity as the target scene to which the video segment belongs; if the maximum similarity does not exceed the second similarity threshold, then prompting the user to manually confirm the target scene to which the video segment belongs.

5. The offline video parsing method according to claim 1, characterized in that, The step of calculating the adaptation score of each detection algorithm in the preset algorithm set corresponding to the target scene based on accuracy, processing speed and resource utilization includes: weighting and summing the accuracy, processing speed and resource utilization of each detection algorithm in the preset algorithm set corresponding to the target scene in the target scene using preset weight coefficients to obtain the adaptation score of each detection algorithm in the preset algorithm set corresponding to the target scene.

6. The offline video parsing method according to claim 1, characterized in that, The optimal detection algorithm based on each video segment detects the corresponding video segment and outputs the detection result of the offline video, including: splitting each video segment into segments according to the optimal processing segment length of each detection algorithm to obtain a segment set; performing target detection on each video segment in the segment set using the corresponding optimal detection algorithm, and outputting the detection result of the offline video.

7. The offline video parsing method according to claim 1, characterized in that, The method further includes generating an alarm if an abnormal target is found in the detection results.

8. An offline video parsing method, characterized in that, include: Offline videos are matched to scenes based on a pre-built scene library to determine the target scene to which the offline video belongs. Based on accuracy, processing speed, and resource utilization, the adaptation score of each detection algorithm in the preset algorithm set corresponding to the target scene is calculated, and the detection algorithm with the highest adaptation score is selected as the optimal detection algorithm for the target scene to which the offline video belongs. The offline video is then checked for scene abrupt changes. If such a point exists, the offline video is segmented according to the scene abrupt change point to obtain several video segments, and the optimal detection algorithm for each video segment is determined. Based on the optimal detection algorithm for each video segment, the corresponding video segment is detected, and the detection result of the offline video is output. If no such point exists, the offline video is detected based on the optimal detection algorithm corresponding to the offline video, and the detection result of the offline video is output.

9. An offline video parsing system, characterized in that, include: The customer display module is used to upload offline videos and display test results; The analysis algorithm module is used to perform scene matching, algorithm matching, scene change point detection, and target detection on the offline video.

10. The offline video parsing system according to claim 9, characterized in that, The system also includes: an offline video processing module, used to analyze the duration and first frame scene features of the offline video, calculate the optimal frame extraction frequency, and extract frames from the offline video based on the optimal frame extraction frequency; and a platform service module, used for distributed scheduling of algorithm resources and structured storage of detection results and alarm data, supporting historical queries and secondary analysis.

Images