Method for multitasking of videos, video analysis device and storage medium

Abstract

The application relates to a multi-task processing method of a video, a video analysis device and a storage medium. The method comprises the following steps: performing topology analysis on algorithm modules of a to-be-executed algorithm package and an executing algorithm package; in the case that the to-be-executed algorithm package and the executing algorithm package have the same algorithm module, merging the same algorithm module, so as to keep the original topology of the to-be-executed algorithm package and merge the to-be-executed algorithm package into the executing algorithm package to obtain a new executing algorithm package; and performing intelligent analysis on an image sequence based on the new executing algorithm package. Through the application, the problem that multi-intelligent hybrid analysis occupies a high amount of analysis resources of a video analysis device is solved, and the amount of analysis resources of the video analysis device occupied by the multi-intelligent hybrid analysis is reduced.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to video multitasking methods, video analysis devices, and storage media. Background Technology

[0002] With the development of technology and the demands of application scenarios, intelligent analysis functions are increasingly being applied to video analysis devices (such as hard disk recorders, network cameras (IP cameras, or IPCs)) that handle video streams.

[0003] In related technologies, the allocation of computer resources (such as memory resources, disk resources, etc.) for intelligent analysis tasks is based on the granularity of the corresponding intelligent analysis algorithm. That is, when running an intelligent analysis task, computer resources are allocated to each algorithm according to its requirements. In multi-intelligent hybrid analysis scenarios, different intelligent analyses need to be performed on the same video stream; therefore, multi-intelligent hybrid analysis places high demands on the computer resources of the video analysis equipment.

[0004] In related technologies, to save disk resources, a method for storing algorithm packages has been proposed. This method separates algorithm package resources into common component resources and specific component resources, and stores them separately. When an intelligent analysis task is acquired, common components are selected from the common component resources, and specific components are selected from the specific component resources, and then combined into an algorithm package. This ensures that common components from different algorithm packages only need to be stored once in the system, saving disk resources. However, this related technology only addresses the problem of large storage space occupied by static program files, without addressing or intending to solve the problem of high resource consumption during program execution.

[0005] In related technologies, no effective solution has yet been proposed to address the issue of high resource consumption of video analysis equipment by multi-intelligent hybrid analysis. Summary of the Invention

[0006] The video multitasking method, video analysis device, and storage medium provided in this embodiment at least solve the problem of high resource consumption of video analysis devices in multi-intelligent hybrid analysis.

[0007] A video analysis device includes: a video decoding module and an intelligent analysis module;

[0008] The video decoding module is used to acquire the video stream and decode the video stream into an image sequence;

[0009] The intelligent analysis module is used to acquire a task to be executed for analyzing the image sequence, and to acquire an algorithm package to be executed corresponding to the task to be executed; to acquire an in-process task for analyzing the image sequence, and to acquire an in-process algorithm package corresponding to the in-process task; wherein, the algorithm package to be executed and the algorithm package in-process each include multiple algorithm modules, and the multiple algorithm modules are combined according to a topological structure corresponding to the algorithm package;

[0010] The intelligent analysis module is also used to perform topological structure analysis of the algorithm modules in the algorithm package to be executed and the algorithm package in execution. If the algorithm package to be executed and the algorithm package in execution have the same algorithm modules, the same algorithm modules are merged to merge the algorithm package to be executed into the algorithm package in execution while maintaining the original topological structure, so as to obtain a new algorithm package in execution. The same algorithm modules are algorithm modules with the same input data and the same processing logic for the input data.

[0011] The intelligent analysis module is also used to perform intelligent analysis on the image sequence based on the new in-process algorithm package.

[0012] In some of these embodiments,

[0013] The intelligent analysis module is also used to: generate a directed topology graph of the algorithm package to be executed and the algorithm package in execution, wherein each node in the directed topology graph represents an algorithm module, and each node is based on the processing logic identifier of the algorithm module;

[0014] When the intelligent analysis module has the same algorithm modules in the algorithm package to be executed and the algorithm package in execution, merging the same algorithm modules to merge the algorithm package to be executed into the algorithm package in execution while maintaining its original topology includes: when the first node with the same identifier in the directed topology graphs of the algorithm package to be executed and the algorithm package in execution has the same identifier, matching the same directed topology subgraphs in the directed topology graph with the first node as the starting node; merging the same directed topology subgraphs in the algorithm package to be executed and the algorithm package in execution to merge the directed topology graph of the algorithm package to be executed into the directed topology graph of the algorithm package in execution while maintaining its original topology; and obtaining the new algorithm package in execution based on the merged directed topology graph.

[0015] In some embodiments, the intelligent analysis module performs topological structure analysis of the algorithm module on the algorithm package to be executed and the algorithm package in execution, including:

[0016] Based on the identifier of the algorithm module, obtain the same algorithm module in the algorithm package to be executed and the algorithm package in execution, wherein the identifier is determined based on the input data and processing logic of the algorithm module.

[0017] In some embodiments, the identifier of the algorithm module includes: the name identifier and version identifier of all algorithm modules preceding the algorithm module in the directed topology, and the name identifier and version identifier of the algorithm module.

[0018] A method for multitasking video processing includes:

[0019] Obtain the task to be executed for analyzing the image sequence, and obtain the algorithm package to be executed corresponding to the task to be executed;

[0020] A task in execution for analyzing the image sequence is obtained, and an algorithm package in execution corresponding to the task in execution is obtained; wherein, the algorithm package to be executed and the algorithm package in execution each include multiple algorithm modules, and the multiple algorithm modules are combined according to the topological structure corresponding to the algorithm package;

[0021] A topological analysis of the algorithm modules is performed on the algorithm package to be executed and the algorithm package in execution. If the algorithm package to be executed and the algorithm package in execution have the same algorithm modules, the same algorithm modules are merged to merge the algorithm package to be executed into the algorithm package in execution while maintaining the original topological structure, so as to obtain a new algorithm package in execution. The same algorithm modules are algorithm modules with the same input data and the same processing logic for the input data.

[0022] The image sequence is intelligently analyzed based on the new in-process algorithm package.

[0023] In some of these embodiments,

[0024] The method further includes: generating a directed topology graph of the algorithm package to be executed and the algorithm package in execution, wherein each node in the directed topology graph represents an algorithm module, and each node is based on the processing logic identifier of the algorithm module;

[0025] When the algorithm package to be executed and the algorithm package in execution have the same algorithm modules, merging the same algorithm modules to merge the algorithm package to be executed into the algorithm package in execution while maintaining its original topology includes: when the first node with the same identifier has the same identifier in the directed topology graphs of the algorithm package to be executed and the algorithm package in execution, matching the same directed topology subgraphs in the directed topology graph with the first node as the starting node; merging the same directed topology subgraphs in the algorithm package to be executed and the algorithm package in execution to merge the directed topology graph of the algorithm package to be executed into the directed topology graph of the algorithm package in execution while maintaining its original topology; and obtaining the new algorithm package in execution based on the merged directed topology graph.

[0026] In some embodiments, performing topological structure analysis of the algorithm modules in the algorithm package to be executed and the algorithm package in execution includes:

[0027] Based on the identifier of the algorithm module, obtain the same algorithm module in the algorithm package to be executed and the algorithm package in execution, wherein the identifier is determined based on the input data and processing logic of the algorithm module.

[0028] In some embodiments, the identifier of the algorithm module includes: the name identifier and version identifier of all algorithm modules preceding the algorithm module in the directed topology, and the name identifier and version identifier of the algorithm module.

[0029] An electronic device includes: a processor; and a memory storing a program, the program including instructions that, when executed by the processor, cause the processor to perform any of the methods described above.

[0030] A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform any of the methods described above.

[0031] The video multitasking processing method, video analysis device, and storage medium provided in this embodiment perform topological structure analysis of the algorithm modules in the algorithm package to be executed and the algorithm package in execution. When the algorithm package to be executed and the algorithm package in execution have the same algorithm modules, the same algorithm modules are merged to merge the algorithm package to be executed into the algorithm package in execution while maintaining the original topological structure, thereby obtaining a new algorithm package in execution. This reduces the amount of analysis resources occupied by the video analysis device in the multi-intelligent hybrid analysis.

[0032] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0033] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0034] Figure 1 This is a flowchart of the A algorithm in this embodiment.

[0035] Figure 2 This is a flowchart of the B algorithm in this embodiment.

[0036] Figure 3 This is a flowchart of a video multitasking method according to this embodiment.

[0037] Figure 4 This is a flowchart of the algorithm package processing during the execution of this embodiment.

[0038] Figure 5 This is a schematic diagram of the video analysis device in this embodiment.

[0039] Figure 6 This is a schematic diagram of the structure of the electronic device in this embodiment. Detailed Implementation

[0040] Embodiments of this embodiment will now be described in more detail with reference to the accompanying drawings. While some embodiments of this embodiment are shown in the drawings, it should be understood that this embodiment can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this embodiment. It should be understood that the accompanying drawings and embodiments are for illustrative purposes only and are not intended to limit the scope of protection of this embodiment.

[0041] It should be understood that the steps described in the method implementation of this embodiment may be performed in different orders and / or in parallel. Furthermore, the method implementation may include additional steps and / or omit the steps shown. The scope of this embodiment is not limited in this respect.

[0042] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0043] It should be noted that the terms "one" and "more" mentioned in this embodiment are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0044] It should be stated that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this embodiment are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0045] The video multitasking method provided in this embodiment can be applied to a system that performs multiple intelligent analysis tasks on the same set of data based on multiple algorithm packages.

[0046] In this embodiment, the video analysis device used for intelligent analysis of the video stream can be, but is not limited to, a hard disk recorder or network camera (IPC) that supports intelligent analysis. The hard disk recorder can be a network video recorder (NVR) or a digital video recorder (DVR). The video stream used for the intelligent analysis task can be the input video stream from a remote IPC, the video stream recorded from a local hard disk, or the video stream from an analog camera connected via an analog signal.

[0047] If there are two intelligent analysis tasks, A and B, corresponding to algorithms A and B respectively, for example, algorithm A can perform intelligent analysis of on-duty status, number of people in the area, mobile phone use, and uniform detection. Algorithm B can perform intelligent analysis of behaviors such as falling, fighting, climbing, getting up, throwing objects, standing up, and sitting down. Figure 1 and Figure 2 The processing flow of Algorithm A and Algorithm B are illustrated below. After the IPC stream is received, image sequences are obtained through stream decoding and parsing. Algorithm A then performs target detection, target location matching, Algorithm A classification, and Algorithm A alarm processing based on the image sequences. Algorithm B, based on the image sequences, performs target detection, target location matching, rule judgment, Algorithm B classification, and Algorithm B alarm processing.

[0048] Depend on Figure 1 and Figure 2It can be seen that there are common processing flows in both Algorithm A and Algorithm B: target detection and target location matching. Furthermore, the input data and data processing logic for these two flows are completely identical; that is, these common processing flows are the same. In related technologies, resource allocation and execution are handled separately for each algorithm: when performing intelligent analysis of video streams based on Algorithm A and Algorithm B respectively, when allocating computer resources for Algorithm A, computer resources will be allocated to the target detection and target location matching modules of Algorithm A; similarly, when allocating computer resources for Algorithm B, computer resources will be allocated to the target detection and target location matching modules of Algorithm B. When running Algorithms A and B, target detection and target location matching must be performed independently on the image sequence. Insufficient computer resources will lead to a decrease in the operating efficiency of Algorithms A and B.

[0049] Computer resources used for intelligent analysis, referred to as analysis resources, include, but are not limited to, memory resources, disk resources, graphics processor resources, and I / O resources. However, the total amount of analysis resources in an IPC or hard disk recorder is limited and precious; real-time analysis of video streams also places certain demands on the operating efficiency of video analysis equipment. In this embodiment, to reduce the problem of high computer analysis resource consumption and low operating efficiency of multiple algorithms in video analysis equipment, a multi-task processing method for video is adopted to merge the same processing flows in multiple algorithms, thereby reducing the repeated running of the same processing flows.

[0050] Figure 3 This is a flowchart of a video multitasking method according to this embodiment, such as... Figure 3 As shown, the process includes the following steps:

[0051] Step S301: Obtain the task to be executed for analyzing the image sequence, and obtain the algorithm package to be executed corresponding to the task to be executed.

[0052] Step S302: Obtain the task in execution for analyzing the image sequence, and obtain the algorithm package in execution corresponding to the task in execution.

[0053] The algorithm package to be executed and the algorithm package in execution each include multiple algorithm modules, and the multiple algorithm modules are combined according to the topological structure corresponding to the algorithm package.

[0054] Step S303: Perform topological analysis of the algorithm modules in the algorithm package to be executed and the algorithm package in execution. If the algorithm packages to be executed and the algorithm packages in execution have the same algorithm modules, merge the same algorithm modules to merge the algorithm package to be executed into the algorithm package in execution while maintaining the original topological structure, and obtain a new algorithm package in execution.

[0055] Among them, the same algorithm module is the algorithm module that has the same input data and the same processing logic for the input data.

[0056] Step S304: Perform intelligent analysis on the image sequence based on the new in-process algorithm package.

[0057] When developing intelligent analysis functions, an algorithm package needs to be developed for each intelligent analysis task. Typically, an algorithm package consists of algorithm modules, which are connected sequentially according to a pre-defined topology to form an algorithm package for the corresponding intelligent analysis task. Each algorithm module processes the data input to the algorithm module according to predetermined processing logic and provides the processing result to the next algorithm module or as the final output of the task processing.

[0058] For example, in Figure 1 In the processing flow of Algorithm A shown in the diagram, the target detection process, target location matching process, Algorithm A classification process, and Algorithm A alarm process are implemented by the target detection module, target location matching module, Algorithm A classification module, and Algorithm A alarm module, respectively. The IPC stream access process, stream parsing process, and stream decoding process can be considered part of the algorithm. However, since these processes are typically identical for most video stream processing algorithms, they can sometimes be considered as common processing flows outside the algorithm itself and implemented using common algorithm modules. However, algorithm packages are usually developed and configured independently. In related technologies, even the same algorithm modules in different algorithm packages need to be executed repeatedly within their respective packages. Common algorithm modules that need to be executed in different algorithm packages include, for example, the target detection algorithm module and the target location matching algorithm module.

[0059] Therefore, after acquiring multiple intelligent analysis tasks, the relevant technology obtains the corresponding algorithm package for each intelligent analysis task, and directly applies for computer resources and performs multi-task processing based on each algorithm package.

[0060] Unlike related technologies, in the video multitasking method provided in this embodiment, after obtaining the algorithm package to be executed for the task to be executed, the algorithm package to be executed and the algorithm package to be executed corresponding to the currently executing task in the system will be analyzed for the topological structure of the algorithm modules to determine whether the two algorithm packages have the same algorithm modules.

[0061] If the two algorithm packages contain the same algorithm modules, there is room for algorithm optimization. The process in step S303 will be adopted to merge the two algorithm packages with the same algorithm modules to obtain a new algorithm package in execution.

[0062] In this embodiment, merging identical algorithm modules refers to combining the identical algorithm module portions of two algorithm packages into one, so that the original algorithm package to be executed and the algorithm package in execution can share the same algorithm module, thereby eliminating the need for the identical algorithm module to perform redundant data processing.

[0063] Accordingly, after merging, the output of the same algorithm module will be copied at least twice, and provided to other algorithm modules following the same algorithm module in the two algorithm packages before the merge. For example, for Figure 1 and Figure 2 In the processing flows of Algorithms A and B shown, the target detection and target location matching processes are implemented by the same algorithm module, while other processes (Algorithm A classification, Algorithm A alarm, rule judgment, Algorithm B classification, Algorithm B alarm, etc.) are implemented by different algorithm modules. Therefore, after merging, Algorithms A and B will reuse the target detection and target location matching modules. After the target location matching module, to implement Algorithm A, the Algorithm A classification module and Algorithm A alarm module will be connected; to implement Algorithm B, the rule judgment module, Algorithm B classification module, and Algorithm B alarm module will be connected. Therefore, after merging the same algorithm modules, the resulting new execution algorithm package will be as follows: Figure 4 As shown.

[0064] When executing a new algorithm package in step S304, computer analysis resources are re-requested for the new algorithm package in operation based on the computer analysis resources required for its execution.

[0065] Figure 4 The new execution algorithm package shown consists of 7 algorithm modules, compared to the 9 algorithm modules that previously made up Algorithm Package A and Algorithm Package B. This reduces the number of algorithm modules by 2. Therefore, when performing multi-task processing of video based on the new execution algorithm package, it will also reduce the computer resources (including but not limited to memory resources, graphics processor resources, I / O resources, etc. used for task intelligent analysis) required by the two algorithm modules, and avoid the same and repetitive processing of data by the target detection module and the target position matching module, thereby improving processing efficiency and reducing computer resource consumption.

[0066] It should be noted that in this embodiment, the same algorithm module needs to meet two conditions simultaneously: first, the input data of the two algorithm modules are exactly the same, for example, in Figure 1 and Figure 2In the processing flows of algorithms A and B shown, the input data is an image sequence obtained by parsing and decoding the video stream. Secondly, the data processing logic of the two algorithm modules is exactly the same; that is, given identical input data, the output data obtained after processing by the two algorithm modules is also identical. For example, in... Figure 1 and Figure 2 In the processing flow of algorithms A and B shown, the object detection module processes the image sequence to obtain the object detection bounding box result for the human body, and the object location matching module processes the image sequence to obtain the location of the object detection bounding box result output by the object detection module. Therefore, Figure 1 and Figure 2 In algorithms A and B shown, the target detection module and the target location matching module belong to the same algorithm module. By merging the same algorithm modules based on the target detection module and the target location matching module, the final result can be obtained. Figure 4 The new algorithm package in execution is shown. If the input data of two algorithm modules are different, or if one of their processing logics is different, then these two algorithm modules do not belong to the same algorithm modules as referred to in this embodiment.

[0067] In this embodiment, the determination of identical algorithm modules is based on the directed topology graph of the algorithm packages. Therefore, the above-mentioned video multi-task processing method further includes generating directed topology graphs for each algorithm package. A directed topology graph, also called a directed graph, refers to a topology graph formed by connecting nodes in a certain order. Each node in the directed topology graph represents an algorithm module, and each node is identified based on the processing logic of the algorithm module. For each algorithm package, a corresponding directed topology graph can be generated based on a certain granularity. This granularity can be at the application function level or the program block level. The application function level means that each node represents a complete application function, for example... Figure 4 This includes object detection and object location matching functions. Program block level refers to each node representing a specific part of a complete application function; for example, in... Figure 4 The target detection module can also be divided into a module for processing frames of the image sequence, and a module for detecting targets in the image sequence obtained by processing frames.

[0068] In this embodiment, the same directed topological subgraphs can be matched in directed topological graphs across multiple algorithm packages based on a subgraph isomorphism algorithm. A subgraph isomorphism algorithm, for example, is the UIImann algorithm. For the graph isomorphism problem, the UIImann algorithm uses enumeration to find subgraph isomorphisms; its purpose is to find all subgraphs in the original graph that are isomorphic to the original graph, given a predefined graph.

[0069] When the number of nodes in a directed topological graph is large, the number of preset graphs will increase exponentially with the number of nodes in the directed topological graph when using the subgraph isomorphism algorithm to enumerate each preset graph in the directed topological graph, which may consume a lot of computation time or resources. Therefore, in this embodiment, the processing of the subgraph isomorphism algorithm is optimized. In this embodiment, in step S303, if the first node with the same identifier is found in the directed topological graphs of the algorithm package to be executed and the algorithm package in execution, the same directed topological subgraphs starting from the first node are matched in the directed topological graph; the same directed topological subgraphs in the algorithm package to be executed and the algorithm package in execution are merged to merge the directed topological graph of the algorithm package to be executed into the directed topological graph of the algorithm package in execution while maintaining the original topological structure; a new algorithm package in execution is obtained based on the merged directed topological graph.

[0070] When using the subgraph isomorphism algorithm to enumerate the preset graph, only all preset graphs starting from the first node of the directed topological graph are enumerated. When matching the preset graph in the original graph, only the topological subgraphs starting from the first node in the directed topological graph are matched, thus reducing many unnecessary matching processes and improving matching efficiency. Furthermore, since the same algorithm module in this embodiment refers to an algorithm module with completely identical input data and processing logic, for two directed topological subgraphs, if the algorithm modules corresponding to the first node are different, even if the input data is the same, these two directed topological subgraphs cannot be the same graph. Similarly, if two directed topological subgraphs have completely identical topological structures, but the nodes connected to the first node of these two directed topological subgraphs are different, this means that the input data (output data of the previous node) of these two directed topological subgraphs cannot be the same. Therefore, these two directed topological subgraphs still cannot be the same graph, and the corresponding algorithm modules in the directed topological subgraphs cannot be the same algorithm modules. Therefore, using the optimized subgraph isomorphism algorithm described above for matching directed topological subgraphs will not result in missed matching of directed topological subgraphs or algorithm modules, and can also improve matching efficiency.

[0071] In this embodiment, the identical algorithm modules in the algorithm package to be executed and the algorithm package in execution can also be obtained based on the identifier of the algorithm module. The identifier of the algorithm module is determined based on its input data and processing logic. That is, if an algorithm module can be identified based on its input data and processing logic, then when matching identical algorithm modules, only text matching of the identifiers of all algorithm modules is required. Compared to using a directed topological graph for subgraph isomorphic matching, the computational efficiency during the matching process can be further improved.

[0072] It is known that each different algorithm module has different processing logic for the input data, or that for two algorithm modules with the same processing logic, different input data will result in different output data. Therefore, the input data of the current algorithm module can be equivalently represented by the name and version identifiers of all the algorithm modules connected to it before it, while the processing logic of the current algorithm module is represented by its own name and version identifier. In this way, the current algorithm module can be identified as an ordered string consisting of the name and version identifiers of all the algorithm modules preceding it, as well as the name and version identifier of the current algorithm module itself.

[0073] For example, consider an algorithm package (comprising four algorithm modules) with a directed topological graph structure A1→B1→C2→D1. The first algorithm module is identified as A1; the second as A1B1; the third as A1B1C2; and the fourth as A1B1C2D1. For another algorithm package with the same directed topological graph structure A1→B1→D1→C2, the first algorithm module is identified as A1; the second as A1B1; the third as A1B1D1; and the fourth as A1B1D1C2. By comparing the identifiers, we can determine that the algorithm modules identified as A1 and A1B1 are the same. Even if other algorithm modules have the same name and version identifier, their different order leads to different input data, thus the third and fourth algorithm modules have different identifiers and are not the same.

[0074] exist Figure 3 In the multi-task processing method shown in the video, the directed topology graph of the algorithm package or the identifier of the algorithm module can be obtained after acquiring the algorithm package by analyzing its topology and algorithm modules. This method can flexibly adapt to existing or newly added algorithm packages. In this method, the analysis of the algorithm package, the splitting and reorganization of algorithm modules involve the use of computer resources.

[0075] To further conserve computer disk resources, the system does not directly store the algorithm packages for each task, but instead stores the algorithm modules that make up each algorithm package. In this case, in steps S301 and S302 above, both the task to be executed and the task being executed can be represented by intelligent analysis task description information, which includes information describing the topological structure of the algorithm package corresponding to the intelligent analysis task.

[0076] Therefore, in steps S301 and S302, when obtaining the algorithm package to be executed and the algorithm package in execution, the corresponding module can be selected from the pre-stored algorithm modules by using the topological structure information of the algorithm package carried in the description information of the algorithm package to be executed and the algorithm package in execution, and combined and connected according to the topological structure information in the description information, so as to obtain the algorithm package to be executed and the algorithm package in execution.

[0077] The above method not only saves computer disk resources, but also allows for the direct use of topology information in the task description information when performing topology analysis on the algorithm package to be executed and the algorithm package in execution, without consuming computer resources to obtain the corresponding topology from the algorithm package to be executed and the algorithm package in execution.

[0078] This embodiment also provides a video analysis device, which may be, for example, but not limited to, a hard disk recorder or network camera (IPC) that supports intelligent analysis. The hard disk recorder may be a network video recorder (NVR) or a digital video recorder (DVR). The video stream used for intelligent analysis tasks may be the input video stream from a remote IPC, the video stream recorded from a local hard disk, or the video stream from an analog camera accessed via an analog signal.

[0079] Figure 5 This is a schematic diagram of the video analysis device in this embodiment, as shown below. Figure 5 As shown, the video analysis device includes a video decoding module 51 and an intelligent analysis module 52.

[0080] The video decoding module 51 is used to acquire the video stream and decode the video stream into an image sequence.

[0081] The intelligent analysis module 52 is connected to the video decoding module 51. The intelligent analysis module 52 is used to obtain the task to be executed for processing the image sequence, and to obtain the algorithm package to be executed corresponding to the task to be executed; to obtain the task in execution for processing the image sequence, and to obtain the algorithm package in execution corresponding to the task in execution; wherein, the algorithm package to be executed and the algorithm package in execution each include multiple algorithm modules, and the multiple algorithm modules are combined according to the topology corresponding to the algorithm package.

[0082] The intelligent analysis module 52 is also used to perform topological structure analysis of the algorithm modules in the algorithm package to be executed and the algorithm package in execution. If the algorithm package to be executed and the algorithm package in execution have the same algorithm modules, the same algorithm modules are merged to merge the algorithm package to be executed into the algorithm package in execution while maintaining the original topological structure, so as to obtain a new algorithm package in execution. The same algorithm modules are algorithm modules with the same input data and the same processing logic for the input data.

[0083] The intelligent analysis module 52 is also used to perform intelligent analysis on image sequences based on the new in-process algorithm package.

[0084] In some embodiments, the video decoding module 51 can be a software decoder or a hardware decoder. For example, the video decoding module 51 can be a graphics processing unit (GPU) that supports hardware decoding, or a central processing unit (CPU) that supports either hardware or software decoding.

[0085] In some embodiments, the video decoding module 51 described above can be a decoder built into the video analysis device, or it can be a device with decoding function connected to the video analysis device.

[0086] In some of these embodiments, the video decoding module 51 described above may include multiple decoders.

[0087] In some implementations, the intelligent analysis module 52 includes multiple intelligent analysis units, which include, but are not limited to, analysis resources such as graphics processors, central processing units, and memory, and may also include storage resources such as memory, as well as I / O resources.

[0088] In some embodiments, the intelligent analysis module 52 is also used to generate a directed topology graph of the algorithm package to be executed and the algorithm package in execution, wherein each node in the directed topology graph represents an algorithm module, and each node is based on the processing logic identifier of the algorithm module.

[0089] When the intelligent analysis module 52 contains the same algorithm modules in the algorithm package to be executed and the algorithm package in execution, it merges the same algorithm modules to maintain the original topology of the algorithm package to be executed into the algorithm package in execution. This includes: when the first node with the same identifier is in the directed topology graphs of the algorithm package to be executed and the algorithm package in execution, matching the same directed topology subgraphs starting from the first node in the directed topology graph; merging the same directed topology subgraphs in the algorithm package to be executed and the algorithm package in execution to maintain the original topology of the directed topology graph of the algorithm package to be executed into the directed topology graph of the algorithm package in execution; and obtaining a new algorithm package in execution based on the merged directed topology graph.

[0090] In some embodiments, the intelligent analysis module performs topological structure analysis of the algorithm modules in the algorithm package to be executed and the algorithm package in execution, including: obtaining the same algorithm modules in the algorithm package to be executed and the algorithm package in execution based on the identifier of the algorithm module, wherein the identifier is determined based on the input data and processing logic of the algorithm module.

[0091] In some embodiments, the identifier of the algorithm module includes: the name identifier and version identifier of all algorithm modules preceding the algorithm module in the directed topology, and the name identifier and version identifier of the algorithm module.

[0092] The video analysis device described in this embodiment, by merging the algorithm package of the intelligent analysis task to be executed with the same algorithm module in the algorithm package of the intelligent analysis task being executed into a shared module, is equivalent to reusing the same computer analysis resources. This reduces the computer analysis resources required to execute intelligent analysis tasks, improves the utilization rate of computer analysis resources of the video analysis device, and also supports running more intelligent analysis tasks simultaneously within the same specification of computer analysis resources, thereby enhancing the competitiveness of the product.

[0093] This embodiment also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, which, when executed by the at least one processor, causes the electronic device to perform a method according to an embodiment of this disclosure.

[0094] This embodiment also provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to an embodiment of this disclosure.

[0095] refer to Figure 6The present invention describes a structural block diagram of an electronic device 600 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0096] like Figure 6 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0097] Multiple components in electronic device 600 are connected to I / O interface 605, including: input unit 606, output unit 607, storage unit 608, and communication unit 609. Input unit 606 can be any type of device capable of inputting information to electronic device 600. Input unit 606 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 607 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 608 may include, but is not limited to, disk and optical disk. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0098] The computing unit 601 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above. For example, in some embodiments, the above methods (e.g. Figure 3 The method illustrated can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 600 via ROM 602 and / or communication unit 609. In some embodiments, computing unit 601 can be configured to perform the method described above by any other suitable means (e.g., by means of firmware).

[0099] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable image sequence processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0100] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0101] As used in this disclosure, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0102] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0103] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0104] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0105] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0106] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0107] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0108] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A video analysis device, comprising: Video decoding module and intelligent analysis module; The video decoding module is used to acquire the video stream and decode the video stream into an image sequence; The intelligent analysis module is used to acquire a task to be executed for analyzing the image sequence, and to acquire an algorithm package to be executed corresponding to the task to be executed; to acquire an in-process task for analyzing the image sequence, and to acquire an in-process algorithm package corresponding to the in-process task; wherein, the algorithm package to be executed and the algorithm package in-process each include multiple algorithm modules, and the multiple algorithm modules are combined according to a topological structure corresponding to the algorithm package; The intelligent analysis module is also used to perform topological structure analysis of the algorithm modules in the algorithm package to be executed and the algorithm package in execution. If the algorithm package to be executed and the algorithm package in execution have the same algorithm modules, the same algorithm modules are merged to merge the algorithm package to be executed into the algorithm package in execution while maintaining the original topological structure, so as to obtain a new algorithm package in execution. The same algorithm modules are algorithm modules with the same input data and the same processing logic for the input data. The intelligent analysis module is also used to perform intelligent analysis on the image sequence based on the new in-process algorithm package.

2. The video analysis device according to claim 1, wherein, The intelligent analysis module is also used to: generate a directed topology graph of the algorithm package to be executed and the algorithm package in execution, wherein each node in the directed topology graph represents an algorithm module, and each node is based on the processing logic identifier of the algorithm module; When the intelligent analysis module has the same algorithm modules in the algorithm package to be executed and the algorithm package in execution, merging the same algorithm modules to merge the algorithm package to be executed into the algorithm package in execution while maintaining its original topology includes: when the first node with the same identifier in the directed topology graphs of the algorithm package to be executed and the algorithm package in execution has the same identifier, matching the same directed topology subgraphs in the directed topology graph with the first node as the starting node; merging the same directed topology subgraphs in the algorithm package to be executed and the algorithm package in execution to merge the directed topology graph of the algorithm package to be executed into the directed topology graph of the algorithm package in execution while maintaining its original topology; and obtaining the new algorithm package in execution based on the merged directed topology graph.

3. The video analysis device according to claim 1, wherein, The intelligent analysis module performs topological structure analysis of the algorithm module for the algorithm package to be executed and the algorithm package in execution, including: Based on the identifier of the algorithm module, obtain the same algorithm module in the algorithm package to be executed and the algorithm package in execution, wherein the identifier is determined based on the input data and processing logic of the algorithm module.

4. The video analysis device according to claim 3, wherein, The identifiers of the algorithm modules include: the name identifiers and version identifiers of all algorithm modules preceding the algorithm module in the directed topology, as well as the name identifier and version identifier of the algorithm module.

5. A method for multi-task processing of video, comprising: Obtain the task to be executed for analyzing the image sequence, and obtain the algorithm package to be executed corresponding to the task to be executed; A task in execution for analyzing the image sequence is obtained, and an algorithm package in execution corresponding to the task in execution is obtained; wherein, the algorithm package to be executed and the algorithm package in execution each include multiple algorithm modules, and the multiple algorithm modules are combined according to the topological structure corresponding to the algorithm package; A topological analysis of the algorithm modules is performed on the algorithm package to be executed and the algorithm package in execution. If the algorithm package to be executed and the algorithm package in execution have the same algorithm modules, the same algorithm modules are merged to merge the algorithm package to be executed into the algorithm package in execution while maintaining the original topological structure, so as to obtain a new algorithm package in execution. The same algorithm modules are algorithm modules with the same input data and the same processing logic for the input data. The image sequence is intelligently analyzed based on the new in-process algorithm package.

6. The method according to claim 5, wherein, The method further includes: generating a directed topology graph of the algorithm package to be executed and the algorithm package in execution, wherein each node in the directed topology graph represents an algorithm module, and each node is based on the processing logic identifier of the algorithm module; When the algorithm package to be executed and the algorithm package in execution have the same algorithm modules, merging the same algorithm modules to merge the algorithm package to be executed into the algorithm package in execution while maintaining its original topology includes: when the first node with the same identifier has the same identifier in the directed topology graphs of the algorithm package to be executed and the algorithm package in execution, matching the same directed topology subgraphs in the directed topology graph with the first node as the starting node; merging the same directed topology subgraphs in the algorithm package to be executed and the algorithm package in execution to merge the directed topology graph of the algorithm package to be executed into the directed topology graph of the algorithm package in execution while maintaining its original topology; and obtaining the new algorithm package in execution based on the merged directed topology graph.

7. The method according to claim 5, wherein, The topological structure analysis of the algorithm modules in the algorithm package to be executed and the algorithm package in execution includes: Based on the identifier of the algorithm module, obtain the same algorithm module in the algorithm package to be executed and the algorithm package in execution, wherein the identifier is determined based on the input data and processing logic of the algorithm module.

8. The method according to claim 7, wherein, The identifiers of the algorithm modules include: the name identifiers and version identifiers of all algorithm modules preceding the algorithm module in the directed topology, as well as the name identifier and version identifier of the algorithm module.

9. An electronic device, comprising: processor; And a memory storing a program, the program comprising instructions, wherein, when executed by the processor, the instructions cause the processor to perform the method according to any one of claims 5 to 8.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 5 to 8.

Images