Video analysis method, system, apparatus, and non-transitory storage medium

Abstract

The application discloses a video analysis method, system and device and a nonvolatile storage medium. The method comprises the following steps: an event detection receiving module acquires event information of a detection event from a network video recorder (NVR), wherein the NVR is used for detecting a detection event appearing in a video recorded by a connected camera; the event detection receiving module sends the event information to a video analysis service module; the video analysis service module calls a video segment corresponding to the detection event from a video storage platform according to the event information, wherein the video storage platform is used for acquiring complete video data shot by the camera from the NVR and storing the complete video data; and the video analysis service module processes the video segment to obtain an analysis result of the detection event. The application solves the technical problem of high video analysis operation cost caused by video analysis on complete video shot by the camera in the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, and in particular, to a video analysis method, system, device and non-volatile storage medium. BACKGROUND

[0002] With the development of video analysis demand of various industries (such as expressway, computer room, supermarket, building, etc.), the combination of massive video data and cloud computing, cloud storage, big data is more and more close, and the requirement for computing power is also higher and higher; taking the computer room of a provincial communication operator as an example, thousands of computer rooms need to be maintained, and the video cameras accessed are as many as 5000, after all the 5000 cameras are started 7x24 hours background real-time video analysis, not only the network bandwidth may become a bottleneck, but also hundreds of high-performance servers with GPU cards are needed for hardware support (assuming that one can simultaneously process 50 channels in parallel), the cost is very high; considering the characteristics of ultra-low human flow in the computer room, the time when a camera has a person passing through in a day may not exceed 2 hours, so 83% of the analysis is meaningless, which will cause huge computing resource cost.

[0003] The increasing number of user end calls causes many video intelligent analysis platforms to be overburdened, forcing many platform manufacturers to cope by improving the performance of hardware facilities or adding the number of hardware servers, but this coping method not only increases the cost of the platform, but also increases the workload of platform maintenance.

[0004] At present, no effective solution has been proposed for the above problems. SUMMARY

[0005] The embodiments of the present application provide a video analysis method, system, device and non-volatile storage medium to at least solve the technical problem of high video analysis running cost caused by video analysis on the complete video shot by the camera in the prior art.

[0006] According to one aspect of the embodiments of the present application, a video analysis method is provided, comprising: an event detection receiving module acquiring event information of a detection event from a network video recorder (NVR), wherein the NVR is used to detect a detection event appearing in a video recorded by a connected camera, and the event information includes the occurrence time of the detection event, the NVR information corresponding to the NVR recording the detection event, and the camera information corresponding to the camera recording the detection event; the event detection receiving module sends the event information to a video analysis service module; the video analysis service module calls a video segment corresponding to the detection event from a video storage platform according to the event information, wherein the video storage platform is used to acquire complete video data shot by the camera from the NVR and store the complete video data; and the video analysis service module processes the video segment to obtain an analysis result of the detection event.

[0007] Optionally, the event information is sent to the video analysis service module by the detection event receiving module, comprising: the event information is sent to the intelligent scheduling module by the detection event receiving module; the target computing node capable of analyzing the detection event is selected from the plurality of servers included in the video analysis service module according to the computing resource state of the video analysis service module by the intelligent scheduling module; and the event information is sent to the target computing node by the intelligent scheduling module.

[0008] Optionally, the target computing node capable of analyzing the detection event is selected from the plurality of servers included in the video analysis service module according to the computing resource state of the video analysis service module by the intelligent scheduling module, comprising: the target server capable of analyzing the detection event is determined according to the computing resource state by the intelligent scheduling module, and the target computing node analyzing the detection event is determined from the target server, wherein the target server is one of the plurality of GPU servers registered on the intelligent scheduling module by the video analysis service module, any one of the GPU servers includes a plurality of computing nodes, and the computing resource state includes the IP addresses of the plurality of GPU servers and the node resource occupation state of any one of the GPU servers.

[0009] Optionally, the video segment corresponding to the detection event is called from the video storage platform by the video analysis service module according to the event information, comprising: the event calling instruction is generated by the video analysis service module according to the event information, wherein the event calling instruction includes: the event video time range, the NVR information and the camera information, and the event video time range includes the occurrence time of the detection event; the event calling instruction is sent to the video storage platform by the video analysis service module; and the video segment returned by the video storage platform is received by the video analysis service module, wherein the time range of the video segment matches the event video time range, and the video segment respectively matches the NVR corresponding to the NVR information and the camera corresponding to the camera information.

[0010] According to another aspect of the embodiments of the present application, a video analysis method is also provided, comprising: receiving event information of a detection event sent by a detection event receiving module, wherein the event information is event information of the detection event acquired by the detection event receiving module from a network video recorder (NVR), the NVR is configured to detect a detection event occurring in a video recorded by a connected camera, and the event information comprises a time of occurrence of the detection event, NVR information corresponding to the NVR recording the detection event, and camera information corresponding to the camera recording the detection event; selecting a target computing node capable of analyzing the detection event from a plurality of servers included in a video analysis service module according to a computing resource state of the video analysis service module; and sending the event information to the target computing node, wherein the target computing node is configured to call a video segment corresponding to the detection event from a video storage platform according to the event information, and to process the video segment to obtain an analysis result of the detection event, and the video storage platform is configured to acquire complete video data recorded by the camera from the NVR and store the complete video data.

[0011] Optionally, the selecting the target computing node capable of analyzing the detection event from the plurality of servers included in the video analysis service module according to the computing resource state of the video analysis service module comprises: determining a target server capable of analyzing the detection event according to the computing resource state, and determining the target computing node analyzing the detection event from the target server, wherein the target server is one of a plurality of graphics card servers registered on an intelligent scheduling module by the video analysis service module, any one of the graphics card servers comprises a plurality of computing nodes, and the computing resource state comprises IP addresses of the plurality of graphics card servers and node resource occupation states of any one of the graphics card servers.

[0012] According to another aspect of the embodiments of the present application, a video analysis system is also provided, comprising: a detection event receiving module configured to acquire event information of a detection event from a network video recorder (NVR), and to send the event information to a video analysis service module, wherein the NVR is configured to detect a detection event occurring in a video recorded by a connected camera, and the event information comprises a time of occurrence of the detection event, NVR information corresponding to the NVR recording the detection event, and camera information corresponding to the camera recording the detection event; the video analysis service module is configured to call a video segment corresponding to the detection event from a video storage platform according to the event information, and to process the video segment to obtain an analysis result of the detection event; and the video storage platform is configured to acquire complete video data recorded by the camera from the NVR and store the complete video data.

[0013] According to another aspect of the embodiments of the present application, a video analysis device is also provided, which comprises: a receiving unit configured to receive event information of a detection event sent by a detection event receiving module, wherein the event information is event information of the detection event acquired by the detection event receiving module from a network video recorder (NVR), the NVR is configured to detect a detection event occurring in a video recorded by a connected camera, and the event information comprises a time of occurrence of the detection event, NVR information corresponding to the NVR recording the detection event, and camera information corresponding to the camera recording the detection event; a screening unit configured to screen a target computing node capable of analyzing the detection event from a plurality of servers included in a video analysis service module according to a computing resource state of the video analysis service module; and a sending unit configured to send the event information to the target computing node, wherein the target computing node is configured to call a video segment corresponding to the detection event from a video storage platform according to the event information, and further configured to process the video segment to obtain an analysis result of the detection event, and the video storage platform is configured to acquire complete video data recorded by the camera from the NVR and store the complete video data.

[0014] According to still another aspect of the embodiments of the present application, a non-volatile storage medium is also provided, which comprises a stored program, wherein the program, when executed, controls a device in which the non-volatile storage medium is located to perform any one of the video analysis methods.

[0015] According to still another aspect of the embodiments of the present application, a computer device is also provided, which comprises a processor configured to execute a program, wherein the program, when executed, performs any one of the video analysis methods.

[0016] In the embodiments of the present application, the camera is mounted on the NVR, the NVR is capable of detecting a detection event, and the occurrence of the detection event detected by the NVR is sent to the video analysis service module, so that the video analysis service module can find a video segment related to the detection event in a video storage platform storing complete video data recorded by the camera, acquire the video segment, and perform video analysis on the video segment, thereby achieving the purpose of using a small amount of computing process to perform video analysis, and realizing the technical effect of reducing the cost of video analysis operation, and further solving the technical problem of high cost of video analysis operation caused by performing video analysis on complete video recorded by the camera in the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are included to provide a further understanding of the present application and are incorporated in and constitute a part of this application, illustrate embodiments of the present application and serve to explain the principles of the present application, and do not limit the present application in any manner. In the drawings:

[0018] Figure 1A hardware structure block diagram of a computer terminal for implementing a video analysis method is shown;

[0019] Figure 2 A flowchart of a video analysis method one according to an embodiment of the present application is shown;

[0020] Figure 3 A schematic diagram of a running flow of a detected event receiving module according to an optional embodiment of the present application is shown;

[0021] Figure 4 A schematic diagram of a running flow of an intelligent scheduling module according to an optional embodiment of the present application is shown;

[0022] Figure 5 A schematic diagram of a pseudo code of an updating resource dictionary according to an optional embodiment of the present application is shown;

[0023] Figure 6 A schematic diagram of a running flow of a video analysis module according to an optional embodiment of the present application is shown;

[0024] Figure 7 A schematic diagram of a pseudo code of a video analysis according to an optional embodiment of the present application is shown;

[0025] Figure 8 A flowchart of a video analysis method two according to an embodiment of the present application is shown;

[0026] Figure 9 A structure block diagram of a video analysis system according to an embodiment of the present application is shown;

[0027] Figure 10 A flowchart of a video analysis device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0028] In order to make the personnel in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by the personnel in the art without creative labor should belong to the protection scope of the present application.

[0029] It should be noted that the terms "first", "second", and the like in the description and claims of the application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged, where appropriate, so that the embodiments of the application described herein can be carried out in other than the order shown or described herein. Furthermore, the terms "comprise" and "have", and any variations thereof, are intended to cover non-exclusive inclusion, for example, processes, methods, systems, products, or devices that include a series of steps or units are not necessarily limited to those clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0030] First, some of the nouns or terms appearing in the description of the embodiments of the application are applicable to the following explanations:

[0031] The network video recorder (NVR) is the storage and forwarding part of the network video monitoring system, which works with the video encoder or network camera.

[0032] According to the embodiments of the application, a method embodiment of video analysis is provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in a different order.

[0033] The method embodiment provided by the first embodiment of the application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 A hardware structure block diagram of a computer terminal for implementing the video analysis method is shown. As shown in Figure 1 The computer terminal 10 can include one or more processors (the processor can include but is not limited to a microprocessor MCU or a programmable logic device FPGA processing device), a memory 104 for storing data. In addition, it can also include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. Those skilled in the art can understand that Figure 1 The structure shown is only schematic, which does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 can include more or less components than those shown in Figure 1 or have a different configuration than Figure 1 shown.

[0034] It should be noted that the one or more processors and / or other data processing circuitry described above can be referred to herein generally as "data processing circuitry". The data processing circuitry can be embodied in whole or in part as software, hardware, firmware, or any combination thereof. In addition, the data processing circuitry can be a single standalone processing module, or incorporated in whole or in part within any one of the other elements of the computer terminal 10. As referred to in the embodiments of the present application, the data processing circuitry functions as a processor to control, for example, the selection of the variable resistance terminal path connected to the interface.

[0035] The memory 104 can be used to store software programs of application software and modules, such as program instructions / data storage means corresponding to the video analysis method of the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, i.e. implements the video analysis method of the application program as described above. The memory 104 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 can further include a memory disposed remotely with respect to the processor, which can be connected to the computer terminal 10 through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0036] The display can be, for example, a touch screen type liquid crystal display (LCD) that can enable a user to interact with the user interface of the computer terminal 10.

[0037] Figure 2 is a flowchart of the video analysis method according to the embodiments of the present application, as shown in Figure 2 The method comprises the following steps:

[0038] In step S202, the detection event receiving module acquires event information of a detection event from a network video recorder (NVR). The NVR is used to detect a detection event occurring in a video recorded by a connected camera, and the event information includes: a time of occurrence of the detection event, NVR information corresponding to the NVR recording the detection event, and camera information corresponding to the camera recording the detection event.

[0039] In the actual application, the demand for video analysis of various industries is increasing, and many video intelligent analysis platforms need to use many cameras to monitor the safety of many areas, but only a small part of the video recorded by the camera will record some events, and the vast majority of the video in other time is meaningless, and the event is the focus of video analysis. The present application provides a video analysis method, which connects the monitoring camera with the network video recorder NVR, the NVR itself has the mobile detection event capability, when the camera shooting screen changes, the mobile detection event capability of the NVR is triggered, the time of the detection event is recorded, and the NVR information corresponding to the detection event of the NVR and the camera information corresponding to the camera recording the detection event are recorded. The detection event receiving module can communicate with the NVR to obtain the event information of the detection event.

[0040] Step S204, the event information is sent to the video analysis service module by the detection event receiving module.

[0041] In this step, after receiving the event information, the detection event receiving module can send the event information to the video analysis service module for further data processing.

[0042] Step S206, the video analysis service module calls the video segment corresponding to the detection event from the video storage platform according to the event information, wherein the video storage platform is used to obtain the complete video data shot by the camera from the NVR and store the complete video data.

[0043] In this step, the complete picture shot by the camera is not directly transmitted to the video analysis server for video analysis, but is stored in the video storage platform. The video analysis service module can determine the NVR that detects the detection event in the video storage platform according to the event information, can determine the camera that shoots the detection event, and can obtain the video segment related to the detection event according to the occurrence time of the detection event.

[0044] Step S208, the video analysis service module processes the video segment to obtain the analysis result of the detection event.

[0045] In this step, after the video analysis service module obtains the video segment related to the detection event, the video segment can be analyzed to obtain the analysis result of the detection event. The method provided by the present application is different from the prior art, which does not need to send the complete video to the video analysis module for complete video analysis, saves the transmission cost and video analysis cost, and improves the video analysis efficiency.

[0046] In the embodiment of the present application, the camera is mounted on the NVR, the NVR can detect the event, the occurrence of the detected event monitored by the NVR is sent to the video analysis service module, the video analysis service module can find the video segment related to the detected event in the video storage platform storing the complete video data shot by the camera, obtain the video segment, perform video analysis on the video segment, and achieve the purpose of using small operation process for video analysis, thereby realizing the technical effect of reducing the cost of video analysis operation, and further solving the technical problem of high cost of video analysis operation caused by video analysis on the complete video shot by the camera in the prior art.

[0047] As an optional embodiment, the detection event receiving module sends the event information to the video analysis service module, including: the detection event receiving module sends the event information to the intelligent scheduling module; the intelligent scheduling module selects a target computing node capable of analyzing the detected event from a plurality of servers included in the video analysis service module according to the computing resource state of the video analysis service module; and the intelligent scheduling module sends the event information to the target computing node.

[0048] Optionally, after the detection event receiving module receives the event information, the event information can be sent to the intelligent scheduling module, and the intelligent scheduling module can reasonably arrange and schedule the analysis calculation of the video analysis service module according to the operation of the video analysis service module. The intelligent scheduling module can select a relatively idle target computing node capable of analyzing the detected event from a plurality of servers in the video analysis service module according to the computing resource state of the video analysis service module, and arrange the target computing node to perform analysis calculation of the detected event. The computing resource state can be a computing capability information dictionary of the video analysis service module, and the computing capability information dictionary includes information of all servers registered in the intelligent scheduling module by the video service module, including parallel capability, used resources and IP address of each server.

[0049] As an optional embodiment, the intelligent scheduling module selects a target computing node capable of analyzing the detected event from a plurality of servers included in the video analysis service module according to the computing resource state of the video analysis service module, including: the intelligent scheduling module determines a target server capable of analyzing the detected event according to the computing resource state, and determines a target computing node analyzing the detected event from the target server, wherein the target server is one of a plurality of GPU servers registered in the intelligent scheduling module by the video analysis service module, any one GPU server includes a plurality of computing nodes, and the computing resource state includes IP addresses of the plurality of GPU servers and node resource occupation states of any one GPU server.

[0050] Optionally, the intelligent scheduling module can determine a server with more free computing resources among the servers registered in the intelligent scheduling module as a target server, and make the target server complete the task of analyzing the current detection event. The target server can be a graphics card server. In fact, the computing nodes in the graphics card server, i.e., the graphics cards in the server, perform the video analysis computation. Therefore, the target computing node for analyzing the current detection event can be determined among the computing nodes included in the target server, and the target computing node can obtain the video segment to be analyzed from the video storage platform and perform video analysis.

[0051] As an optional embodiment, the video analysis service module can call the video segment corresponding to the detection event from the video storage platform according to the event information, including: the video analysis service module generates an event calling instruction according to the event information, wherein the event calling instruction includes: event video time range, NVR information and camera information, the event video time range includes the occurrence time of the detection event; the video analysis service module sends the event calling instruction to the video storage platform; and the video analysis service module receives the video segment returned by the video storage platform, wherein the time range of the video segment matches the event video time range, and the video segment matches the NVR corresponding to the NVR information and the camera corresponding to the camera information respectively.

[0052] Optionally, the video analysis service module can generate an event calling instruction for calling the video segment from the video storage platform according to the event information. The event calling instruction can indicate the event range of the video segment to be called, the NVR that detects the detection event, and the camera that captures the detection event, so that the target computing node can successfully call the video including the detection event from the video storage platform. The event video time range can be one minute, and the event trigger time can be taken as the time midpoint of the event video time range.

[0053] As a specific embodiment, Figure 3 is a schematic diagram of the running process of the detection event receiving module according to an optional embodiment of the present application, as shown in Figure 3As shown, the camera is mounted on the NVR, one NVR mounts multiple cameras, when the number of cameras is large, multiple NVRs are needed to mount all the cameras; the detection event receiving module realizes real-time mobile detection event receiving of the camera full coverage by connecting the mobile detection event capability of the NVR; the information (EventInfo) contained in each mobile detection event includes: event occurrence time (EventTime), triggering NVR device information (NVR IP, name, device code) and triggering camera device information (camera IP, name, device code); when receiving the mobile detection event triggered by the NVR, the event information (EventInfo) is sent to the moveEventQueue message queue middleware (such as Kafka).

[0054] The analysis intelligent scheduling module connects the detection event module, is responsible for obtaining the detection event needing to be processed from the moveEventQueue; the south connects the video analysis service module, is responsible for the analysis server dynamic registration discovery function, the analysis server computing ability acquisition function and the analysis intelligent scheduling function; Figure 4 The schematic diagram of the running process of the intelligent scheduling module provided by the optional embodiment of the application is as shown in Figure 4 As shown, the intelligent scheduling module maintains the computing ability information dictionary (abilityDict) of all analysis servers, as the basis for analysis intelligent scheduling, the dictionary maintains all the registered servers, including parallel ability (max_parallel), used resources (used), IP address and the like; when the moveEventQueue has a task needing to be processed, the intelligent scheduling module selects a certain computing node according to the current computing resource usage (abilityDict), when the task is issued to the node for analysis, the resource dictionary (abilityDict) is updated, Figure 5 The schematic diagram of the pseudo code for updating the resource dictionary provided by the optional embodiment of the application is as shown in Figure 5 When the video segment corresponding to the detection event processed by Server1 is processed, the analysis scheduling module will update abilityDict[Server1][“used”]-=1 in time; # recycle and release computing resources; since the video analysis module managed by the scheduling module includes multiple computing nodes, each computing node supports multiple parallel analysis, because the scheduling module supports max_parallel*N detection event tasks to be simultaneously and parallelly issued for execution, N is the number of computing nodes, and max_parallel is the maximum number of paths supported by a single node.

[0055]

[0056]

[0057] Figure 6 is a schematic diagram of a video analysis module running flow provided according to an optional embodiment of the present application, as shown in Figure 6 The video analysis service module is a video analysis computing resource pool composed of one or more servers, interacts with the analysis scheduling module, and can dynamically increase or decrease the analysis servers according to actual needs; the video analysis service module listens to the mobile detection analysis tasks issued by the analysis scheduling module, obtains the event time points and NVR and camera IP information of the tasks; according to the received mobile detection event information, the corresponding camera video stream rtsp addresses before and after 30 seconds are obtained from the video access and storage platform; a sub-thread is started to decode the video stream, complete the video analysis, and store the structured data; taking a face recognition application as an example, Figure 7 is a schematic diagram of a pseudo code of video analysis provided according to an optional embodiment of the present application, as shown in Figure 7 When a detection event is processed, the analysis scheduling module is notified to update the computing resource usage of the analysis server, and the actual release of the related CPU, GPU and memory resources is also performed.

[0058] The present application uses an intelligent analysis scheduling method based on mobile detection event driving, and has the following advantages and effects: when the number of cameras is large (such as more than 5000), only 1 / 20 of the GPU server quantity of the existing online analysis technology is needed to realize full coverage of all camera analysis; when the number of cameras is small, the same number of GPU servers can execute more analysis tasks, reducing the hardware cost of the GPU servers; through NVR mobile detection event driving, the time points with analysis value are accurately positioned, and the related video end is extracted to schedule analysis, avoiding waste of computing resources; the discovery and registration mechanism between the analysis server and the scheduling module designed and implemented by the present application has good dynamic scalability, and can realize on-demand allocation of computing resources to the existing system; the existing uninterrupted online real-time video analysis scheme will cause video stream transmission to be not smooth when the number of concurrent analysis cameras is large, resulting in frame loss and lag, affecting the quality of video analysis, while the method provided by the present application can improve the quality of video analysis; the present application uses a message queue to buffer the mobile detection event analysis tasks to be processed, when the peak time, the mobile detection event quantity is greater than the support capacity, the queue is executed, so that each video segment data of the analysis can be smoothly transmitted without frame loss and lag, thereby ensuring the analysis quality and improving the effective utilization rate of network bandwidth.

[0059] Figure 8 is a flowchart of a video analysis method two provided according to an embodiment of the present application, as shown in Figure 8 The method comprises the following steps:

[0060] Step S802, receiving the event information of the detection event sent by the detection event receiving module, wherein the event information is the event information of the detection event obtained by the detection event receiving module from the network video recorder (NVR), the NVR is used to detect the detection event occurred in the video recorded by the connected camera, and the event information includes the occurrence time of the detection event, the NVR information corresponding to the NVR recording the detection event, and the camera information corresponding to the camera recording the detection event.

[0061] In this step, the monitoring camera is connected with the network video recorder (NVR), the NVR itself has the mobile detection event capability, when the picture recorded by the camera changes, the mobile detection event capability of the NVR is triggered, the time when the detection event occurs is recorded, and the NVR information corresponding to the NVR recording the detection event and the camera information corresponding to the camera recording the detection event are recorded. The detection event receiving module can communicate with the NVR to obtain the event information of the detection event. After the detection event receiving module receives the event information, the event information can be sent to the intelligent scheduling module, and the intelligent scheduling module can reasonably arrange and schedule the analysis and calculation of the video analysis service module according to the running state of the video analysis service module.

[0062] Step S804, according to the computing resource state of the video analysis service module, screening the target computing node capable of analyzing the detection event from the multiple servers included in the video analysis service module.

[0063] In this step, the intelligent scheduling module can select the relatively idle target computing node capable of analyzing the detection event from the multiple servers in the video analysis service module according to the computing resource state of the video analysis service module, and arrange the target computing node to analyze and calculate the detection event. The computing resource state can be a computing capability information dictionary of the video analysis service module, and the computing capability information dictionary includes the information of all servers registered in the intelligent scheduling module by the video service module, including the parallel capability, used resource and IP address of each server.

[0064] Step S806, sending the event information to the target computing node, wherein the target computing node is used to call the video segment corresponding to the detection event from the video storage platform according to the event information, and is also used to process the video segment to obtain the analysis result of the detection event, and the video storage platform is used to obtain the complete video data recorded by the camera from the NVR and store the complete video data.

[0065] In this step, the complete picture captured by the camera is not directly transmitted to the video analysis server for video analysis, but is stored in the video storage platform. The target computing node can determine the NVR that detects the detection event in the video storage platform according to the event information, can determine the camera that captures the detection event, and can obtain the video segment related to the detection event according to the occurrence time of the detection event. After the video analysis service module obtains the video segment related to the detection event, the video segment can be analyzed to obtain the analysis result of the detection event. The method provided by the application is different from the prior art, and the complete video captured by the camera does not need to be sent to the video analysis module for complete video analysis, thereby saving transmission cost and video analysis cost and improving video analysis efficiency.

[0066] In the embodiment of the application, the camera is mounted on the NVR, the NVR can detect events, and the occurrence of the detection event detected by the NVR is sent to the video analysis service module, so that the video analysis service module can find the video segment related to the detection event in the video storage platform storing the complete video data captured by the camera, and obtain the video segment, and perform video analysis on the video segment, thereby achieving the purpose of using a small amount of calculation to perform video analysis, thereby realizing the technical effect of reducing the cost of video analysis operation, and further solving the technical problem of high cost of video analysis operation caused by the complete video captured by the camera in the prior art.

[0067] Optionally, the target computing node capable of analyzing the detection event is selected from the plurality of servers included in the video analysis service module according to the computing resource state of the video analysis service module, including: determining the target server capable of analyzing the detection event according to the computing resource state, and determining the target computing node analyzing the detection event from the target server, wherein the target server is one of the plurality of GPU servers registered in the intelligent scheduling module of the video analysis service module, any one GPU server includes a plurality of computing nodes, and the computing resource state includes the IP address of each of the plurality of GPU servers and the node resource occupation state of any one GPU server.

[0068] Optionally, the intelligent scheduling module can determine the target server as the server that is relatively idle in the servers registered in the intelligent scheduling module according to the computing capability information dictionary, so that the target server completes the task of analyzing the detection event. The target server can be a GPU server, and the actual video analysis calculation is performed by the computing node in the GPU server, that is, the GPU in the server. Therefore, the target computing node analyzing the detection event can be determined in the computing node included in the target server, so that the target computing node obtains the video segment to be analyzed from the video storage platform and performs video analysis.

[0069] It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all described as a combination of a series of actions, but those skilled in the art should know that the present application is not limited by the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0070] Through the description of the above embodiments, those skilled in the art can clearly understand that the video analysis method according to the above embodiments can be realized by means of software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the method described in each embodiment of the present application.

[0071] According to the embodiments of the present application, a system for implementing the above-mentioned video analysis method is also provided, Figure 9 is a structural block diagram of a video analysis system provided by the embodiments of the present application, as Figure 9 shown, the video analysis system includes a detection event receiving module 92, a video analysis service module 94 and a video storage platform 96, which will be described below.

[0072] The detection event receiving module 92 is configured to obtain event information of a detection event from a network video recorder (NVR), and is also configured to send the event information to the video analysis service module, wherein the NVR is configured to detect a detection event occurring in a video recorded by a connected camera, and the event information includes: a time of occurrence of the detection event, NVR information corresponding to the NVR recording the detection event, and camera information corresponding to the camera recording the detection event.

[0073] The video analysis service module 94 is connected with the detection event receiving module 92, and is configured to call a video segment corresponding to the detection event from the video storage platform according to the event information, and is also configured to process the video segment to obtain an analysis result of the detection event.

[0074] The video storage platform 96 is connected with the video analysis service module 94, and is configured to obtain complete video data recorded by the camera from the NVR and store the complete video data.

[0075] It should be noted that the above detection event receiving module 92, video analysis service module 94 and video storage platform 96 correspond to steps S202 to S208 in the embodiment, and the instances and application scenarios realized by the plurality of modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above embodiment. It should be noted that the above modules can be run in the computer terminal 10 provided in the embodiment as part of the device.

[0076] According to the embodiment of the application, a device for implementing the above-mentioned video analysis method two is also provided, Figure 10 is a flow diagram of a video analysis device provided by the embodiment of the application, as Figure 10 shown, the video analysis device comprises a receiving unit 12, a screening unit 14 and a sending unit 16, which will be described below.

[0077] The receiving unit 12 is configured to receive event information of a detection event sent by a detection event receiving module, wherein the event information is event information of the detection event acquired by the detection event receiving module from a network video recorder (NVR), the NVR is configured to detect a detection event occurring in a video recorded by a connected camera, and the event information comprises a time of occurrence of the detection event, NVR information corresponding to the NVR recording the detection event, and camera information corresponding to the camera recording the detection event.

[0078] The screening unit 14 is connected with the receiving unit 12 and is configured to select a target computing node capable of analyzing the detection event from a plurality of servers included in a video analysis service module according to a computing resource state of the video analysis service module.

[0079] The sending unit 16 is connected with the screening unit 14 and is configured to send the event information to the target computing node, wherein the target computing node is configured to call a video segment corresponding to the detection event from a video storage platform according to the event information, and is further configured to process the video segment to obtain an analysis result of the detection event, and the video storage platform is configured to acquire complete video data recorded by the camera from the NVR and store the complete video data.

[0080] It should be noted that the above receiving unit 12, screening unit 14 and sending unit 16 correspond to steps S802 to S806 in the embodiment, and the instances and application scenarios realized by the plurality of modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above embodiment. It should be noted that the above modules can be run in the computer terminal 10 provided in the embodiment as part of the device.

[0081] The embodiment of the application can provide a computer device. Optionally, in the embodiment, the computer device can be located in at least one network device of a plurality of network devices of a computer network. The computer device comprises a memory and a processor.

[0082] The memory can be used to store software programs and modules, such as program instructions / modules corresponding to the video analysis method and device in the embodiments of the present application. The processor executes various functions and data processing by running the software programs and modules stored in the memory, that is, implements the video analysis method described above. The memory can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory can further include a memory remotely arranged with respect to the processor, which can be connected to the computer terminal through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0083] The processor can call information and application programs stored in the memory through the transmission device to execute the following steps: the detection event receiving module obtains event information of a detection event from a network video recorder (NVR), wherein the NVR is used to detect a detection event occurring in a video recorded by a connected camera, and the event information includes occurrence time of the detection event, NVR information corresponding to the NVR recording the detection event, and camera information corresponding to the camera recording the detection event; the detection event receiving module sends the event information to a video analysis service module; the video analysis service module calls a video segment corresponding to the detection event from a video storage platform according to the event information, wherein the video storage platform is used to obtain complete video data recorded by the camera from the NVR and store the complete video data; and the video analysis service module processes the video segment to obtain an analysis result of the detection event.

[0084] Optionally, the processor can further execute program codes of the following steps: the detection event receiving module sends the event information to the video analysis service module, including: the detection event receiving module sends the event information to an intelligent scheduling module; the intelligent scheduling module selects a target computing node capable of analyzing the detection event from a plurality of servers included in the video analysis service module according to a computing resource state of the video analysis service module; and the intelligent scheduling module sends the event information to the target computing node.

[0085] Optionally, the processor can further execute program codes of the following steps: the intelligent scheduling module screens a target computing node capable of analyzing the detected event from a plurality of servers included in the video analysis service module according to a computing resource state of the video analysis service module, including: the intelligent scheduling module determines a target server capable of analyzing the detected event according to the computing resource state, and determines a target computing node analyzing the detected event from the target server, wherein the target server is one of a plurality of GPU servers registered on the intelligent scheduling module by the video analysis service module, any one GPU server includes a plurality of computing nodes, and the computing resource state includes IP addresses of the plurality of GPU servers and node resource occupation states of any one GPU server.

[0086] Optionally, the processor can further execute program codes of the following steps: the video analysis service module calls a video clip corresponding to the detected event from a video storage platform according to the event information, including: the video analysis service module generates an event calling instruction according to the event information, wherein the event calling instruction includes: an event video time range, NVR information, and camera information, and the event video time range includes a time of occurrence of the detected event; the video analysis service module sends the event calling instruction to the video storage platform; and the video analysis service module receives a video clip returned by the video storage platform, wherein a time range of the video clip matches the event video time range, and the video clip matches an NVR corresponding to the NVR information and a camera corresponding to the camera information respectively.

[0087] Optionally, the processor can further execute program codes of the following steps: receiving event information of the detected event sent by the detected event receiving module, wherein the event information is event information of the detected event acquired by the detected event receiving module from a network video recorder (NVR), the NVR is used for detecting the detected event appearing in a video recorded by a connected camera, and the event information includes: a time of occurrence of the detected event, NVR information corresponding to an NVR recording the detected event, and camera information corresponding to a camera recording the detected event; screening a target computing node capable of analyzing the detected event from a plurality of servers included in the video analysis service module according to a computing resource state of the video analysis service module; and sending the event information to the target computing node, wherein the target computing node is used for calling a video clip corresponding to the detected event from a video storage platform according to the event information, and is also used for processing the video clip to obtain an analysis result of the detected event, and the video storage platform is used for acquiring complete video data recorded by the camera from the NVR and storing the complete video data.

[0088] Optionally, the processor can further execute program codes of the following steps: filtering, according to the computing resource state of the video analysis service module, a target computing node capable of analyzing the detected event from a plurality of servers included in the video analysis service module, comprising: determining, according to the computing resource state, a target server capable of analyzing the detected event, and determining, from the target server, a target computing node for analyzing the detected event, wherein the target server is one of a plurality of GPU servers registered by the video analysis service module on the intelligent scheduling module, any one of the GPU servers includes a plurality of computing nodes, and the computing resource state includes IP addresses of the plurality of GPU servers and node resource occupation states of any one of the GPU servers.

[0089] In the embodiment of the present application, the camera is mounted on the NVR, the NVR can detect events, the occurrence of the detected event monitored by the NVR is sent to the video analysis service module, the video analysis service module can find a video segment related to the detected event in the video storage platform storing complete video data shot by the camera, obtain the video segment, perform video analysis on the video segment, and achieve the purpose of using a small amount of computation to perform video analysis, thereby realizing the technical effect of reducing the cost of video analysis operation, and further solving the technical problem of high cost of video analysis operation caused by performing video analysis on complete video shot by the camera in the prior art.

[0090] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiments can be completed by instructing the hardware related to the terminal device by a program, and the program can be stored in a non-volatile storage medium, which can include a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0091] The embodiment of the present application further provides a non-volatile storage medium. Optionally, in the embodiment, the non-volatile storage medium can be used to save the program codes executed by the video analysis method provided in the above-mentioned embodiments.

[0092] Optionally, in the embodiment, the non-volatile storage medium can be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0093] Optionally, in the embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: the detection event receiving module acquires event information of a detection event from a network video recorder (NVR), wherein the NVR is configured to detect a detection event occurring in a video recorded by a connected camera, and the event information includes a time of occurrence of the detection event, NVR information corresponding to the NVR recording the detection event, and camera information corresponding to the camera recording the detection event; the detection event receiving module sends the event information to the video analysis service module; the video analysis service module calls a video segment corresponding to the detection event from a video storage platform according to the event information, wherein the video storage platform is configured to acquire complete video data recorded by the camera from the NVR and store the complete video data; and the video analysis service module processes the video segment to obtain an analysis result of the detection event.

[0094] Optionally, in the embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: the detection event receiving module sends the event information to the video analysis service module, including: the detection event receiving module sends the event information to an intelligent scheduling module; the intelligent scheduling module filters a target computing node capable of analyzing the detection event from a plurality of servers included in the video analysis service module according to a computing resource state of the video analysis service module; and the intelligent scheduling module sends the event information to the target computing node.

[0095] Optionally, in the embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: the intelligent scheduling module filters a target computing node capable of analyzing the detection event from a plurality of servers included in the video analysis service module according to a computing resource state of the video analysis service module, including: the intelligent scheduling module determines a target server capable of analyzing the detection event according to the computing resource state, and determines the target computing node analyzing the detection event from the target server, wherein the target server is one of a plurality of graphics card servers registered on the intelligent scheduling module by the video analysis service module, any one of the graphics card servers includes a plurality of computing nodes, and the computing resource state includes IP addresses of the plurality of graphics card servers and a node resource occupation state of any one of the graphics card servers.

[0096] Optionally, in the embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: the video analysis service module calls a video clip corresponding to the detected event from a video storage platform according to the event information, comprising: the video analysis service module generates an event calling instruction according to the event information, wherein the event calling instruction comprises: event video time range, NVR information and camera information, the event video time range comprises the occurrence time of the detected event; the video analysis service module sends the event calling instruction to the video storage platform; the video analysis service module receives the video clip returned by the video storage platform, wherein the time range of the video clip matches the event video time range, and the video clip matches the NVR corresponding to the NVR information and the camera corresponding to the camera information respectively.

[0097] Optionally, in the embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: receiving event information of a detected event sent by a detected event receiving module, wherein the event information is the event information of the detected event obtained by the detected event receiving module from a network video recorder (NVR), the NVR is used to detect a detected event occurring in a video recorded by a connected camera, and the event information comprises: occurrence time of the detected event, NVR information corresponding to an NVR recording the detected event, and camera information corresponding to a camera recording the detected event; according to the computing resource state of the video analysis service module, a target computing node capable of analyzing the detected event is selected from a plurality of servers included in the video analysis service module; and the event information is sent to the target computing node, wherein the target computing node is used to call a video clip corresponding to the detected event from a video storage platform according to the event information, and is also used to process the video clip to obtain an analysis result of the detected event, and the video storage platform is used to obtain complete video data recorded by the camera from the NVR and store the complete video data.

[0098] Optionally, in the embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: according to the computing resource state of the video analysis service module, a target computing node capable of analyzing the detected event is selected from a plurality of servers included in the video analysis service module, comprising: according to the computing resource state, a target server capable of analyzing the detected event is determined, and a target computing node analyzing the detected event is determined from the target server, wherein the target server is one of a plurality of GPU servers registered on an intelligent scheduling module by the video analysis service module, any one GPU server comprises a plurality of computing nodes, and the computing resource state comprises IP addresses of the plurality of GPU servers and node resource occupation states of any one GPU server.

[0099] The above-mentioned serial numbers of the embodiments of the application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0100] In the above-mentioned embodiments of the present application, the description of each embodiment is focused on, and the part not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0101] In several embodiments provided in the present application, it should be understood that the disclosed technical content can be implemented by other ways. Among them, the above-mentioned device embodiments are only schematic, for example, the division of the units can be a logical function division, and actual implementation can have another division way, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or modules shown or discussed can be indirect coupling or communication connection through some interfaces, units or modules, which can be electrical or other forms.

[0102] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed to multiple units. Part or all of the units can be selected to achieve the purpose of the embodiment scheme according to actual needs.

[0103] In addition, each functional unit in each embodiment of the present application can be integrated in a processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0104] The integrated unit, if realized in the form of software functional unit and sold or used as an independent product, can be stored in a non-volatile storage medium. Based on such understanding, the technical solutions of the present application essentially or the part of the prior art that makes a contribution or the whole or part of the technical solutions can be embodied in the form of software product, which is stored in a storage medium and includes a plurality of instructions for making a computer device (which can be a personal computer, a server or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and various program code storage media.

[0105] The above-mentioned is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be regarded as the protection scope of the present application.

Claims

1. A video analysis method, characterized in that, include: The detection event receiving module obtains event information of the detection event from the network video recorder (NVR). The NVR is used to detect the detection event appearing in the video recorded by the connected camera. The event information includes: the occurrence time of the detection event, the NVR information corresponding to the NVR that recorded the detection event, and the camera information corresponding to the camera that recorded the detection event. The event receiving module sends the event information to the video analysis service module; The video analysis service module retrieves the video segment corresponding to the detected event from the video storage platform based on the event information. The video storage platform is used to obtain and store the complete video data captured by the camera from the NVR. The video analysis service module processes the video segment to obtain the analysis results of the detected event.

2. The method according to claim 1, characterized in that, The event detection receiving module sends the event information to the video analysis service module, including: The event receiving module sends the event information to the intelligent scheduling module; The intelligent scheduling module selects target computing nodes capable of analyzing the detected events from the multiple servers included in the video analysis service module based on the computing resource status of the video analysis service module. The intelligent scheduling module sends the event information to the target computing node.

3. The method according to claim 2, characterized in that, The intelligent scheduling module, based on the computing resource status of the video analysis service module, selects target computing nodes capable of analyzing the detected events from the multiple servers included in the video analysis service module, including: The intelligent scheduling module determines the target server capable of analyzing the detected event based on the computing resource status, and determines the target computing node for analyzing the detected event from the target server. The target server is one of multiple graphics card servers registered on the intelligent scheduling module by the video analysis service module. Each graphics card server includes multiple computing nodes. The computing resource status includes the IP addresses of each of the multiple graphics card servers and the node resource occupancy status of any graphics card server.

4. The method according to claim 1, characterized in that, The video analysis service module retrieves the video segment corresponding to the detected event from the video storage platform based on the event information, including: The video analysis service module generates an event invocation instruction based on the event information. The event invocation instruction includes: the event video time range, the NVR information, and the camera information. The event video time range includes the occurrence time of the detected event. The video analytics service module sends the event invocation command to the video storage platform; The video analysis service module receives the video segments returned by the video storage platform, wherein the time range of the video segments matches the time range of the event video, and the video segments are matched with the NVR corresponding to the NVR information and the camera corresponding to the camera information, respectively.

5. A video analysis method, characterized in that, include: The system receives event information of a detected event sent by a detection event receiving module. The event information is obtained by the detection event receiving module from a network video recorder (NVR). The NVR is used to detect the detected event in the video recorded by a connected camera. The event information includes: the occurrence time of the detected event, the NVR information corresponding to the NVR that recorded the detected event, and the camera information corresponding to the camera that recorded the detected event. Based on the computing resource status of the video analysis service module, target computing nodes capable of analyzing the detected events are selected from the multiple servers included in the video analysis service module. The event information is sent to the target computing node, wherein the target computing node is used to retrieve the video segment corresponding to the detected event from the video storage platform according to the event information, and is also used to process the video segment to obtain the analysis result of the detected event. The video storage platform is used to obtain the complete video data captured by the camera from the NVR and store the complete video data.

6. The method according to claim 5, characterized in that, Based on the computing resource status of the video analysis service module, the process of selecting target computing nodes capable of analyzing the detected events from the multiple servers included in the video analysis service module includes: Based on the computing resource status, a target server capable of analyzing the detected event is determined, and a target computing node for analyzing the detected event is determined from the target server. The target server is one of multiple graphics card servers registered by the video analysis service module on the intelligent scheduling module. Each graphics card server includes multiple computing nodes. The computing resource status includes the IP addresses of each of the multiple graphics card servers and the node resource occupancy status of any one graphics card server.

7. A video analysis system, characterized in that, include: The detection event receiving module is used to obtain event information of the detection event from the network video recorder (NVR) and to send the event information to the video analysis service module. The NVR is used to detect the detection event in the video recorded by the connected camera. The event information includes: the occurrence time of the detection event, the NVR information corresponding to the NVR that recorded the detection event, and the camera information corresponding to the camera that recorded the detection event. The video analysis service module is used to retrieve the video segment corresponding to the detected event from the video storage platform according to the event information, and is also used to process the video segment to obtain the analysis result of the detected event; The video storage platform is used to acquire and store the complete video data captured by the camera from the NVR.

8. A video analysis device, characterized in that, include: A receiving unit is configured to receive event information of a detected event sent by a detection event receiving module. The event information is event information of a detected event obtained by the detection event receiving module from a network video recorder (NVR). The NVR is used to detect the detected event appearing in the video recorded by a connected camera. The event information includes: the occurrence time of the detected event, NVR information corresponding to the NVR that recorded the detected event, and camera information corresponding to the camera that recorded the detected event. The filtering unit is used to filter out target computing nodes capable of analyzing the detected events from the multiple servers included in the video analysis service module based on the computing resource status of the video analysis service module. The sending unit is used to send the event information to the target computing node, wherein the target computing node is used to retrieve the video segment corresponding to the detected event from the video storage platform according to the event information, and is also used to process the video segment to obtain the analysis result of the detected event, and the video storage platform is used to obtain the complete video data captured by the camera from the NVR and store the complete video data.

9. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored program, wherein, when the program is executed, it controls the device containing the non-volatile storage medium to perform the video analysis method according to any one of claims 1 to 6.

10. A computer device, characterized in that, include: Memory and processor The memory stores computer programs; The processor is configured to execute a computer program stored in the memory, wherein when the computer program is executed, the processor performs the video analysis method according to any one of claims 1 to 6.

Images