Artificial intelligence-based monitoring video analysis method, system and storage medium

Abstract

The application discloses an artificial intelligence-based monitoring video analysis method, a monitoring video analysis system and a storage medium. When an abnormal person is detected in an entrance area, a first moving track of the abnormal person is determined based on a first monitoring video corresponding to the entrance area and a pre-stored cell three-dimensional model. Then, a target camera device is determined according to the first moving track, and a second monitoring video collected by the target camera device is acquired. A second moving track of the abnormal person is determined according to the second monitoring video. If the second moving track overlaps with a preset sensitive area, risk early warning information is output. Since the risk early warning information can be automatically output, the monitoring video analysis efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of communication, and in particular to a monitoring video analysis method based on artificial intelligence, a monitoring video analysis system and a storage medium. BACKGROUND

[0002] In the scene of urban residential area, the monitoring video can realize the dead angle-free coverage of the key area, and strengthen the monitoring of the key area, to ensure the comprehensiveness and pertinence of safety prevention. The security personnel can quickly find potential security risks according to the monitoring video without on-site patrol. However, in the related scheme, the observation and analysis of the monitoring video seriously depend on the staff, resulting in a large amount of manpower consumption, and thus there is the defect of low monitoring video analysis efficiency. SUMMARY

[0003] The main purpose of the present application is to provide a monitoring video analysis method based on artificial intelligence, a monitoring video analysis system and a storage medium, which aims to solve the technical problem of low monitoring video analysis efficiency in the related art.

[0004] To achieve the above-mentioned purpose, the embodiment of the present application provides a monitoring video analysis method based on artificial intelligence, which comprises:

[0005] When an abnormal person is detected in an entrance area, a first moving track of the abnormal person is determined based on a first monitoring video corresponding to the entrance area and a pre-stored three-dimensional model of the residential area;

[0006] According to the first moving track, a target camera device is determined;

[0007] A second monitoring video collected by the target camera device is acquired, and a second moving track of the abnormal person is determined according to the second monitoring video;

[0008] If the second moving track overlaps with a preset sensitive area, a risk warning information is output.

[0009] In the embodiment of the present application, after the step of determining the second moving track of the abnormal person according to the second monitoring video, the method further comprises:

[0010] If the second moving track does not overlap with the preset sensitive area, the first moving track is updated according to the second moving track, and the step of determining the target camera device according to the first moving track is executed.

[0011] In the embodiment of the present application, before the step of outputting the risk warning information, the method further comprises:

[0012] If the second movement track overlaps with the preset sensitive area, a type of the target sensitive area overlapping with the second movement track is determined, and a target analysis model is determined based on the type;

[0013] The third monitoring video corresponding to the target sensitive area is sent to the target analysis model to obtain an abnormal analysis feedback result of the target analysis model, wherein different analysis models are used for identification of different abnormal behaviors;

[0014] The level and content of the risk warning information are determined according to the abnormal analysis feedback result.

[0015] In the embodiments of the present application, before the step of sending the monitoring video corresponding to the target sensitive area to the target analysis model to obtain an abnormal analysis feedback result of the target analysis model, the method further comprises:

[0016] Based on the first monitoring video, an identification feature of the abnormal personnel is obtained;

[0017] According to the identification feature, irrelevant data in the third monitoring video is eliminated.

[0018] In the embodiments of the present application, before the step of determining the first movement track of the abnormal personnel based on the first monitoring video corresponding to the entrance area and the pre-stored cell three-dimensional model when the abnormal personnel is detected in the entrance area, the method further comprises:

[0019] The first monitoring video is obtained, and a personnel feature in the monitoring video is extracted;

[0020] The personnel feature is input into a pre-trained classification model to obtain a classification result;

[0021] If the classification result is an abnormal personnel, it is determined that the abnormal personnel is detected in the entrance area.

[0022] In the embodiments of the present application, the method further comprises:

[0023] According to network public data, a label-feature pair corresponding to a first abnormal personnel is generated;

[0024] According to blacklist data, a label-feature pair corresponding to a second abnormal personnel is generated;

[0025] According to a monitoring video annotation result, a label-feature pair corresponding to a third abnormal personnel is generated;

[0026] According to the label-feature pairs corresponding to the first abnormal personnel, the second abnormal personnel and the third abnormal personnel, the classification model is trained.

[0027] In the embodiments of the present application, after the step of acquiring the second monitoring video collected by the target camera and determining the second moving track of the abnormal personnel according to the second monitoring video, the method further comprises:

[0028] determining a target period according to the time point at which the abnormal personnel arrives at the entrance area;

[0029] acquiring a third moving track corresponding to a normal personnel in the target period;

[0030] if the second moving track matches the third moving track, sending a warning information based on the communication mode associated with the normal personnel.

[0031] In the embodiments of the present application, before the step of determining the first moving track of the abnormal personnel based on the first monitoring video corresponding to the entrance area and the pre-stored three-dimensional model of the cell, the method further comprises:

[0032] when detecting that the automatic gate corresponding to the entrance area is opened, determining an opening mode;

[0033] if the opening mode is manual opening by a security personnel or remote opening by a resident, it is determined that the abnormal personnel is detected at the entrance area.

[0034] The embodiments of the present application also provide a monitoring video analysis system, which comprises:

[0035] an acquisition module configured to, when an abnormal personnel is detected at an entrance area, determine a first moving track of the abnormal personnel based on a first monitoring video corresponding to the entrance area and a pre-stored three-dimensional model of a cell;

[0036] a determination module configured to determine a target camera according to the first moving track;

[0037] a generation module configured to acquire a second monitoring video collected by the target camera and determine a second moving track of the abnormal personnel according to the second monitoring video;

[0038] an output module configured to output a risk warning information if the second moving track overlaps with a preset sensitive area.

[0039] The embodiments of the present application also provide a storage medium, which is a computer readable storage medium, and a computer program is stored on the storage medium, and the computer program is executed by a processor to implement the steps of the monitoring video analysis method based on artificial intelligence.

[0040] The embodiment of the application discloses a monitoring video analysis method and system based on artificial intelligence. After detecting an abnormal person at an entrance area of a community, the method can determine a first moving track of the abnormal person according to a first monitoring video of the entrance area, then predict a moving direction of the abnormal person in the future according to the first moving track and a three-dimensional model of the community, determine a target camera device based on the moving direction, and then determine a second moving track of the abnormal person based on a second monitoring video collected by the target camera device. Finally, when the second moving track overlaps with a preset sensitive area, risk warning information is output to remind relevant personnel to pay more attention to relevant monitoring videos.

[0041] In the technical scheme provided by the embodiment, on one hand, the abnormal person entering the community can be identified based on the track analysis, and if special attention is needed, the relevant personnel can be prompted to pay more attention to the person through risk warning, without the security personnel actively paying attention to the monitoring video all the time to find the risk. In this way, the efficiency of monitoring video analysis is effectively improved. On the other hand, in the technical scheme provided by the application, the track analysis is only performed when the abnormal person is detected, and in the track determination process, the target camera device is predicted based on the first moving track, so that when the second moving track is determined, only the monitoring video collected by the target camera device needs to be processed. In this way, from the two dimensions of the starting condition and the analysis process, the data content of intelligent analysis is reduced, and the computing overhead can be effectively reduced. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is a flowchart of an embodiment of the monitoring video analysis method based on artificial intelligence involved in the embodiment of the application;

[0043] Figure 2 is a schematic diagram of the position relationship of a monitoring area involved in the embodiment of the application;

[0044] Figure 3 is a data processing architecture diagram for generating a moving track involved in the embodiment of the application;

[0045] Figure 4 is a flowchart of another embodiment of the monitoring video analysis method based on artificial intelligence involved in the embodiment of the application;

[0046] Figure 5 is a structural schematic diagram of the monitoring video analysis device of the application;

[0047] Figure 6 is a modular structure schematic diagram of the monitoring video analysis system of the application.

[0048] The implementation, functional features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are merely illustrative of the present application and should not be used to limit the present application in any manner.

[0050] In the community security scene, the monitoring video can realize the dead angle-free coverage of the key area and strengthen the monitoring of the key area, ensure the comprehensiveness and pertinence of the security protection, and significantly improve the security protection ability of the community, and provide a safer living environment for residents.

[0051] In the community security application scene based on video monitoring, the monitoring system plays a crucial role in pre-warning and post-backtracking. For example, the security personnel in the monitoring room can determine the on-site picture based on the real-time monitoring picture without on-site patrol, so as to pre-warn according to the on-site picture. Or, when events such as theft and pet loss occur, the post-backtracking can be achieved based on the stored monitoring video.

[0052] However, the current monitoring system, whether pre-warning or post-backtracking, depends on the monitoring management personnel to manually understand and analyze the monitoring video. Such a way leads to a large demand for human resources on the one hand, and on the other hand, due to the one-sidedness, subjectivity of manual analysis and uncertainty caused by the length of attention concentration of personnel, the analysis efficiency of the monitoring video is relatively low.

[0053] To solve the above defects existing in the related art, the present application proposes a monitoring video analysis method based on artificial intelligence. After detecting an abnormal person in the entrance area of the community, the method can determine the first moving track of the abnormal person according to the first monitoring video of the entrance area, then predict the moving direction of the abnormal person in the future according to the first moving track and the three-dimensional model of the community, and determine the target camera device based on this, and then determine the second moving track of the abnormal person based on the second monitoring video collected by the target camera device. Finally, when the second moving track overlaps with the preset sensitive area, the risk warning information is output to remind the relevant personnel to improve the attention degree to the relevant monitoring video.

[0054] In the technical solution provided in the embodiment, on one hand, based on trajectory analysis, it can be determined whether the abnormal person entering the community needs special attention, and if so, relevant personnel are prompted to increase attention to the person through risk warning, without the need for security personnel to actively monitor the video at all times to discover risks. In this way, the efficiency of monitoring video analysis is effectively improved. On the other hand, in the technical solution provided in the application, trajectory analysis is performed only when an abnormal person is detected, and in the trajectory determination process, the relevant target camera is predicted based on the first moving trajectory, so that when the second moving trajectory is determined, only the monitoring video collected by the target camera needs to be processed. In terms of both the starting condition and the analysis process, the data content of intelligent analysis is reduced, and the computing overhead can be effectively reduced.

[0055] For ease of understanding, the technical solutions in the embodiments of the application will be described below with reference to the drawings in the embodiments of the application.

[0056] Please refer to Figure 1 In an optional embodiment, the monitoring video analysis method based on artificial intelligence comprises the following steps S10-S30:

[0057] S10: when an abnormal person is detected at an entrance area, based on the first monitoring video corresponding to the entrance area and a pre-stored three-dimensional model of the community, a first moving trajectory of the abnormal person is determined.

[0058] In the embodiment, the data processing unit of the system for executing the monitoring video analysis method based on artificial intelligence can be set locally or in the cloud, and the embodiment does not limit this.

[0059] The above-mentioned entrance area can refer to the entrance area of a residential community, or a monitoring environment. For example, when the scheme is used in a road monitoring system, a road intersection or the starting area of a road section can be customized as the above-mentioned entrance area according to the needs. The embodiment does not make specific limitations. In order for those skilled in the art to more accurately understand the spirit of the scheme, the technical solution provided in the embodiment will be further described in combination with the community security scene. However, the scheme provided in the embodiment is not limited to being used in the community scene, but can also be applied to abnormal behavior analysis and detection of any monitoring coverage area.

[0060] In an optional embodiment, the embodiment can detect whether an abnormal person appears at the entrance area based on the first monitoring video corresponding to the entrance area. If so, the first moving trajectory of the abnormal person at the entrance area is further determined based on the first monitoring video.

[0061] In the embodiment, first, a first monitoring video is acquired, then a personnel feature of a person in the monitoring video is extracted, and the personnel feature is input into a classification model, if the person is an abnormal person, the classification result output by the classification model is an abnormal person. Wherein, the system is provided with a video acquisition module, which can acquire a video stream from a monitoring camera corresponding to an entrance area. Then, by using computer vision technology, the face feature and / or body feature of a human object in the monitoring video are detected and extracted, and input into a pre-trained classification model. The classification model receives the extracted personnel feature, and preliminarily judges whether the person is an abnormal person.

[0062] Exemplarily, the monitoring camera supports the RTSP protocol, so the video acquisition module uses libraries such as OpenCV to acquire a video stream in real time. Then, a deep learning model (such as YOLO, SSD, etc.) is used to detect the person in the video. For the detected face, a face detection algorithm (such as MTCNN, FaceBoxes, etc.) is used to locate the face area, and then a face recognition algorithm (such as DeepFace, FaceNet, etc.) is used to extract a face feature vector. For the detected body, a pose estimation algorithm (such as OpenPose, DeepLabCut, etc.) can be used to extract body key points, or a convolutional neural network can be used to extract a body feature vector. And the extracted body feature vector or face feature vector is input into the classification model to determine whether the person is a resident of the community or an abnormal person.

[0063] It should be noted that the abnormal person can be positioned as other personnel outside the residents, or different groups can be defined as abnormal persons according to requirements, such as express delivery personnel, visitors, etc. For this, the present embodiment is not limited, and the abnormal person mentioned here is the personnel corresponding to the abnormal sample in the model training process, which is only a training label.

[0064] Optionally, as a training method of a classification model, a label-feature pair corresponding to a first abnormal person can be generated according to network public data, for example, according to the information disclosed by the relevant department, an abnormal person sample label corresponding to the first abnormal person is constructed. For example, a wanted person disclosed on the network can be acquired as the first abnormal person. In addition, a blacklist and a whitelist can also be provided, wherein the design of adding, deleting and modifying personnel in the blacklist and the whitelist can be customized according to requirements, and the present embodiment will not be repeated. Further, the label-feature pair corresponding to the abnormal person is defined based on the blacklist, and the label-feature pair of the normal person is defined based on the whitelist. The label-feature pair corresponding to the third abnormal person and the label-feature pair corresponding to the normal person can also be generated based on the historical monitoring data and the monitoring video annotation result, and then the classification model is trained according to the label-feature pairs corresponding to the first abnormal person, the second abnormal person and the third abnormal person.

[0065] As another optional implementation, the abnormal personnel can also be identified when the personnel registers or scans the code to enter. The entrance of the cell can be provided with an automatic gate, and the non-cell residents can register as visitors by registration or calling the residents, etc. to obtain the opening permission of the automatic gate. Therefore, whether the personnel entering the cell is an abnormal personnel can be determined based on the obtaining manner of the opening permission of the automatic gate.

[0066] Exemplarily, if it is detected that the obtaining manner of the opening permission of the automatic gate is the resident card, and the user identification result is the resident, etc., it is determined that the personnel is normal. If it is detected that the opening manner of the automatic gate is manually opened by the security personnel, or remotely opened by the resident, it is identified that the personnel is abnormal. Because in the actual application scenario, the manual opening by the security personnel can be the delivery personnel or the visitor after manual registration. Similarly, the remote opening by the resident can also be the visitor or the delivery personnel, and therefore it can be determined that the personnel entering at this time is abnormal.

[0067] When the abnormal personnel is detected, the first moving track of the abnormal personnel can be further determined based on the first monitoring video corresponding to the entrance area. In an optional implementation, after the first monitoring video is obtained, the analysis model can intercept the picture frames in the first monitoring video based on the time sequence and the preset time interval, and then input the picture frames into the pre-trained feature extraction module to identify the preset markers in each picture frame and the interval relative position relationship between the abnormal personnel and the preset markers. Then, in the three-dimensional model of the cell, the current position of the abnormal personnel is marked based on the preset markers and the relative position relationship. Further, based on the time corresponding to each picture frame, the positions of the abnormal personnel in each picture frame are sequentially mapped to the three-dimensional model of the cell, and then the first moving track can be fitted according to the discrete positions and the position corresponding time sequence.

[0068] It should be noted that in the technical scheme provided in the embodiment, the angle at which each camera device collects the monitoring video is fixed and known. The recognition model can be trained in advance based on the screen of the monitoring video, so that the recognition model can recognize the preset marker. The preset marker can be a fixed building in the cell or other fixed objects. In this way, during the processing of the screen frames of the monitoring video, the region of interest of each screen frame can be determined in advance based on the shooting angle, and then during the marker recognition process, the region of interest of the marker can be directly delimited. In the recognition process, only the position of the abnormal person needs to be dynamically determined, and the abnormal person is a moving human body, which has a low recognition difficulty. Moreover, after the human body and the preset marker are recognized, since the shooting angle is fixed, the difficulty of calculating the relative position relationship between the two is also low. Based on the above content, those skilled in the art can certainly determine that, compared with the scheme of modeling the environment through a video and then fitting a trajectory according to the video data, the technical scheme provided in the present application can greatly reduce the computational overhead and the model training difficulty. Reducing the processing overhead is the core content of ensuring timely output of the early warning and reducing the early warning lag.

[0069] Step S20: determining a target camera device according to the first moving trajectory;

[0070] After the first moving trajectory is determined, the moving direction of the abnormal person after entering the cell can be predicted based on the moving trajectory. Then, according to the moving direction, it can be determined which target camera device the abnormal person will enter after leaving the monitoring range of the camera device corresponding to the first monitoring video. Alternatively, after the first moving trajectory is determined, a monitoring coverage area plan is obtained, and then a sub-area covered by the end of the first moving trajectory in the monitoring coverage area plan is taken as a target sub-area, and the monitoring device corresponding to the target sub-area is taken as the target camera device.

[0071] Exemplarily, please refer to Figure 2 After it is determined that the moving trajectory of the abnormal person in the entrance area is from point A to point B, it can be determined that the sub-area covered by the end of the first moving trajectory is area 1, and thus the camera device corresponding to area 1 can be taken as the target camera device. The monitoring range of the target camera device is area 1. The monitoring videos corresponding to areas 2 and 3 can not be analyzed. Compared with the scheme of recognizing based on the video content and then determining that the abnormal person is not in the content video and ignoring the video, the scheme provided in the embodiment can directly screen the video stream according to the trajectory, which can further reduce the computational overhead of the recognition system, thereby improving the response speed of the data processing system.

[0072] Step S30: acquiring a second monitoring video collected by the target camera, and determining a second moving track of the abnormal person according to the second monitoring video;

[0073] In this embodiment, when the second monitoring video is acquired, the second moving track of the abnormal person can be determined according to the second monitoring video. Based on the determination manner of the first moving track, when the second moving track is determined, a sub-identification model associated with the second monitoring video can be determined based on the unique identification mark of the target camera. In the technical solution provided in this embodiment, the identification model needs to identify the preset marker and the abnormal person in the video frame, and the identification of the abnormal person can be realized by using a general personnel identification scheme. For the identification of the preset marker, since the position of the monitoring device is fixed, a sub-identification model can be set based on each monitoring device (i.e., the target camera). In the training process, the sub-identification model only needs to identify the preset marker in the specified region of interest based on the image collected by the camera, and therefore, it can be realized based on a lightweight marker identification model.

[0074] For example, the present application provides a data processing architecture for generating a moving track, and the following description is for a track generation module. Please refer to Figure 3 The track generation module provided in this embodiment includes a lightweight marker identification model corresponding to each monitoring device (i.e., a camera. In the implementation process, the monitoring region or the monitoring device can be distinguished, and this embodiment is exemplified by distinguishing the monitoring device). The lightweight marker identification model is used to identify the preset marker in the input picture frame. The track generation module also includes an abnormal person identification model, which is used to identify the abnormal person in the picture frame. After identifying the preset marker and the abnormal person, the relative position can be calculated based on the identification result. Since the relative position relationship between the shooting angle and the preset marker and the camera is fixed and unchangeable, the relative position relationship between the abnormal person and the preset marker can be calculated based on this.

[0075] Further, after the relative position relationship between the preset marker and the abnormal person is calculated, the coordinates of the abnormal person can be mapped to the three-dimensional model of the cell based on the time sequence corresponding to the picture frame and the fixed coordinates of the preset marker in the three-dimensional model of the cell, and the second moving track can be fitted based on the mapped position of the abnormal person and the time sequence information of the corresponding picture frame.

[0076] In the technical solution provided in this embodiment, since the identification of the marker can be realized by a lightweight model, only the identification of the abnormal person needs to be determined in the process of fitting the track, and the position of the abnormal person can be anchored based on the preset marker after the abnormal person is identified. In this way, the track of the abnormal person in the monitoring region can be quickly determined.

[0077] Step S40: outputting risk warning information if the second moving track overlaps with the preset sensitive area

[0078] After determining the second moving track, it can be determined whether the second moving track overlaps with the preset sensitive area. If yes, the branch warning information is outputted. The setting of the sensitive area can be customized according to the actual application scenario. For example, the abnormal area can be set based on the identity of the abnormal personnel, or the sensitive area can be set according to the functional attributes of each area.

[0079] For example, in a scheme of setting a sensitive area according to the identity of an abnormal personnel, if the abnormal personnel is a delivery man, other areas outside the delivery destination can be set as sensitive areas. Please refer to Figure 2 When the delivery destination of the delivery man is 1 building, other areas outside 1 building can be set as sensitive areas. For example, when the second moving track overlaps with the area corresponding to 2 building, it is determined that the second moving track overlaps with the preset sensitive area. At this time, the risk warning information can be outputted to prompt the monitoring and management personnel to pay attention to the real-time monitoring of the area to discover the risk in advance.

[0080] In a scheme of setting a sensitive area according to the functional attributes of an area, flammable and explosive material storage, elevator shaft, high-voltage electric well or other dangerous areas can be set as sensitive areas. When the abnormal personnel enters the sensitive area, the risk warning information is issued to remind the security personnel that a dangerous event may occur in the area to achieve risk warning.

[0081] Alternatively, when the second moving track does not overlap with the preset sensitive area, it can be further determined whether the second moving track enters the monitoring area overlap position. If yes, it means that the abnormal personnel may leave the monitoring area corresponding to the target camera and enter the monitoring area corresponding to another device. Therefore, the first moving track can be updated according to the second moving track, and the target camera is determined based on the second moving track according to the first moving track to determine the target camera corresponding to the next monitoring area entered by the abnormal personnel, and the subsequent step of determining whether it enters the sensitive area is repeated to complete the handover of the monitoring area. In the process of the next monitoring area, the principle of determining the above-mentioned second moving track is the same, so this embodiment will not be repeated here.

[0082] In the technical scheme provided in the embodiment, whether the abnormal personnel enters the sensitive area can be determined according to the track of the abnormal personnel, and a risk warning is issued when the abnormal personnel enters the sensitive area, so as to remind the security personnel to pay attention to the real-time monitoring screen of the corresponding area. In this way, the analysis efficiency of the monitoring video is effectively improved. The security personnel does not need to pay attention to all monitoring screens at all times. In addition, the mobile track determination scheme provided in the embodiment predicts the next shooting device, i.e., the target shooting device, that can shoot the abnormal personnel through the entry track, and in the process of generating the track, the preset marker is used to anchor the abnormal personnel, so that the calculation overhead in the track generation process is effectively simplified, i.e., the calculation overhead is saved from two dimensions of the number of data to be analyzed and the data processing process, so that the speed of track generation is greatly improved under the same hardware conditions.

[0083] Please refer to Figure 4 In another embodiment of the present application, based on the above embodiment, after the step S30, the method further comprises:

[0084] S50: If the second mobile track overlaps with the preset sensitive area, the type of the target sensitive area overlapping with the second mobile track is determined, and a target analysis model is determined based on the type;

[0085] S60: The third monitoring video corresponding to the target sensitive area is sent to the target analysis model, and an abnormal analysis feedback result of the target analysis model is obtained;

[0086] S70: The level and content of the risk warning information are determined according to the abnormal analysis feedback result.

[0087] In the embodiment, when the types of sensitive areas are different, the behavior categories that can occur are also different, so different abnormal behavior recognition models can be trained for different scenarios. For example, if the sensitive area is a building entrance or a community public area, the recognition content can be loitering, tailing, fighting, etc., that is, the corresponding monitoring model can be input into the corresponding artificial intelligence model to identify the abnormal behavior. If the sensitive area is a place for storing flammable and explosive goods, an elevator shaft, a high-voltage electric shaft, etc., whether there is a falling risk, a falling event, a fire, an electric shock risk, etc. is mainly identified. Different analysis models are used for identification of different abnormal behaviors

[0088] Therefore, if the second movement track overlaps with the preset sensitive area, the type of the target sensitive area overlapping with the second movement track can be determined, a target analysis model is determined based on the type, the third monitoring video corresponding to the target sensitive area is sent to the target analysis model, and an abnormal analysis feedback result of the target analysis model is obtained. Different analysis models are used for identification of different abnormal behaviors, so that the level and content of the risk warning information are determined according to the abnormal analysis feedback result, and more accurate risk warning information is output based on the level and content.

[0089] Optionally, in order to further reduce the identification overhead, the third monitoring video corresponding to the target sensitive area is sent to the target analysis model, and the identification feature of the abnormal personnel is obtained according to the first monitoring video, so that the monitoring video segment containing the abnormal personnel in the third monitoring video is identified based on the identification feature, and irrelevant segments not containing the abnormal personnel are eliminated.

[0090] Optionally, after the step of obtaining the second movement track of the abnormal personnel according to the second monitoring video collected by the target camera, the method further includes determining a target period according to the time point at which the abnormal personnel arrives at the entrance area, for example, 5 minutes before the time point as the target period, and then generating a third movement track of a normal personnel entering the target period. If the second movement track matches the third movement track, there may be an abnormal personnel tailing. Therefore, the pre-warning information can be sent based on the communication mode associated with the normal personnel.

[0091] The present application provides a monitoring video analysis device, which comprises at least one processor and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the monitoring video analysis method based on artificial intelligence in the above-mentioned embodiment one.

[0092] Reference will be made to the following description Figure 5 which shows a structural schematic diagram of a monitoring video analysis device suitable for being used to implement the embodiments of the present application. The monitoring video analysis device in the embodiments of the present application can include but is not limited to devices such as mobile phones, tablets, PCs, servers and smart wearable devices. Figure 5 The monitoring video analysis device shown is only an example and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0093] As Figure 5As shown, the monitoring video analysis device can include a processing apparatus 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1002 or loaded from a storage apparatus 1003 into a random access memory (RAM) 1004. Various programs and data required for operation of the monitoring video analysis device are also stored in the RAM 1004. The processing apparatus 1001, the ROM 1002, and the RAM 1004 are connected to each other by a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input apparatus 1007 including, for example, a touch screen, a touch pad, a keyboard, etc.; an output apparatus 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage apparatus 1003 including, for example, a hard disk, etc.; and a communication apparatus 1009. The communication apparatus 1009 can allow the monitoring video analysis device to communicate wirelessly or by wire with other devices to exchange data. Although the monitoring video analysis device having various systems is shown in the figure, it should be understood that all of the systems shown are not required to be implemented or possessed. More or fewer systems can be alternatively implemented or possessed.

[0094] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication apparatus, or installed from the storage apparatus 1003, or installed from the ROM 1002. When the computer program is executed by the processing apparatus 1001, the above-mentioned functions defined in the methods of embodiments of the present disclosure are performed.

[0095] The monitoring video analysis device provided by the present disclosure adopts the monitoring video analysis device method in the above embodiments, and solves the technical problem of low monitoring video analysis efficiency. Compared with the related art, the monitoring video analysis device provided by the present disclosure has the same beneficial effects as the monitoring video analysis method based on artificial intelligence provided by the above embodiments, and other technical features in the monitoring video analysis device are the same as the features disclosed in the above method embodiments, which will not be repeated here.

[0096] It should be understood that various parts of the present application can be realized in hardware, software, firmware, or a combination thereof. In the description of the above-described embodiments, specific features, structures, materials or characteristics can be combined in any appropriate manner in any one or more embodiments or examples.

[0097] The above description is merely that of a specific implementation of the present application, and the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, and all such changes or replacements should be encompassed within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0098] Please refer to Figure 6 The present application provides a monitoring video analysis system, the monitoring video analysis system 100 comprises:

[0099] The acquisition module 110 is configured to, when detecting an abnormal person in an entrance area, determine a first moving track of the abnormal person based on a first monitoring video corresponding to the entrance area and a pre-stored cell three-dimensional model.

[0100] The determination module 120 is configured to determine a target camera device according to the first moving track.

[0101] The generation module 130 is configured to acquire a second monitoring video collected by the target camera device, and determine a second moving track of the abnormal person according to the second monitoring video.

[0102] The output module 140 is configured to output a risk warning information if the second moving track overlaps with a preset sensitive area.

[0103] The present application provides a computer readable storage medium having computer readable program instructions (i.e. computer programs) stored thereon, the computer readable program instructions being used to execute the above-mentioned artificial intelligence-based monitoring video analysis method.

[0104] The computer readable storage medium provided in the present application may, for example, be a U disk, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted in any suitable medium, including but not limited to electrical wires, optical cables, RF (Radio Frequency), and the like, or any suitable combination of the above.

[0105] The above computer readable storage medium can be contained in the monitoring video analysis device, or can exist separately without being assembled into the monitoring video analysis device.

[0106] The above computer readable storage medium carries one or more programs, which, when executed by the monitoring video analysis device, enable the monitoring video analysis device to improve the monitoring video analysis efficiency based on the above method.

[0107] Computer program code for carrying out operations of the present application can be written in one or more programming languages or combinations of languages including object oriented programming languages such as Java, Smalltalk, C++ or conventional procedural programming languages such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0108] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0109] The modules involved in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the name of the module does not constitute a limitation on the unit itself.

[0110] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e. computer program) for executing the above-mentioned artificial intelligence-based monitoring video analysis method, and can solve the technical problem of low monitoring video analysis efficiency. Compared with the related art, the computer readable storage medium provided by the present application has the same beneficial effects as the artificial intelligence-based monitoring video analysis method provided by the above-mentioned embodiments, which will not be repeated here.

[0111] The embodiments of the present application provide a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the above-mentioned artificial intelligence-based monitoring video analysis method are implemented.

[0112] The computer program product provided by the present application can solve the technical problem of how to improve the user experience. Compared with the related art, the computer program product provided by the embodiments of the present application has the same beneficial effects as the artificial intelligence-based monitoring video analysis method provided by the above-mentioned embodiments, which will not be repeated here.

[0113] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent flow transformation made by using the content of the specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent processing scope of the present application

[0114] It should be noted that, in the present document, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the element. In the absence of explicit limitations, the intervals given in this application include the boundary values.

[0115] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and necessary general hardware platforms, of course, they can also be realized by hardware, but in many cases the former is a better embodiment.

[0116] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the present application specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for monitoring video analysis based on artificial intelligence, characterized in that, The artificial intelligence-based monitoring video analysis method comprises: acquiring a video stream from a monitoring camera corresponding to an entrance area, then detecting and extracting facial features and / or body features of a human-shaped object from the monitoring video as personnel features using computer vision technology, and inputting the personnel features into a pre-trained classification model, the classification model receiving the extracted personnel features and determining whether the personnel is an abnormal personnel; when the abnormal personnel is detected at the entrance area, determining a first moving track of the abnormal personnel based on a first monitoring video corresponding to the entrance area and a pre-stored three-dimensional model of a community; determining a target camera device according to the first moving track, wherein a moving direction of the abnormal personnel after entering the community is predicted based on the first moving track, and the target camera device corresponding to a monitoring range entered by the abnormal personnel after leaving a monitoring range of a camera device corresponding to the first monitoring video is determined according to the moving direction; acquiring a second monitoring video collected by the target camera device, and determining a second moving track of the abnormal personnel according to the second monitoring video; if the second moving track overlaps with a preset sensitive area, outputting risk warning information; wherein the step of determining the first moving track of the abnormal personnel based on the first monitoring video corresponding to the entrance area and the pre-stored three-dimensional model of the community comprises: after the first monitoring video is acquired, picture frames in the first monitoring video are intercepted based on time sequence and a preset time interval, the picture frames are input into a pre-trained feature extraction module, a preset marker and a relative position relationship between the abnormal personnel and the preset marker in each picture frame are identified by the feature extraction module; in the three-dimensional model of the community, the current position of the abnormal personnel is marked based on the preset marker and the relative position relationship; after the current position of the abnormal personnel in each picture frame is mapped to the three-dimensional model of the community based on the time corresponding to each picture frame, the first moving track is fitted according to discrete positions and position corresponding time sequences. 2.The AI-based monitoring video analysis method of claim 1, wherein, after the step of determining the second moving track of the abnormal personnel according to the second monitoring video, the method further comprises: if the second moving track does not overlap with the preset sensitive area, the first moving track is updated according to the second moving track, and the step of determining the target camera device according to the first moving track is executed. 3.The AI-based monitoring video analysis method of claim 1, wherein, before the step of outputting the risk warning information, the method further comprises: if the second moving track overlaps with the preset sensitive area, the type of the target sensitive area overlapping with the second moving track is determined, and a target analysis model is determined based on the type; a third monitoring video corresponding to the target sensitive area is sent to the target analysis model to obtain an abnormal analysis feedback result of the target analysis model, wherein different analysis models are used for identification of different abnormal behaviors; the level and content of the risk warning information are determined according to the abnormal analysis feedback result. 4.The AI-based monitoring video analysis method of claim 3, wherein, Before the step of sending the monitoring video corresponding to the target sensitive area to the target analysis model to obtain an abnormal analysis feedback result of the target analysis model, the method further includes: obtaining an identification feature of the abnormal personnel based on the first monitoring video; and eliminating irrelevant data in the third monitoring video according to the identification feature. 5.The AI-based monitoring video analysis method of claim 1, wherein, The step of receiving the extracted personnel feature and determining whether the personnel is an abnormal personnel by the classification model includes: inputting the personnel feature into the pre-trained classification model to obtain a classification result; and determining that the abnormal personnel is detected at the entrance area if the classification result is an abnormal personnel. 6.The AI-based monitoring video analysis method of claim 5, wherein, The method further includes: generating a label-feature pair corresponding to a first abnormal personnel according to network public data; generating the label-feature pair corresponding to a second abnormal personnel according to blacklist data; generating the label-feature pair corresponding to a third abnormal personnel according to a monitoring video annotation result; and training the classification model according to the label-feature pairs corresponding to the first abnormal personnel, the second abnormal personnel and the third abnormal personnel. 7.The AI-based monitoring video analysis method of claim 1, wherein, After the step of obtaining a second monitoring video collected by the target camera device, and determining a second moving track of the abnormal personnel according to the second monitoring video, the method further includes: determining a target time period according to a time point at which the abnormal personnel arrives at the entrance area; obtaining a third moving track corresponding to a normal personnel in the target time period; and sending a warning information based on a communication mode associated with the normal personnel if the second moving track matches the third moving track. 8.The AI-based monitoring video analysis method of claim 1, wherein, Before the step of determining a first moving track of the abnormal personnel based on the first monitoring video corresponding to the entrance area and a pre-stored cell three-dimensional model, the method further includes: determining an opening mode when detecting that an automatic gate corresponding to the entrance area is opened; and determining that the abnormal personnel is detected at the entrance area if the opening mode is manual opening by a security personnel or remote opening by a resident.

9. A surveillance video analytics system characterized by, The monitoring video analysis system includes: an acquisition module: configured to acquire a video stream from a monitoring camera corresponding to an entrance area, and then detect and extract a face feature and / or a body feature of a human object from the monitoring video as a personnel feature by using a computer vision technology, and input the personnel feature into a pre-trained classification model, the classification model receives the extracted personnel feature and determines whether the personnel is an abnormal personnel; and when the abnormal personnel is detected at the entrance area, determine a first moving track of the abnormal personnel based on a first monitoring video corresponding to the entrance area and a pre-stored cell three-dimensional model; wherein the step of determining the first moving track of the abnormal personnel based on the first monitoring video corresponding to the entrance area and the pre-stored cell three-dimensional model includes: After the first monitoring video is acquired, picture frames in the first monitoring video are intercepted based on time sequence and a preset time interval, the picture frames are input to a pre-trained feature extraction module, a preset marker in each of the picture frames is identified by the feature extraction module, and a relative position relationship between the abnormal personnel and the preset marker is identified; In the three-dimensional model of the community, a current position of the abnormal personnel is marked based on the preset marker and the relative position relationship; After the current position of the abnormal personnel in each of the picture frames is mapped to the three-dimensional model of the community based on a time corresponding to each of the picture frames, the first moving track is fitted according to discrete positions and position corresponding time sequences; The determining module is configured to determine a target camera device according to the first moving track, wherein a moving direction of the abnormal personnel after entering the community is predicted based on the first moving track, and the target camera device corresponding to a monitoring range entered by the abnormal personnel after leaving a monitoring range of a camera device corresponding to the first monitoring video is determined according to the moving direction; The generating module is configured to acquire a second monitoring video collected by the target camera device, and determine a second moving track of the abnormal personnel according to the second monitoring video; The output module is configured to output risk warning information if the second moving track overlaps with a preset sensitive area.

10. A storage medium, characterized by The storage medium is a computer readable storage medium, and the storage medium stores a computer program. The computer program is executed by the processor to implement the steps of the monitoring video analysis method based on artificial intelligence according to any one of claims 1 to 8.

Images