Monitoring video analysis method and system based on artificial intelligence, and storage medium

Abstract

The invention discloses a monitoring video analysis method based on artificial intelligence, a monitoring video analysis system and a storage medium, and the method comprises the steps: determining a first movement track of an abnormal person based on a first monitoring video corresponding to an entrance region and a pre-stored community three-dimensional model when the abnormal person is detected in the entrance region, the method comprises the steps of obtaining a first movement track of an abnormal person, determining a target camera device according to the first movement track, obtaining a second monitoring video collected by the target camera device, determining a second movement track of the abnormal person according to the second monitoring video, and outputting risk early warning information if the second movement track is overlapped with a preset sensitive area. The risk early warning information can be automatically output, so that the monitoring video analysis efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of communications, and in particular to a surveillance video analysis method, a surveillance video analysis system and a storage medium based on artificial intelligence. Background Art

[0002] In urban residential areas, surveillance videos can achieve full coverage of key areas and strengthen monitoring of key areas to ensure comprehensive and targeted security precautions. This allows security personnel to quickly discover potential security risks based on surveillance videos without having to patrol on site. However, in related solutions, the observation and analysis of surveillance videos heavily relies on staff, resulting in a large amount of manpower consumption, and thus the defect of low efficiency in surveillance video analysis. Summary of the invention

[0003] The main purpose of this application is to provide a surveillance video analysis method, a surveillance video analysis system and a storage medium based on artificial intelligence, aiming to solve the technical problem of low efficiency of surveillance video analysis in related technologies.

[0004] To achieve the above objectives, the present application provides a surveillance video analysis method based on artificial intelligence, the method comprising:

[0005] When an abnormal person is detected in the entrance area, a first movement trajectory of the abnormal person is determined based on a first monitoring video corresponding to the entrance area and a pre-stored three-dimensional model of the community;

[0006] Determining a target camera device according to the first movement trajectory;

[0007] Acquire a second monitoring video captured by the target camera device, and determine a second movement trajectory of the abnormal person according to the second monitoring video;

[0008] If the second moving trajectory overlaps with a preset sensitive area, risk warning information is output.

[0009] In the embodiment of the present application, after the step of determining the second movement trajectory of the abnormal person according to the second monitoring video, the method further includes:

[0010] If the second movement trajectory does not overlap with the preset sensitive area, the first movement trajectory is updated according to the second movement trajectory, and the process jumps to the step of determining the target camera device according to the first movement trajectory.

[0011] In the embodiment of the present application, before the step of outputting risk warning information, the method further includes:

[0012] If the second movement trajectory overlaps with the preset sensitive area, determining the type of the target sensitive area overlapping with the second movement trajectory, and determining a target analysis model based on the type;

[0013] Sending the third monitoring video corresponding to the target sensitive area to the target analysis model to obtain an abnormal analysis feedback result of the target analysis model, wherein different analysis models are used for identifying different abnormal behaviors;

[0014] The level and content of the risk warning information are determined according to the abnormal analysis feedback results.

[0015] In the embodiment of the present application, before the step of sending the surveillance video corresponding to the target sensitive area to the target analysis model to obtain the abnormal analysis feedback result of the target analysis model, the method further includes:

[0016] Acquire identification features of the abnormal person based on the first surveillance video;

[0017] Irrelevant data in the third surveillance video is eliminated according to the identification feature.

[0018] In the embodiment of the present application, when an abnormal person is detected in the entrance area, before the step of determining the first movement trajectory of the abnormal person based on the first monitoring video corresponding to the entrance area and the pre-stored three-dimensional model of the community, the method further includes:

[0019] Acquire the first surveillance video, and extract features of people in the surveillance video;

[0020] Inputting the personnel characteristics into a pre-trained classification model to obtain a classification result;

[0021] If the classification result is an abnormal person, it is determined that the abnormal person is detected in the entrance area.

[0022] In the embodiment of the present application, the method further includes:

[0023] Generate a label-feature pair corresponding to the first abnormal person based on the public data on the Internet;

[0024] Generating the label-feature pair corresponding to the second abnormal person according to the blacklist data;

[0025] Generate the label-feature pair corresponding to the third abnormal person according to the monitoring video annotation result;

[0026] The classification model is trained according to the label-feature pairs corresponding to the first abnormal person, the second abnormal person and the third abnormal person.

[0027] In the embodiment of the present application, after the step of obtaining the second monitoring video captured by the target camera device and determining the second movement trajectory of the abnormal person according to the second monitoring video, the method further includes:

[0028] Determine a target time period according to the time point when the abnormal person arrives at the entrance area;

[0029] Obtaining a third movement trajectory corresponding to a normal person within the target time period;

[0030] If the second movement trajectory matches the third movement trajectory, a warning message is sent based on the communication method associated with the normal person.

[0031] In the embodiment of the present application, before the step of determining the first movement trajectory of the abnormal person based on the first surveillance video corresponding to the entrance area and the pre-stored three-dimensional model of the community, the method further includes:

[0032] When it is detected that the automatic gate corresponding to the entrance area is opened, determining the opening method;

[0033] If the opening method is manual opening by a security personnel, or remote opening by a resident, it is determined that the abnormal person is detected in the entrance area.

[0034] The present application also provides a surveillance video analysis system, the surveillance video analysis system comprising:

[0035] Acquisition module: when an abnormal person is detected in the entrance area, based on the first monitoring video corresponding to the entrance area and the pre-stored three-dimensional model of the community, determine the first movement trajectory of the abnormal person;

[0036] A determination module: used for determining a target camera device according to the first moving trajectory;

[0037] A generating module: used for acquiring a second monitoring video captured by the target camera device, and determining a second movement trajectory of the abnormal person according to the second monitoring video;

[0038] Output module: used to output risk warning information if the second movement trajectory overlaps with a preset sensitive area.

[0039] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the surveillance video analysis method based on artificial intelligence as described above are implemented.

[0040] The embodiment of the present application discloses a surveillance video analysis method based on artificial intelligence and a surveillance video analysis system. After the method detects an abnormal person in the entrance area of ​​a community, the method can determine the first movement trajectory of the abnormal person based on the first surveillance video of the entrance area, and then predict the subsequent movement direction of the abnormal person based on the first movement trajectory and the three-dimensional model of the community, and determine the target camera device based on this, and then determine the second movement trajectory of the abnormal person based on the second surveillance video collected by the target camera device. Finally, when the second movement trajectory overlaps with the preset sensitive area, the risk warning information is output to remind relevant personnel to pay more attention to the relevant surveillance video.

[0041] In the technical solution provided in this embodiment, on the one hand, based on trajectory analysis, it can be identified whether abnormal persons entering the community need special attention. If so, risk warnings can be used to prompt relevant personnel to increase their attention to the person, without the need for security personnel to actively pay attention to surveillance videos at all times to discover risks. This effectively improves the efficiency of surveillance video analysis. On the other hand, in the technical solution provided by this application, trajectory analysis is only performed when abnormal persons are detected, and in the process of trajectory determination, the relevant target camera device is predicted based on the first mobile trajectory, so that when determining the second mobile trajectory, only the surveillance video captured by the target camera device needs to be processed. In this way, the data content of the intelligent analysis is reduced from two dimensions, namely, the starting conditions and the analysis process, and the computing overhead can be effectively reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a flowchart of an embodiment of a surveillance video analysis method based on artificial intelligence involved in the embodiment scheme of the present application;

[0043] Figure 2 It is a schematic diagram of the position relationship of the monitoring areas involved in the embodiment of the present application;

[0044] Figure 3 A data processing architecture diagram for generating a mobile trajectory involved in an embodiment of the present application;

[0045] Figure 4 This is a flow chart of another embodiment of the artificial intelligence-based surveillance video analysis method involved in the embodiment of the present application;

[0046] Figure 5 This is a schematic diagram of the structure of the monitoring video analysis equipment for this application;

[0047] Figure 6 This is a schematic diagram of the modular structure of the surveillance video analysis system of this application.

[0048] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] In community security scenarios, surveillance videos can achieve full coverage of key areas and strengthen monitoring of key areas to ensure the comprehensiveness and pertinence of security precautions. This can significantly enhance the community's security capabilities and provide residents with a safer living environment.

[0051] In the application scenario of community security based on video surveillance, the monitoring system plays a vital role in pre-warning and post-event backtracking. For example, security personnel in the monitoring room can determine the scene based on the real-time monitoring screen without on-site patrol, so as to make pre-warning based on the scene screen. Or, in the event of theft, pet loss and other incidents, it can also be based on the stored monitoring video to do post-event backtracking.

[0052] However, the current monitoring system, whether it is pre-warning or post-event backtracking, relies on monitoring managers to manually understand and analyze monitoring videos. This approach not only leads to a large demand for human resources, but also is limited by the one-sidedness and subjectivity of manual analysis and the uncertainty caused by factors such as the length of time people can concentrate, resulting in a relatively low efficiency in monitoring video analysis.

[0053] In order to solve the above-mentioned defects in the related technology, this application proposes a surveillance video analysis method based on artificial intelligence. After detecting an abnormal person in the entrance area of ​​a community, the method can determine the first movement trajectory of the abnormal person based on the first surveillance video of the entrance area, and then predict the subsequent movement direction of the abnormal person based on the first movement trajectory and the three-dimensional model of the community, and determine the target camera device based on this, and then determine the second movement trajectory of the abnormal person based on the second surveillance video collected by the target camera device. Finally, when the second movement trajectory overlaps with the preset sensitive area, the risk warning information is output to remind relevant personnel to pay more attention to the relevant surveillance video.

[0054] In the technical solution provided in this embodiment, on the one hand, based on trajectory analysis, it can be identified whether abnormal persons entering the community need special attention. If so, risk warnings can be used to prompt relevant personnel to increase their attention to the person, without the need for security personnel to actively pay attention to surveillance videos at all times to discover risks. This effectively improves the efficiency of surveillance video analysis. On the other hand, in the technical solution provided by this application, trajectory analysis is only performed when abnormal persons are detected, and in the process of trajectory determination, the relevant target camera device is predicted based on the first mobile trajectory, so that when determining the second mobile trajectory, only the surveillance video captured by the target camera device needs to be processed. In this way, the data content of the intelligent analysis is reduced from two dimensions, namely, the starting conditions and the analysis process, and the computing overhead can be effectively reduced.

[0055] For ease of understanding, the technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0056] Please refer to Figure 1 In an optional implementation scheme, the surveillance video analysis method based on artificial intelligence includes the following steps S10 to S30:

[0057] S10: When an abnormal person is detected in the entrance area, a first movement trajectory of the abnormal person is determined based on a first monitoring video corresponding to the entrance area and a pre-stored three-dimensional model of the community;

[0058] In this embodiment, the data processing unit of the system for executing the artificial intelligence-based surveillance video analysis method can be set locally or in the cloud, and this embodiment does not limit this.

[0059] The above-mentioned entrance area may refer to the entrance area of ​​a residential community, or it may be a disclosure area set up in a monitoring environment. For example, when the solution is used in a road monitoring system, an intersection or the starting area of ​​a certain road section can be customized as the above-mentioned entrance area according to needs. This embodiment is not specifically limited. In order for those skilled in the art to more accurately understand the spirit of the solution, the technical solution provided in this embodiment is further explained below in conjunction with the community security scenario. However, the solution provided in this embodiment is not limited to use in community scenarios, and can also be applied to abnormal behavior analysis and detection corresponding to any monitoring coverage area.

[0060] In an optional implementation, this embodiment can detect whether an abnormal person appears in the entrance area based on the first monitoring video corresponding to the entrance area. If yes, the first movement trajectory of the abnormal person in the entrance area is further determined based on the first monitoring video.

[0061] In this embodiment, the first surveillance video is first acquired, and then the personnel features of the person in the surveillance video are extracted, and the personnel features are input into the classification model. If the person is an abnormal person, the classification result output by the classification model is an abnormal person. Among them, the system is provided with a video acquisition module, which can acquire a video stream from the surveillance camera corresponding to the entrance area. Then, using computer vision technology, personnel features such as facial features and / or body features of human-shaped objects are detected and extracted from the surveillance video, and are input into the pre-trained classification model. The classification model receives the extracted personnel features and preliminarily determines whether the person is an abnormal person.

[0062] Exemplarily, the surveillance camera supports the RTSP protocol, so the video acquisition module uses OpenCV and other libraries to acquire video streams in real time. Then use deep learning models (such as YOLO, SSD, etc.) to detect people in the video. For the detected faces, use face detection algorithms (such as MTCNN, FaceBoxes, etc.) to locate the face area, and then use face recognition algorithms (such as DeepFace, FaceNet, etc.) to extract face feature vectors. For the detected human body, a posture estimation algorithm (such as OpenPose, DeepLabCut, etc.) can be used to extract human key points, or a convolutional neural network can be used to extract human feature vectors. And the extracted human feature vector or face feature vector is input into the classification model to determine whether the person is a resident of the community or an abnormal person.

[0063] It should be noted that abnormal persons can be positioned as other persons outside the residents, or different groups can be defined as abnormal persons according to needs, such as express delivery personnel, visitors, etc. This embodiment does not limit this. The abnormal persons mentioned here are the persons corresponding to the abnormal samples during the model training process, which is only a training label.

[0064] Optionally, as a training method for a classification model, a label-feature pair corresponding to the first abnormal person can be generated based on public network data. For example, a sample label of an abnormal person corresponding to the first abnormal person can be constructed based on information disclosed by relevant departments. For example, a wanted person disclosed on the network can be obtained as the first abnormal person. In addition, a blacklist and a whitelist can also be provided, wherein the design of adding, deleting and modifying personnel in the blacklist and the whitelist can be customized according to needs, and this embodiment will not be repeated. Then, the label-feature pair corresponding to the abnormal person is defined based on the blacklist, and the label-feature pair of the normal person is defined based on the whitelist. It is also possible to perform annotation based on historical monitoring data, and generate the label-feature pair corresponding to the third abnormal person and the label-feature pair corresponding to the normal person according to the monitoring video annotation results, and then train the classification model according to the label-feature pair corresponding to the first abnormal person, the second abnormal person and the third abnormal person.

[0065] As another optional implementation scheme, abnormal persons can also be identified when people register or scan the code to enter. An automatic gate can be set at the entrance of the community, and non-residents of the community can register as visitors by registering or calling residents to obtain the opening authority of the automatic gate. Therefore, it can be determined whether the person entering the community is an abnormal person based on the way the opening authority of the automatic gate is obtained.

[0066] For example, if it is detected that the method of obtaining the automatic gate opening authority is a resident card, and the user identification result is a resident, etc., then it is determined to be a normal person. If it is detected that the automatic gate is opened manually by a security personnel, or remotely by a resident, it is identified as an abnormal person. Because in actual application scenarios, the security personnel manually open it, which may be a delivery person who has been manually registered, or a visitor. Similarly, the remote opening by a resident may also be a visitor or a delivery person, so it can be determined that the person entering at this time is an abnormal person.

[0067] When an abnormal person is detected, the first movement trajectory of the abnormal person can be further determined based on the first surveillance video corresponding to the entrance area. In an optional implementation scheme, after acquiring the first surveillance video, the analysis model can intercept the picture frames in the first surveillance video based on the time sequence and the preset time interval, and then input the picture frames into the pre-trained feature extraction module, and identify the preset markers in each picture frame through the feature extraction module, as well as the relative position relationship between the abnormal person and the preset marker. Then, in the three-dimensional model of the community, the current position of the abnormal person is marked based on the preset marker and the relative position relationship. Furthermore, based on the corresponding time of each picture frame, the corresponding position of the abnormal person in each picture frame is mapped to the three-dimensional model of the community in turn, and then the above-mentioned first movement trajectory can be fitted according to the discrete position and the corresponding time sequence of the position.

[0068] It should be noted that in the technical solution provided in this embodiment, since the angles at which each camera device collects surveillance video are fixed and known. The recognition model can be trained in advance based on the screen of the surveillance video so that the recognition model can recognize the preset marker. Among them, the preset marker can be a fixed building in the community, or other objects with fixed positions. In this way, for the picture frame processing of the surveillance video, based on its shooting angle, the region of interest of each picture frame can be predetermined, and then in the marker recognition process, the region of interest of the marker can be directly delineated. In this recognition process, only the position of the abnormal person needs to be dynamically determined, and the abnormal person is a moving human body, and the recognition difficulty is relatively low. In addition, after the human body and the preset marker are identified, since the shooting angle is fixed, the difficulty of solving the relative position relationship between the two is also relatively low. Based on the above content, it can be determined without doubt by those skilled in the art that the technical solution provided by this application can greatly reduce the computational overhead and the difficulty of model training compared to the solution of modeling the environment through video and then fitting the trajectory according to the video data. And reducing the processing overhead is the core content of ensuring that the warning can be output in time and reducing the hysteresis of the warning.

[0069] Step S20: determining a target camera device according to the first moving trajectory;

[0070] After determining the first moving trajectory, the moving direction of the abnormal person after entering the community can be predicted based on the moving trajectory. Then, based on the moving direction, it can be determined which target camera device's corresponding monitoring range the abnormal person will enter after leaving the monitoring range of the camera device corresponding to the first monitoring video. Optionally, after determining the first moving trajectory, a monitoring coverage area plan is obtained, and then in the monitoring coverage area plan, the sub-area covered by the end of the first moving trajectory is used as the target area, and the monitoring device corresponding to the target sub-area is used as the target camera device.

[0071] For example, please refer to Figure 2 , after determining that the movement trajectory of the abnormal person in the entrance area is from point A to point B, it can be determined that the sub-area covered by the end of the first movement trajectory is area 1, so the camera device corresponding to area 1 can be used as the target camera device. The monitoring range of the target camera device is area 1. The surveillance videos corresponding to areas 2 and 3 do not need to be analyzed. Compared with the solution based on video content recognition, and then determining that there is no abnormal person in the content video and then ignoring the video, the solution provided in this embodiment can directly screen the video stream according to the trajectory, which can further reduce the computational overhead of the recognition system, thereby improving the response speed of the data processing system.

[0072] Step S30: acquiring a second monitoring video captured by the target camera device, and determining a second movement trajectory of the abnormal person according to the second monitoring video;

[0073] In this embodiment, after the second surveillance video is acquired, the second movement trajectory of the abnormal person can be determined according to the second surveillance video. Based on the determination method of the first movement trajectory, when determining the second movement trajectory, the sub-recognition model associated with the second surveillance video can be determined based on the unique identification identifier of the target camera device. In the technical solution provided in this embodiment, the recognition model needs to identify the preset markers and abnormal persons in the video frame, and the identification of abnormal persons can be implemented by a general personnel identification scheme. For the recognition of preset markers, since the position of the monitoring device is fixed, a sub-recognition model can be set based on each monitoring device (i.e., the target shooting device). During the training process, the sub-recognition model only needs to identify the preset markers of the specified area of ​​interest based on the image captured by the camera device. Therefore, it can be implemented based on a lightweight marker recognition model.

[0074] Exemplarily, the present application provides a data processing architecture for generating a mobile trajectory, which is described below as a trajectory generation module. Figure 3 , this embodiment provides a trajectory generation module including a plurality of monitoring devices (i.e., shooting devices, which can be distinguished based on the monitoring area or the monitoring device during the implementation process, and this embodiment uses the monitoring device distinction as an example) corresponding to a lightweight marker recognition model, and the lightweight marker recognition model is used to identify the preset markers in the input picture frame. The trajectory generation module also includes an abnormal person recognition model, which is used to identify abnormal persons in the picture frame. After the preset markers and abnormal persons are identified, the relative position can be solved based on the recognition results. Among them, since the shooting angle and the relative position relationship between the preset marker and the camera device are fixed, the relative position relationship between the abnormal person and the preset marker can be calculated based on this.

[0075] Furthermore, after calculating the relative position relationship between the preset marker and the abnormal person, the coordinates of the abnormal person can be mapped to the three-dimensional model of the community based on the timing corresponding to the picture frame and the fixed coordinates of the preset marker in the three-dimensional model of the community, and the second movement trajectory can be fitted based on the mapped position of the abnormal person and the timing information of the corresponding picture frame.

[0076] In the technical solution provided by this embodiment, since the identification of the markers can be realized through the lightweight model, in the process of fitting the trajectory, it is only necessary to determine the identification of the abnormal person. After the abnormal person is identified, the position of the abnormal person can be anchored based on the preset markers. In this way, the trajectory of the abnormal person in the monitoring area can be quickly determined.

[0077] Step S40: If the second moving trajectory overlaps with the preset sensitive area, output risk warning information

[0078] After determining the second movement trajectory, it can be determined whether the second movement trajectory coincides with the preset sensitive area. If so, the line division warning information is output. Among them, the setting of the sensitive area can be customized according to the actual application scenario. For example, the abnormal area can be set based on the identity of the abnormal person, or the sensitive area can be set according to the functional attributes of each area.

[0079] For example, in a scheme of setting sensitive areas according to the identity of an abnormal person, if the abnormal person is a delivery person, other areas other than the delivery destination can be set as sensitive areas. Figure 2 , when the deliveryman's delivery destination is Building 1, other areas outside Building 1 can be set as sensitive areas. For example, when the second moving trajectory overlaps with the area corresponding to Building 2, it is determined that the second moving trajectory overlaps with the preset sensitive area. At this time, risk warning information can be output to prompt monitoring managers to pay attention to real-time monitoring of the area to detect risks in advance.

[0080] In a scheme for setting up sensitive areas according to the functional attributes of the area, places where flammable and explosive items are stored, elevator shafts, high-voltage power shafts or other dangerous areas can be set as sensitive areas. When abnormal personnel enter the sensitive areas, risk warning information is issued to remind security personnel that dangerous incidents may occur in the area, so as to achieve risk warning.

[0081] Optionally, when the second movement trajectory does not overlap with the preset sensitive area, it can be further determined whether the second movement trajectory has entered the overlapping position of the monitoring area. If so, it means that the abnormal person may leave the monitoring area corresponding to the target camera device and enter the monitoring area corresponding to another device. Therefore, the first movement trajectory can be updated according to the second movement trajectory, and the target camera device can be determined based on the first movement trajectory to determine the target camera device corresponding to the next monitoring area entered by the abnormal person, and the subsequent steps of determining whether it has entered the sensitive area are repeated to complete the handover of the monitoring area. In the process of the next monitoring area, the principle of the above-mentioned second movement trajectory determination method is the same, so this embodiment will not be repeated.

[0082] In the technical solution provided in this embodiment, it is possible to determine whether an abnormal person has entered a sensitive area based on the trajectory of the abnormal person, and when the abnormal person enters a sensitive area, a risk warning is issued to remind security personnel to pay attention to the real-time monitoring screen of the corresponding area. This effectively improves the analysis efficiency of the surveillance video. There is no need for security personnel to pay attention to all monitoring screens at all times. In addition, the mobile trajectory determination solution provided in this embodiment predicts the next shooting device that can capture the abnormal person, that is, the target shooting device, through the entrance trajectory, and in the process of generating the trajectory, the abnormal person is anchored by a preset marker, which effectively simplifies the computational overhead in the trajectory generation process, that is, from the two dimensions of the number of data to be analyzed and the data processing process, the computational overhead is saved, so that under the same hardware conditions, the speed of trajectory generation is greatly improved.

[0083] Please refer to Figure 4 In another embodiment of the present application, based on the above embodiment, after step S30, it further includes:

[0084] S50: If the second movement trajectory overlaps with the preset sensitive area, determine the type of the target sensitive area overlapping with the second movement trajectory, and determine a target analysis model based on the type;

[0085] S60: sending the third monitoring video corresponding to the target sensitive area to the target analysis model to obtain an abnormal analysis feedback result of the target analysis model;

[0086] S70: Determine the level and content of the risk warning information according to the abnormal analysis feedback result.

[0087] In this embodiment, when the sensitive area types are different, the types of behaviors that may occur are also different, so different abnormal behavior recognition models can be trained for different scenarios. For example, if the sensitive area is a building entrance, or a community public area, the recognition content can be wandering, stalking, fighting, etc., that is, the corresponding monitoring model can be input into the corresponding artificial intelligence model to identify such abnormal behaviors. If the sensitive area is a storage place for flammable and explosive items, an elevator shaft, a high-voltage power shaft, etc., it mainly identifies whether there is a risk of falling, a falling incident, a fire, an electric shock risk, etc. Among them, different analysis models are used to identify different abnormal behaviors.

[0088] Therefore, if the second moving trajectory overlaps with the preset sensitive area, the type of the target sensitive area overlapping with the second moving trajectory can be determined, and the target analysis model can be determined based on the type, and the third monitoring video corresponding to the target sensitive area is sent to the target analysis model to obtain the abnormal analysis feedback result of the target analysis model, wherein different analysis models are used to identify different abnormal behaviors, so as to determine the level and content of the risk warning information according to the abnormal analysis feedback result, and output more accurate risk warning information based on the level and content.

[0089] Optionally, in order to further reduce the recognition overhead, the third surveillance video corresponding to the target sensitive area can be sent to the target analysis model, and the identification features of the abnormal person can be obtained based on the first surveillance video, so as to identify the surveillance video segments containing the abnormal person in the third surveillance video based on the identification features, and eliminate irrelevant segments that do not contain the abnormal person.

[0090] Optionally, after obtaining the second monitoring video captured by the target camera device and determining the second movement trajectory of the abnormal person according to the second monitoring video, the method further includes determining a target time period according to the time point when the abnormal person arrives at the entrance area, for example, 5 minutes before the time point as the target time period, and then generating a third movement trajectory of normal people entering the target time period. If the second movement trajectory matches the third movement trajectory, there may be an abnormal person following. Therefore, an early warning message can be sent based on the communication method associated with the normal person.

[0091] The present application provides a surveillance video analysis device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the artificial intelligence-based surveillance video analysis method in the above-mentioned embodiment one.

[0092] Reference below Figure 5 , which shows a schematic diagram of the structure of a surveillance video analysis device suitable for implementing the embodiment of the present application. The surveillance video analysis device in the embodiment of the present application may include but is not limited to mobile phones, tablets, PCs, servers and smart wearable devices. Figure 5 The surveillance video analysis device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0093] like Figure 5As shown, the monitoring video analysis device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the monitoring video analysis device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the surveillance video analysis device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a surveillance video analysis device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0094] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0095] The monitoring video analysis device provided by the present application adopts the monitoring video analysis device method in the above embodiment to solve the technical problem of low efficiency of monitoring video analysis. Compared with the related art, the beneficial effects of the monitoring video analysis device provided by the present application are the same as the beneficial effects of the monitoring video analysis method based on artificial intelligence provided by the above embodiment, and the other technical features in the monitoring video analysis device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0096] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0097] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0098] Please refer to Figure 6 The present application provides a surveillance video analysis system, the surveillance video analysis system 100 comprising:

[0099] Acquisition module 110: when an abnormal person is detected in the entrance area, based on a first monitoring video corresponding to the entrance area and a pre-stored three-dimensional model of the community, determines a first movement trajectory of the abnormal person;

[0100] Determining module 120: used to determine a target camera device according to the first moving trajectory;

[0101] Generating module 130: used to obtain a second monitoring video captured by the target camera device, and determine a second movement trajectory of the abnormal person according to the second monitoring video;

[0102] Output module 140: used to output risk warning information if the second movement trajectory overlaps with a preset sensitive area.

[0103] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the artificial intelligence-based surveillance video analysis method in the above-mentioned embodiment.

[0104] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0105] The computer-readable storage medium may be included in the surveillance video analysis device; or may exist independently without being assembled into the surveillance video analysis device.

[0106] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the monitoring video analysis device, the monitoring video analysis device can improve the efficiency of monitoring video analysis based on the method.

[0107] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0108] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0109] The modules involved in the embodiments described in the present application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0110] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned surveillance video analysis method based on artificial intelligence, and can solve the technical problem of low efficiency of surveillance video analysis. Compared with the related art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the surveillance video analysis method based on artificial intelligence provided in the above-mentioned embodiment, and will not be repeated here.

[0111] An embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned artificial intelligence-based surveillance video analysis method.

[0112] The computer program product provided in this application can solve the technical problem of how to improve the user experience. Compared with the related art, the beneficial effects of the computer program product provided in the embodiment of this application are the same as the beneficial effects of the surveillance video analysis method based on artificial intelligence provided in the above embodiment, which will not be repeated here.

[0113] The above are only preferred embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent scope of the present application.

[0114] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element. In the intervals given in this application, the boundary values ​​are all included unless explicitly defined.

[0115] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0116] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A surveillance video analysis method based on artificial intelligence, characterized in that: The surveillance video analysis method based on artificial intelligence includes: When an abnormal person is detected in the entrance area, a first movement trajectory of the abnormal person is determined based on a first monitoring video corresponding to the entrance area and a pre-stored three-dimensional model of the community; Determining a target camera device according to the first movement trajectory; Acquire a second monitoring video captured by the target camera device, and determine a second movement trajectory of the abnormal person according to the second monitoring video; If the second moving trajectory overlaps with a preset sensitive area, risk warning information is output.

2. The surveillance video analysis method based on artificial intelligence according to claim 1, characterized in that: After the step of determining the second movement trajectory of the abnormal person according to the second monitoring video, the method further includes: If the second movement trajectory does not overlap with the preset sensitive area, the first movement trajectory is updated according to the second movement trajectory, and the process jumps to the step of determining the target camera device according to the first movement trajectory.

3. The surveillance video analysis method based on artificial intelligence according to claim 1, characterized in that: Before the step of outputting risk warning information, the method further includes: If the second movement trajectory overlaps with the preset sensitive area, determining the type of the target sensitive area overlapping with the second movement trajectory, and determining a target analysis model based on the type; Sending the third monitoring video corresponding to the target sensitive area to the target analysis model to obtain an abnormal analysis feedback result of the target analysis model, wherein different analysis models are used for identifying different abnormal behaviors; The level and content of the risk warning information are determined according to the abnormal analysis feedback results.

4. The surveillance video analysis method based on artificial intelligence as claimed in claim 3, characterized in that: Before the step of sending the surveillance video corresponding to the target sensitive area to the target analysis model to obtain the abnormal analysis feedback result of the target analysis model, the method further includes: Acquire identification features of the abnormal person based on the first surveillance video; Irrelevant data in the third surveillance video is eliminated according to the identification feature.

5. The surveillance video analysis method based on artificial intelligence according to claim 1, characterized in that: Before the step of determining a first movement trajectory of the abnormal person based on a first surveillance video corresponding to the entrance area and a pre-stored three-dimensional model of a community when an abnormal person is detected in the entrance area, the method further includes: Acquire the first surveillance video, and extract features of people in the surveillance video; Inputting the personnel characteristics into a pre-trained classification model to obtain a classification result; If the classification result is an abnormal person, it is determined that the abnormal person is detected in the entrance area.

6. The surveillance video analysis method based on artificial intelligence according to claim 5, characterized in that: The method further comprises: Generate a label-feature pair corresponding to the first abnormal person based on the public data on the Internet; Generating the label-feature pair corresponding to the second abnormal person according to the blacklist data; Generate the label-feature pair corresponding to the third abnormal person according to the monitoring video annotation result; The classification model is trained according to the label-feature pairs corresponding to the first abnormal person, the second abnormal person and the third abnormal person.

7. The surveillance video analysis method based on artificial intelligence according to claim 1, characterized in that: After the step of acquiring the second monitoring video captured by the target camera device and determining the second movement trajectory of the abnormal person according to the second monitoring video, the method further includes: Determine a target time period according to the time point when the abnormal person arrives at the entrance area; Obtaining a third movement trajectory corresponding to a normal person within the target time period; If the second movement trajectory matches the third movement trajectory, a warning message is sent based on the communication method associated with the normal person.

8. The surveillance video analysis method based on artificial intelligence according to claim 1, characterized in that: Before the step of determining the first movement trajectory of the abnormal person based on the first monitoring video corresponding to the entrance area and the pre-stored three-dimensional model of the community, the method further includes: When it is detected that the automatic gate corresponding to the entrance area is opened, determining the opening method; If the opening method is manual opening by a security personnel, or remote opening by a resident, it is determined that the abnormal person is detected in the entrance area.

9. A surveillance video analysis system, characterized in that: The monitoring video analysis system comprises: Acquisition module: when an abnormal person is detected in the entrance area, based on the first monitoring video corresponding to the entrance area and the pre-stored three-dimensional model of the community, determine the first movement trajectory of the abnormal person; A determination module: used for determining a target camera device according to the first moving trajectory; A generating module: used for acquiring a second monitoring video captured by the target camera device, and determining a second movement trajectory of the abnormal person according to the second monitoring video; Output module: used to output risk warning information if the second movement trajectory overlaps with a preset sensitive area.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the surveillance video analysis method based on artificial intelligence as described in any one of claims 1 to 8 are implemented.

Images