A video surveillance system based on artificial intelligence

By combining fixed-point and mobile monitoring modules in the artificial intelligence video surveillance system, using drones for close-range monitoring and automated charging, the problem of difficulty in identifying and positioning abnormal targets when they are too far away is solved, and the flexibility of the monitoring system is improved and autonomy enhances the ability to respond to emergencies.

CN118827934BActive Publication Date: 2024-11-22FUJIAN BATONG INTELLIGENT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411281073.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-13
Publication Date
2024-11-22
Estimated Expiration
2044-09-13

AI Technical Summary

Technical Problem

The existing artificial intelligence video surveillance system is difficult to obtain clear image data when the abnormal target distance is too far, resulting in a decrease in recognition probability and making accurate positioning and size prediction difficult.

Method used

The combination of fixed-point monitoring module and mobile monitoring module is adopted to move and approach abnormal targets through the drone module, and combine the abnormal identification module to achieve close monitoring and automated charging, improving the flexibility and continuous working ability of the system.

Benefits of technology

It greatly improves the flexibility and coverage of monitoring, achieves close-range high-definition monitoring of abnormal targets, enhances the system's autonomy and ability to deal with emergencies, and provides more accurate and comprehensive big data support.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118827934B_ABST
    Figure CN118827934B_ABST
Patent Text Reader

Abstract

The present application provides a video surveillance system based on artificial intelligence, which is mainly composed of three parts: a fixed-point surveillance module, a mobile surveillance module and an identification control module. The fixed-point surveillance module is installed at a fixed position and includes modules such as camera, direction adjustment, direction sensing, positioning and communication. The mobile surveillance module is based on a drone and is also equipped with camera, direction sensing, positioning and communication modules. The identification control module is responsible for analyzing the video images collected by the two surveillance modules and controlling them. The fixed-point module can adjust the camera direction and obtain video images of the surrounding environment. The mobile module can fly to a specific location for shooting according to instructions. The identification control module performs intelligent analysis on the collected video and controls the behavior of the two surveillance modules accordingly. This design combining fixed and mobile and multi-module collaboration greatly improves the flexibility and coverage of the surveillance system and can adapt to various complex surveillance scene requirements.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of video surveillance, and in particular to a video surveillance system based on artificial intelligence. Background Art

[0002] As an important technical means in the field of security, with the continuous advancement of science and technology, traditional video surveillance systems have gradually developed towards intelligent directions based on artificial intelligence.

[0003] Traditional video surveillance systems mainly rely on manual operation and viewing. Although this method can grasp the situation of the monitored area in real time, long-term manual viewing can easily cause fatigue and distraction, thereby reducing the effectiveness of monitoring. In addition, the massive amount of video data poses a huge challenge to post-playback and analysis, making it difficult to quickly locate and extract valuable information.

[0004] Faced with these challenges, video surveillance systems based on artificial intelligence have emerged. Through advanced technologies such as deep learning and computer vision, these systems can automatically analyze video content, identify target objects, detect abnormal behaviors, and conduct intelligent early warnings. The introduction of artificial intelligence technology has greatly improved the automation and intelligence level of the monitoring system, reduced the burden of manual viewing, and improved the accuracy of detecting abnormal events.

[0005] However, the current artificial intelligence video surveillance system still faces some technical difficulties. For example, when the abnormal target is too far away from the video surveillance system, it is impossible to obtain clear image data for analysis, identification and target tracking, which will cause the probability of abnormal identification to drop significantly. It is also difficult to locate and estimate the size of the abnormal situation through image data. Summary of the invention

[0006] The present application provides a video surveillance system based on artificial intelligence, comprising:

[0007] Fixed-point monitoring module, used to obtain surrounding video images at the installation location;

[0008] Mobile monitoring module, for moving to the target location to obtain the corresponding video image;

[0009] An identification control module for analyzing and identifying the video images acquired by the fixed-point monitoring module and the mobile monitoring module, and controlling the fixed-point monitoring module and the mobile monitoring module;

[0010] Wherein, the fixed-point monitoring module, the mobile monitoring module and the identification control module are communicatively connected;

[0011] The fixed-point monitoring module includes a first camera module, a direction adjustment module, a first direction sensing module, a first positioning module and a first communication module, wherein the first direction sensing module is arranged in the first camera module, the first camera module is arranged in the direction adjustment module, and the first camera module, the direction adjustment module, the first direction sensing module and the first positioning module are respectively data-connected to the first communication module;

[0012] Wherein, the mobile monitoring module includes a drone module, a second camera module, a second direction sensing module, a second positioning module and a second communication module, the second camera module, the second direction sensing module, the second positioning module and the second communication module are respectively arranged in the drone module, and the second camera module, the second direction sensing module and the second positioning module are respectively data-connected to the second communication module;

[0013] Wherein, the first communication module, the second communication module and the identification control module are data connected.

[0014] By adopting the above technical solution, the artificial intelligence-based video surveillance system can greatly improve the flexibility and coverage of monitoring by combining fixed-point monitoring modules and mobile monitoring modules, and combine with the abnormal recognition module to identify abnormal targets, thereby realizing close-range monitoring of abnormal targets and obtaining sufficient video image data.

[0015] Optionally, the fixed-point monitoring module further includes a landing platform and a wireless charging transmitting module, the wireless charging transmitting module is arranged on the landing platform, and the mobile monitoring module further includes a wireless charging receiving module, the wireless charging receiving module is arranged on the drone module;

[0016] Wherein, the drone module can selectively land on the landing platform, and the wireless charging transmitting module and the wireless charging receiving module can selectively form a power supply connection.

[0017] By adopting the above technical solution, the artificial intelligence-based video surveillance system can set a landing platform and a wireless charging function on the fixed-point monitoring module. The system realizes the automatic charging of the mobile monitoring module, which can greatly improve the continuous working ability and autonomy of the system. The drone can automatically return to the landing platform when charging is needed, and the charging process can be completed without human intervention, which greatly reduces the maintenance workload, ensures the continuous availability of the mobile monitoring module, and significantly improves the efficiency and reliability of the entire video surveillance system.

[0018] Optionally, the artificial intelligence-based video surveillance system further includes an abnormal monitoring strategy, and the abnormal monitoring strategy includes the following steps:

[0019] A1, acquiring corresponding fixed-point video data through the first camera module of the fixed-point monitoring module;

[0020] A2, acquiring corresponding fixed-point video direction data through the first direction sensing module of the fixed-point monitoring module;

[0021] A3, acquiring corresponding fixed-point video positioning data through the first positioning module of the fixed-point monitoring module;

[0022] A4, transmitting the fixed-point video data, the fixed-point video direction data and the fixed-point video positioning data to the identification control module through the first communication module of the fixed-point monitoring module;

[0023] A5, generating fixed-point abnormal target detection frame data through the recognition control module with a pre-trained abnormal recognition model according to the fixed-point video data;

[0024] A6, determining first abnormal target orientation data according to the fixed-point abnormal target detection frame data, the fixed-point video direction data and the fixed-point video positioning data using a preset target orientation estimation method;

[0025] A7, transmitting the first abnormal target position data to the mobile monitoring module;

[0026] A8, the drone module of the mobile monitoring module moves according to the first abnormal target position data and obtains corresponding mobile video data through the second camera module;

[0027] A9, acquiring corresponding mobile video direction data through the second direction sensing module of the mobile monitoring module;

[0028] A10, acquiring corresponding mobile video positioning data through the second positioning module of the mobile monitoring module;

[0029] A11, transmitting the mobile video data, the mobile video direction data and the mobile video positioning data to the identification control module through the second communication module of the mobile monitoring module.

[0030] By adopting the above technical solution, the artificial intelligence-based video surveillance system can achieve seamless collaboration of monitoring through fixed-point monitoring modules and mobile monitoring modules, and can improve the ability to obtain monitoring information. In particular, when an abnormal target is found, the mobile monitoring module can move closer to the abnormal target to obtain close-range high-definition monitoring video for analysis and viewing, which not only improves the accuracy and comprehensiveness of monitoring, but also enhances the system's ability to respond to emergencies, providing strong information support for security management.

[0031] Optionally, the target position estimation method comprises the following steps:

[0032] B1, determining the corresponding midpoint pixel position data of the first detection frame according to the fixed-point abnormal target detection frame data;

[0033] B2, determining a corresponding fixed-point abnormal target offset vector according to the preset fixed-point video midpoint pixel position data and the first detection frame midpoint pixel position data;

[0034] B3, adjusting the orientation of the first camera module through the direction adjustment module according to the fixed-point abnormal target offset vector until the modulus of the fixed-point abnormal target offset vector is less than a preset calibration pixel threshold;

[0035] B4, acquiring corresponding fixed-point video direction data through the first direction sensing module and defining it as first calibration direction data;

[0036] B5. Determine the corresponding first abnormal target orientation data according to the first calibration direction data and the fixed-point video positioning data.

[0037] By adopting the above-mentioned technical solution, the artificial intelligence-based video surveillance system can combine the video image obtained by the fixed-point monitoring module with the detection frame data of the abnormal target, so that the first camera module can be calibrated to the abnormal target, and then the corresponding direction data can be obtained through the first direction sensing module. Combined with the positioning data and direction data of the first camera module, the orientation of the abnormal target can be estimated to provide a basis for the moving direction of the mobile monitoring module.

[0038] Optionally, the artificial intelligence-based video surveillance system further includes an abnormal target positioning strategy:

[0039] C1, generating moving abnormal target detection frame data through the recognition control module using an abnormal recognition model according to the moving video data;

[0040] C2, determining the corresponding second detection frame midpoint pixel position data according to the moving abnormal target detection frame data;

[0041] C3, determining the corresponding abnormal moving target offset vector according to the preset moving video midpoint pixel position data and the second detection frame midpoint pixel position data;

[0042] C4, adjusting the orientation of the second camera module through the drone module according to the abnormal moving target offset vector until the modulus of the abnormal moving target offset vector is less than the calibration pixel threshold;

[0043] C5, acquiring corresponding mobile video direction data through the second direction sensing module and defining it as second calibration direction data, and acquiring corresponding mobile video positioning data through the second positioning module and defining it as second calibration positioning data;

[0044] C6, determining corresponding second abnormal target azimuth data according to the second calibration direction data and the second calibration positioning data;

[0045] C7, calculating corresponding intersection coordinate data according to the first abnormal target orientation data and the second abnormal target orientation data and defining them as abnormal target positioning data.

[0046] By adopting the above technical solution, the artificial intelligence-based video surveillance system can determine the corresponding second abnormal target position data through the mobile monitoring module, and combine the first abnormal target position data and the second abnormal target position data to realize the positioning of the abnormal target, thereby enhancing the system's tracking and positioning capabilities for abnormal targets, and providing more reliable and accurate positioning support for security monitoring and emergency response.

[0047] Optionally, the artificial intelligence-based video surveillance system further includes a target size estimation strategy:

[0048] D1, calculates the corresponding target distance based on the abnormal target positioning data and the fixed-point video positioning data;

[0049] D2, determine the corresponding horizontal pixel length and vertical pixel length of the detection frame according to the fixed-point abnormal target detection frame data;

[0050] D3, calculating a corresponding horizontal pixel size ratio and a vertical pixel size ratio according to a preset horizontal field of view angle, a preset vertical field of view angle, a preset video image pixel height, a preset video image pixel width and a target distance;

[0051] D4, calculate the corresponding estimated horizontal length of the target according to the horizontal pixel length of the detection frame and the horizontal pixel size ratio;

[0052] D5, calculates the corresponding estimated vertical length of the target based on the vertical pixel length of the detection box and the vertical pixel size ratio.

[0053] By adopting the above technical solution, the artificial intelligence-based video surveillance system can estimate the distance of the abnormal target through the abnormal target positioning data and the fixed-point video positioning data, and estimate the correspondence between the real size and the image pixels according to the distance of the abnormal target and the pixel length and pixel width of the abnormal target in the video image in combination with the optical parameters of the camera module and the image resolution, and then estimate the actual size of the abnormal target, which is conducive to evaluating potential threats, identifying suspicious objects or conducting on-site analysis, and provides more comprehensive and accurate data support for security monitoring and event assessment.

[0054] Optionally, the artificial intelligence-based video surveillance system further includes a surround camera strategy:

[0055] E1, taking the abnormal target positioning data as the center of the circle, circling the abnormal target through the mobile monitoring module according to the preset flight altitude and the preset circling distance and obtaining the corresponding abnormal target circling video data;

[0056] E2, determining the multi-angle abnormal target detection frame data corresponding to each angle through the abnormal recognition model according to the abnormal target surrounding video data;

[0057] E3, respectively determining the minimum horizontal pixel length and the maximum horizontal pixel length in all multi-angle abnormal target detection frame data;

[0058] E4, calculate the estimated area of ​​the corresponding abnormal target according to the product of the minimum horizontal pixel length and the maximum horizontal pixel length;

[0059] E5, calculates the average vertical pixel length based on all multi-angle abnormal target detection box data;

[0060] E6, calculate the estimated area occupied by the abnormal target by multiplying the estimated area occupied by the abnormal target and the average vertical pixel length.

[0061] By adopting the above technical solution, the artificial intelligence-based video surveillance system can fly around the abnormal target according to the abnormal target positioning data and obtain the corresponding surround video image. On the one hand, it can obtain comprehensive image data of the abnormal target, and on the other hand, it can estimate the volume size of the abnormal target based on the surround video image, overcoming the visual bias that may be caused by single-angle observation, and can also provide more comprehensive and three-dimensional target information, which is conducive to identifying objects of complex shapes, assessing the scale of potential threats, and conducting accurate situation assessments.

[0062] The present application also provides a video monitoring method based on artificial intelligence, comprising the following steps:

[0063] Obtain corresponding fixed-point video data through a preset fixed-point monitoring module;

[0064] Obtaining corresponding fixed-point video direction data through the fixed-point monitoring module;

[0065] Obtaining corresponding fixed-point video positioning data through the fixed-point monitoring module;

[0066] Generate fixed-point abnormal target detection frame data based on fixed-point video data using a pre-trained abnormal recognition model;

[0067] Determine the abnormal target orientation data according to the fixed-point abnormal target detection frame data, the fixed-point video direction data and the fixed-point video positioning data using a preset target orientation estimation method;

[0068] Move the preset mobile monitoring module according to the abnormal target position data and obtain the corresponding mobile video data;

[0069] Acquiring corresponding mobile video direction data through the mobile monitoring module;

[0070] Acquiring corresponding mobile video positioning data through the mobile monitoring module;

[0071] The mobile video data, the mobile video direction data and the mobile video positioning data are transmitted to a preset recognition control module.

[0072] By adopting the above technical solution, the artificial intelligence-based video surveillance method can achieve seamless collaboration of monitoring through fixed-point monitoring modules and mobile monitoring modules, and can improve the ability to obtain monitoring information. In particular, when an abnormal target is found, the mobile monitoring module can move closer to the abnormal target to obtain close-range high-definition monitoring video for analysis and viewing, which not only improves the accuracy and comprehensiveness of monitoring, but also enhances the system's ability to respond to emergencies, providing strong information support for security management.

[0073] In summary, the present application includes at least one of the following beneficial technical effects:

[0074] 1. By combining the fixed-point monitoring module and the mobile monitoring module, the flexibility and coverage of monitoring can be greatly improved. By combining the abnormal recognition module to identify abnormal targets, close-range monitoring of abnormal targets can be achieved to obtain sufficient video image data.

[0075] 2. By setting up a landing platform and wireless charging function on the fixed-point monitoring module, the system realizes the automatic charging of the mobile monitoring module, which can greatly improve the continuous working ability and autonomy of the system. The drone can automatically return to the landing platform when charging is needed, and the charging process can be completed without human intervention, which greatly reduces the maintenance workload, ensures the continuous availability of the mobile monitoring module, and significantly improves the efficiency and reliability of the entire video surveillance system.

[0076] 3. Seamless monitoring collaboration can be achieved through fixed-point monitoring modules and mobile monitoring modules, which can improve the ability to obtain monitoring information. Especially when abnormal targets are found, the mobile monitoring module can move closer to the abnormal targets to obtain close-range high-definition monitoring videos for analysis and viewing. This not only improves the accuracy and comprehensiveness of monitoring, but also enhances the system's ability to respond to emergencies, providing strong information support for security management. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 It is a schematic diagram of the principle of a video monitoring system based on artificial intelligence of the present invention.

[0078] Figure 2It is a process schematic diagram of a video monitoring method based on artificial intelligence of the present invention. DETAILED DESCRIPTION

[0079] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0080] The embodiments of the present application are further described in detail below in conjunction with the drawings in the specification.

[0081] refer to Figure 1 The present invention provides a video surveillance system based on artificial intelligence, which obtains corresponding video images, identifies and discovers abnormal targets with a pre-trained recognition model, and further monitors the abnormal targets to obtain sufficient information about the abnormal situation for background analysis or staff viewing.

[0082] The video surveillance system based on artificial intelligence includes:

[0083] A fixed-point monitoring module 10 is used to obtain surrounding video images at the installation location;

[0084] A mobile monitoring module 20 is used to move to a target location to obtain a corresponding video image;

[0085] an identification control module 30 for analyzing and identifying the video images acquired by the fixed-point monitoring module 10 and the mobile monitoring module 20, and controlling the fixed-point monitoring module 10 and the mobile monitoring module 20;

[0086] Wherein, the fixed-point monitoring module 10, the mobile monitoring module 20 and the identification control module 30 are communicatively connected;

[0087] The fixed-point monitoring module 10 includes a first camera module 11, a direction adjustment module 12, a first direction sensor module 13, a first positioning module 14 and a first communication module 15, wherein the first direction sensor module 13 is arranged in the first camera module 11, the first camera module 11 is arranged in the direction adjustment module 12, and the first camera module 11, the direction adjustment module 12, the first direction sensor module 13 and the first positioning module 14 are respectively data-connected to the first communication module 15;

[0088] The mobile monitoring module 20 includes a drone module 21, a second camera module 22, a second direction sensor module 23, a second positioning module 24 and a second communication module 25, wherein the second camera module 22, the second direction sensor module 23, the second positioning module 24 and the second communication module 25 are respectively arranged in the drone module 21, and the second camera module 22, the second direction sensor module 23 and the second positioning module 24 are respectively data-connected to the second communication module 25;

[0089] The first communication module 15 , the second communication module 25 and the identification control module 30 are data connected.

[0090] The fixed-point monitoring module 10 is mainly used to shoot the surrounding environment from a fixed position to obtain corresponding video image data, and can transmit the video image data to the recognition control module 30 for video analysis, and can adjust the shooting direction according to the control signal of the recognition control module 30.

[0091] The first camera module 11 is mainly used to obtain corresponding video image data for the fixed-point monitoring module 10;

[0092] The direction adjustment module 12 is mainly used to adjust the shooting direction of the first camera module 11 according to control;

[0093] The first direction sensing module 13 is mainly used to obtain data of the shooting direction of the first camera module 11;

[0094] The first positioning module 14 is mainly used to obtain the positioning data of the shooting position where the fixed-point monitoring module 10 is located;

[0095] The first communication module 15 is mainly used for data communication between the fixed-point monitoring module 10 , the mobile monitoring module 20 and the identification control module 30 .

[0096] The mobile monitoring module 20 is mainly used to move closer to the target to be photographed through the drone module 21 to obtain higher-definition video data, and transmit the video data back to the recognition control module 30 to supplement the video image data taken by the fixed-point monitoring module 10.

[0097] The drone module 21 is mainly used to move and carry the second camera module 22 to approach the location of an abnormal target or abnormal situation;

[0098] The second camera module 22 is mainly used to capture video image data of abnormal targets or abnormal situations at a close distance;

[0099] The second direction sensing module 23 is mainly used to obtain the shooting direction of the second camera module 22;

[0100] The second positioning module 24 is mainly used to obtain the positioning data of the drone module 21, that is, the video shooting position data of the second camera module 22;

[0101] The second communication module 25 is mainly used for data communication between the mobile monitoring module 20 and the fixed-point monitoring module and the identification control module 30 .

[0102] Furthermore, since the fixed-point monitoring module 10 is fixedly installed, the precise positioning data of the fixed-point monitoring module 10 can be accurately measured in advance, and combined with the real-time positioning data of the fixed-point monitoring module 10 and the mobile monitoring module 20, the RTK positioning method can be implemented, which can greatly improve the positioning accuracy of the mobile monitoring module 20, thereby enabling precise control thereof.

[0103] The recognition control module 30 is mainly used to analyze and identify the video images sent back by the fixed-point monitoring module 10 and the mobile monitoring module 20 to detect abnormal situations or abnormal targets. A corresponding pre-trained recognition model can be set in the recognition control module 30 to complete the recognition of abnormal situations.

[0104] After the fixed-point monitoring module 10 identifies and finds an abnormal target or abnormal situation, more detailed video data can be obtained by controlling the mobile monitoring module 20 to approach the location of the abnormal target or abnormal situation for close-up shooting.

[0105] Through the above technical scheme, the artificial intelligence-based video surveillance system can greatly improve the flexibility and coverage of monitoring by combining fixed-point monitoring modules and mobile monitoring modules, and combine with the abnormal recognition module to identify abnormal targets, so as to realize close-range monitoring of abnormal targets and obtain sufficient video image data.

[0106] Furthermore, the fixed-point monitoring module 10 further includes a landing platform 16 and a wireless charging transmitting module 17, wherein the wireless charging transmitting module 17 is disposed on the landing platform 16, and the mobile monitoring module 20 further includes a wireless charging receiving module 26, wherein the wireless charging receiving module 26 is disposed on the drone module 21;

[0107] The drone module 21 can selectively land on the landing platform 16 , and the wireless charging transmitting module 17 and the wireless charging receiving module 26 can selectively form a power supply connection.

[0108] The landing platform 16 is mainly used for the mobile monitoring module 20 to land and dock;

[0109] The wireless charging transmitter module 17 is mainly used to wirelessly transmit electrical energy to the outside;

[0110] The wireless charging receiving module 26 is mainly used to receive wirelessly transmitted electric energy;

[0111] When the mobile monitoring module 20 lands on the landing platform 16, the drone module 21 can be charged through the wireless charging transmitting module 17 and the wireless charging receiving module 26, so that the drone module 21 can have more power when it needs to take off and move.

[0112] Through the above technical solutions, the artificial intelligence-based video surveillance system can set a landing platform and wireless charging function on the fixed-point monitoring module. The system realizes the automatic charging of the mobile monitoring module, which can greatly improve the continuous working ability and autonomy of the system. The drone can automatically return to the landing platform when charging is needed, and the charging process can be completed without human intervention, which greatly reduces the maintenance workload, ensures the continuous availability of the mobile monitoring module, and significantly improves the efficiency and reliability of the entire video surveillance system.

[0113] Furthermore, the artificial intelligence-based video surveillance system also includes an abnormal monitoring strategy, and the abnormal monitoring strategy includes the following steps:

[0114] A1, acquiring corresponding fixed-point video data through the first camera module 11 of the fixed-point monitoring module 10;

[0115] The fixed-point video data is video image data captured by the first camera module 11 of the fixed-point monitoring module 10 .

[0116] A2, acquiring corresponding fixed-point video direction data through the first direction sensing module 13 of the fixed-point monitoring module 10;

[0117] The fixed-point video direction data is the direction data continuously acquired by the first direction sensor module 13 of the fixed-point monitoring module 10 during video shooting, and may include pitch angle data and horizontal angle data.

[0118] A3, acquiring corresponding fixed-point video positioning data through the first positioning module 14 of the fixed-point monitoring module 10;

[0119] The fixed-point video positioning data is the positioning data of the position of the fixed-point monitoring module 10, that is, the positioning data when the fixed-point video data is acquired. The fixed-point video positioning data can be obtained by real-time measurement by the first positioning module 14, or a pre-measured precise positioning value can be used. Since the point monitoring module 10 is fixedly installed, no displacement will occur.

[0120] A4, transmitting the fixed-point video data, the fixed-point video direction data and the fixed-point video positioning data to the identification control module 30 through the first communication module 15 of the fixed-point monitoring module 10;

[0121] The transmission fixed-point video data, the fixed-point video direction data and the fixed-point video positioning data acquired by the fixed-point monitoring module 10 are transmitted to the recognition control module 30 through the first communication module 15 for analysis and processing.

[0122] A5, generating fixed-point abnormal target detection frame data through the recognition control module 30 with a pre-trained abnormal recognition model according to the fixed-point video data;

[0123] The abnormality recognition model is an artificial intelligence model that is pre-trained and generated according to the monitoring usage scenario. It can select and identify specific targets in video images and determine whether the target is abnormal;

[0124] The fixed-point abnormal target detection frame data is the data of the target detection frame of the abnormal target in the fixed-point video data, which may include information such as pixel area, pixel length, pixel width, etc., and can be implemented through a variety of target detection algorithms. Usually, the target detection frame is a rectangle.

[0125] A6, determining first abnormal target orientation data according to the fixed-point abnormal target detection frame data, the fixed-point video direction data and the fixed-point video positioning data using a preset target orientation estimation method;

[0126] The target orientation estimation method is a pre-set algorithm that estimates the area where the abnormal target is located by combining the fixed-point abnormal target detection frame data, the fixed-point video direction data, and the fixed-point video positioning data;

[0127] The first abnormal target position data is the position data of the abnormal target estimated based on the video image data, direction data and positioning data collected by the fixed-point monitoring module 10 .

[0128] A7, transmitting the first abnormal target position data to the mobile monitoring module 20;

[0129] The first abnormal target position data is transmitted to the mobile monitoring module 20 so that the mobile monitoring module 20 can move closer to the abnormal target.

[0130] A8, the drone module 21 of the mobile monitoring module 20 moves according to the first abnormal target position data and obtains corresponding mobile video data through the second camera module 22;

[0131] The mobile video data is the video data captured by the second camera module 22 of the drone module 21;

[0132] When moving close to the abnormal target according to the first abnormal target position data, video images are continuously captured during the movement, so that the continuous dynamics of the abnormal target can be obtained for continuous monitoring and analysis.

[0133] A9, obtaining corresponding mobile video direction data through the second direction sensing module 23 of the mobile monitoring module 20;

[0134] The mobile video direction data is the direction data continuously acquired by the second direction sensor module 23 during the movement of the mobile monitoring module 20 , that is, the shooting direction data of the second camera module 22 .

[0135] A10, obtaining corresponding mobile video positioning data through the second positioning module 24 of the mobile monitoring module 20;

[0136] The mobile video positioning data is the positioning data continuously acquired by the second positioning module 24 during the movement of the mobile monitoring module 20, that is, the positioning data of the second camera module 22 when shooting the video.

[0137] A11, transmitting the mobile video data, the mobile video direction data and the mobile video positioning data to the recognition control module 30 through the second communication module 25 of the mobile monitoring module 20;

[0138] The mobile video data, mobile video direction data and mobile video positioning data acquired by the mobile monitoring module 20 are sent to the recognition control module 30 to provide a closer monitoring video for further recognition analysis or manual monitoring.

[0139] Through the above steps, the artificial intelligence-based video surveillance system can achieve seamless collaboration of monitoring through fixed-point monitoring modules and mobile monitoring modules, and can improve the ability to obtain monitoring information. In particular, when an abnormal target is found, the mobile monitoring module can move close to the abnormal target to obtain close-range high-definition monitoring video for analysis and viewing, which not only improves the accuracy and comprehensiveness of monitoring, but also enhances the system's ability to respond to emergencies, providing strong information support for security management.

[0140] Furthermore, the target position estimation method comprises the following steps:

[0141] B1, determining the corresponding midpoint pixel position data of the first detection frame according to the fixed-point abnormal target detection frame data;

[0142] The midpoint pixel position data of the first detection frame is the pixel coordinates of the center position of the detection frame corresponding to the fixed-point abnormal target detection frame data;

[0143] For a rectangular detection box, the intersection of two diagonals can be selected as the corresponding midpoint.

[0144] B2, determining a corresponding fixed-point abnormal target offset vector according to the preset fixed-point video midpoint pixel position data and the first detection frame midpoint pixel position data;

[0145] The fixed-point video midpoint pixel position data is the pre-set midpoint position of the fixed-point video data, which can be determined according to the resolution of the video image, that is, the intersection of the diagonal lines of the rectangle corresponding to the size of the video image can be used as;

[0146] The fixed-point abnormal target offset vector is a vector calculated and determined based on the midpoint pixel position data of the fixed-point video and the midpoint pixel position data of the first detection frame, and is used to reflect the orientation information and distance information between the midpoint pixel position data of the fixed-point video and the midpoint pixel position data of the first detection frame.

[0147] B3, adjusting the orientation of the first camera module 11 through the direction adjustment module 12 according to the fixed-point abnormal target offset vector, until the modulus of the fixed-point abnormal target offset vector is less than a preset calibration pixel threshold;

[0148] The calibration pixel threshold is a pre-set reference value used to determine the magnitude of the fixed-point abnormal target offset vector;

[0149] The fixed-point abnormal target offset vector can be used as a direction adjustment basis for the direction adjustment module 12, and can be adjusted so that the midpoint pixel position data of the fixed-point video and the midpoint pixel position data of the first detection frame are as close to coincide as possible, that is, the modulus of the fixed-point abnormal target offset vector is as small as possible;

[0150] On the one hand, the abnormal target can be always placed in the center of the video image to ensure the integrity of the video information; on the other hand, the orientation of the abnormal target can be determined by the direction of the first camera module 11 .

[0151] B4, acquiring corresponding fixed-point video direction data through the first direction sensing module 13 and defining it as first calibration direction data;

[0152] The first calibration direction data is the direction data corresponding to the first camera module 11 when the midpoint pixel position data of the fixed-point video and the midpoint pixel position data of the first detection frame are close to coincident.

[0153] B5. Determine the corresponding first abnormal target orientation data according to the first calibration direction data and the fixed-point video positioning data.

[0154] The first abnormal target position data is the position information determined based on the fixed-point video positioning data and the first calibration position data, that is, the position information of the abnormal target estimated by the fixed-point monitoring module 10, which can reflect the position information of the abnormal target based on the fixed-point video positioning data.

[0155] Through the above steps, the artificial intelligence-based video surveillance system can combine the video image obtained by the fixed-point monitoring module with the detection frame data of the abnormal target, so that the first camera module can be calibrated to the abnormal target, and then the corresponding direction data can be obtained through the first direction sensing module. Combined with the positioning data and direction data of the first camera module, the orientation of the abnormal target can be estimated to provide a basis for the moving direction of the mobile monitoring module.

[0156] Furthermore, the artificial intelligence-based video surveillance system also includes an abnormal target positioning strategy:

[0157] C1, generating moving abnormal target detection frame data by using the recognition control module 30 with an abnormal recognition model according to the moving video data;

[0158] The moving abnormal target detection frame data is the detection frame data of the abnormal target determined by the recognition control module 30 through recognition of the moving video data.

[0159] C2, determining the corresponding second detection frame midpoint pixel position data according to the moving abnormal target detection frame data;

[0160] The midpoint pixel position data of the second detection frame is the position data of the midpoint pixel of the detection frame corresponding to the moving abnormal target detection frame data.

[0161] C3, determining the corresponding abnormal moving target offset vector according to the preset moving video midpoint pixel position data and the second detection frame midpoint pixel position data;

[0162] The moving video midpoint pixel position data is the pre-set midpoint position of the moving video data;

[0163] The moving abnormal target offset vector is a vector calculated based on the midpoint pixel position data of the moving video and the midpoint pixel position data of the second detection frame, and is used to reflect the orientation information and distance information between the midpoint pixel position data of the moving video and the midpoint pixel position data of the second detection frame.

[0164] C4, adjusting the orientation of the second camera module 22 through the drone module 21 according to the abnormal moving target offset vector until the modulus of the abnormal moving target offset vector is less than the calibration pixel threshold;

[0165] The orientation of the second camera module 22 is adjusted by the drone module 21 so that the calibration pixel threshold is as small as possible, thereby making the midpoint pixel position data of the moving video and the midpoint pixel position data of the second detection frame coincide as closely as possible;

[0166] The drone module 21 may adjust the second camera module 22 by changing its own flight posture, or by using a direction adjustment device additionally installed on the drone module 21 .

[0167] C5, obtaining corresponding mobile video direction data through the second direction sensing module 23 and defining it as second calibration direction data, and obtaining corresponding mobile video positioning data through the second positioning module 24 and defining it as second calibration positioning data;

[0168] The second calibration direction data is the direction data corresponding to the second camera module 22 when the midpoint pixel position data of the moving video and the midpoint pixel position data of the second detection frame are close to coincident.

[0169] C6, determining corresponding second abnormal target azimuth data according to the second calibration direction data and the second calibration positioning data;

[0170] The second abnormal target position data is the position information determined based on the combination of the mobile video positioning data and the second calibration position data, that is, the position information of the abnormal target estimated by the mobile monitoring module 20, which can reflect the position information of the abnormal target based on the mobile video positioning data.

[0171] C7, calculating corresponding intersection coordinate data according to the first abnormal target orientation data and the second abnormal target orientation data and defining the data as abnormal target positioning data;

[0172] The abnormal target positioning data is the estimated positioning data of the abnormal target;

[0173] The corresponding intersection coordinates can be determined by combining the first abnormal target position data and the second abnormal target position data, and the intersection coordinates can be identified as the location of the abnormal target;

[0174] On the other hand, combined with the RTK positioning method, the accuracy of abnormal target positioning can be greatly improved.

[0175] Through the above steps, the artificial intelligence-based video surveillance system can determine the corresponding second abnormal target position data through the mobile monitoring module, and combine the first abnormal target position data and the second abnormal target position data to realize the positioning of the abnormal target, thereby enhancing the system's tracking and positioning capabilities for abnormal targets, and providing more reliable and accurate positioning support for security monitoring and emergency response.

[0176] Furthermore, the artificial intelligence-based video surveillance system also includes a target size estimation strategy:

[0177] D1, calculates the corresponding target distance based on the abnormal target positioning data and the fixed-point video positioning data;

[0178] The target distance is the distance calculated based on the abnormal target positioning data and the fixed-point video positioning data, and can be calculated and determined using the Euclidean distance formula based on the positioning data of the two points.

[0179] D2, determine the corresponding horizontal pixel length and vertical pixel length of the detection frame according to the fixed-point abnormal target detection frame data;

[0180] The horizontal pixel length of the detection frame is the pixel length of the detection frame corresponding to the fixed-point abnormal target detection frame data in the horizontal direction;

[0181] The vertical pixel length of the detection frame is the pixel length of the detection frame corresponding to the fixed-point abnormal target detection frame data in the vertical direction.

[0182] D3, calculating a corresponding horizontal pixel size ratio and a vertical pixel size ratio according to a preset horizontal field of view angle, a preset vertical field of view angle, a preset video image pixel height, a preset video image pixel width and a target distance;

[0183] The horizontal field of view angle is a predetermined field of view angle of the first camera module 11 in the horizontal direction;

[0184] The vertical field of view angle is a predetermined field of view angle of the first camera module 11 in the vertical direction;

[0185] The horizontal field of view angle and the vertical field of view angle can be determined when the camera module is purchased, or calculated based on the focal length of the camera module and the size of the camera sensor;

[0186] For example, if the focal length of the camera module and the size of the camera sensor are known, the horizontal field of view angle and the vertical field of view angle can be calculated by the following formula:

[0187] HFOV = 2 * arctan((Ws / 2) / f);

[0188] VFOV = 2 * arctan((Hs / 2) / f);

[0189] Where HFOV is the horizontal field of view, f is the focal length, Ws is the sensor width, VFOV is the vertical field of view, and Hs is the sensor height;

[0190] The video image pixel height is the height of the video image resolution;

[0191] The pixel width of the video image is the width of the video image resolution;

[0192] The horizontal pixel size ratio is the ratio coefficient between the actual size and the pixel size in the video image in the horizontal direction, that is, the actual size corresponding to each pixel in the horizontal direction;

[0193] The vertical pixel size ratio is the ratio coefficient in the vertical direction between the actual size and the pixel size in the video image, that is, the actual size corresponding to each pixel in the numerical direction;

[0194] The horizontal pixel size ratio and the vertical pixel size ratio can be calculated by the following formula:

[0195] Sw = 2 * D * tan(HFOV / 2) / Wi;

[0196] Sh = 2 * D * tan(VFOV / 2) / Hi;

[0197] Among them, Sw is the horizontal pixel size ratio, D is the target distance, Wi is the video image pixel width, Sh is the vertical pixel size ratio, and Hi is the video image pixel height.

[0198] D4, calculate the corresponding estimated horizontal length of the target according to the horizontal pixel length of the detection frame and the horizontal pixel size ratio;

[0199] The estimated horizontal length of the target is the actual horizontal width of the estimated abnormal target, which can be obtained by multiplying the horizontal pixel length of the detection box and the horizontal pixel size ratio.

[0200] D5, calculate the corresponding estimated vertical length of the target according to the vertical pixel length of the detection frame and the vertical pixel size ratio;

[0201] The estimated vertical length of the target is the actual vertical height of the estimated abnormal target, which can be obtained by multiplying the vertical pixel length of the detection box and the vertical pixel size ratio.

[0202] Through the above steps, the artificial intelligence-based video surveillance system can estimate the distance of the abnormal target through the abnormal target positioning data and the fixed-point video positioning data, and estimate the correspondence between the real size and the image pixels according to the distance of the abnormal target and the pixel length and pixel width of the abnormal target in the video image in combination with the optical parameters of the camera module and the image resolution, and then estimate the actual size of the abnormal target, which is conducive to evaluating potential threats, identifying suspicious objects or conducting on-site analysis, and provides more comprehensive and accurate data support for security monitoring and event assessment.

[0203] Furthermore, the artificial intelligence-based video surveillance system also includes a surround camera strategy:

[0204] E1, taking the abnormal target positioning data as the center of the circle, the mobile monitoring module 20 is used to circle around the abnormal target according to the preset flight altitude and the preset circle distance, and the corresponding abnormal target circle video data is obtained;

[0205] The flight altitude is a preset flight altitude of the drone module 21;

[0206] The surrounding distance is a preset distance used to determine the distance between the mobile monitoring module 20 and the abnormal target;

[0207] The abnormal target surrounding video data is the video image data captured when the mobile monitoring module 20 surrounds the abnormal target.

[0208] E2, determining the multi-angle abnormal target detection frame data corresponding to each angle through the abnormal recognition model according to the abnormal target surrounding video data;

[0209] The multi-angle abnormal target detection frame data is the data of the target detection frame determined by the abnormal recognition model in the video image corresponding to each angle in the abnormal target surrounding video data.

[0210] E3, respectively determining the minimum horizontal pixel length and the maximum horizontal pixel length in all multi-angle abnormal target detection frame data;

[0211] The minimum horizontal pixel length is the minimum value of the horizontal pixel lengths of the target detection frames of all multi-angle abnormal target detection frame data, which can reflect the minimum horizontal width of the abnormal target;

[0212] The maximum horizontal pixel length is the maximum value among the horizontal pixel lengths of the target detection frames of all multi-angle abnormal target detection frame data, and can reflect the maximum horizontal width of the abnormal target.

[0213] E4, calculate the estimated area of ​​the corresponding abnormal target according to the product of the minimum horizontal pixel length and the maximum horizontal pixel length;

[0214] The estimated area of ​​the abnormal target is the area of ​​the abnormal target estimated by multiplying the minimum horizontal pixel length and the maximum horizontal pixel length.

[0215] E5, calculates the average vertical pixel length based on all multi-angle abnormal target detection box data;

[0216] The average vertical pixel length is the average of the vertical pixel lengths of the target detection frames of all multi-angle abnormal target detection frame data, reflecting the estimated height of the abnormal target.

[0217] E6, calculate the estimated area occupied by the abnormal target by multiplying the estimated area occupied by the abnormal target and the average vertical pixel length;

[0218] The estimated space occupied by the abnormal target is an estimated value of the space occupied by the abnormal target, which can reflect the volume of the abnormal target.

[0219] Through the above steps, the artificial intelligence-based video surveillance system can fly around the abnormal target according to the abnormal target positioning data and obtain the corresponding surround video image. On the one hand, it can obtain comprehensive image data of the abnormal target, and on the other hand, it can estimate the volume size of the abnormal target based on the surround video image, overcoming the visual bias that may be caused by single-angle observation, and can also provide more comprehensive and three-dimensional target information, which is conducive to identifying objects of complex shapes, assessing the scale of potential threats, and conducting accurate situation assessments.

[0220] refer to Figure 2 The present invention also provides a video monitoring method based on artificial intelligence, comprising the following steps:

[0221] F1, obtain the corresponding fixed-point video data through the preset fixed-point monitoring module;

[0222] F2, obtaining corresponding fixed-point video direction data through the fixed-point monitoring module;

[0223] F3, obtaining corresponding fixed-point video positioning data through the fixed-point monitoring module;

[0224] F4, generates fixed-point abnormal target detection frame data based on the fixed-point video data using a pre-trained abnormal recognition model;

[0225] F5, determining abnormal target orientation data according to the fixed-point abnormal target detection frame data, the fixed-point video direction data and the fixed-point video positioning data using a preset target orientation estimation method;

[0226] F6, moving the preset mobile monitoring module according to the abnormal target position data and obtaining the corresponding mobile video data;

[0227] F7, obtaining corresponding mobile video direction data through the mobile monitoring module;

[0228] F8, obtaining corresponding mobile video positioning data through the mobile monitoring module;

[0229] F9, transmits the moving video data, the moving video direction data and the moving video positioning data to the preset recognition control module.

[0230] Through the above steps, the artificial intelligence-based video surveillance method can achieve seamless collaboration of monitoring through fixed-point monitoring modules and mobile monitoring modules, and can improve the ability to obtain monitoring information. In particular, when an abnormal target is found, the mobile monitoring module can move close to the abnormal target to obtain close-range high-definition monitoring video for analysis and viewing, which not only improves the accuracy and comprehensiveness of monitoring, but also enhances the system's ability to respond to emergencies, providing strong information support for security management.

[0231] The above are all preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Any feature disclosed in this specification (including the abstract and drawings), unless otherwise stated, can be replaced by other equivalent or alternative features with similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.

Claims

1. A video surveillance system based on artificial intelligence, characterized in that: include: Fixed-point monitoring module, used to obtain surrounding video images at the installation location; Mobile monitoring module, for moving to the target location to obtain the corresponding video image; An identification control module for analyzing and identifying the video images acquired by the fixed-point monitoring module and the mobile monitoring module, and controlling the fixed-point monitoring module and the mobile monitoring module; Wherein, the fixed-point monitoring module, the mobile monitoring module and the identification control module are communicatively connected; The fixed-point monitoring module includes a first camera module, a direction adjustment module, a first direction sensing module, a first positioning module and a first communication module, wherein the first direction sensing module is arranged in the first camera module, the first camera module is arranged in the direction adjustment module, and the first camera module, the direction adjustment module, the first direction sensing module and the first positioning module are respectively data-connected to the first communication module; Wherein, the mobile monitoring module includes a drone module, a second camera module, a second direction sensing module, a second positioning module and a second communication module, the second camera module, the second direction sensing module, the second positioning module and the second communication module are respectively arranged in the drone module, and the second camera module, the second direction sensing module and the second positioning module are respectively data-connected to the second communication module; Wherein, the first communication module, the second communication module and the identification control module are data connected; The artificial intelligence-based video surveillance system further includes an abnormal monitoring strategy, which includes the following steps: A1, acquiring corresponding fixed-point video data through the first camera module of the fixed-point monitoring module; A2, acquiring corresponding fixed-point video direction data through the first direction sensing module of the fixed-point monitoring module; A3, acquiring corresponding fixed-point video positioning data through the first positioning module of the fixed-point monitoring module; A4, transmitting the fixed-point video data, the fixed-point video direction data and the fixed-point video positioning data to the identification control module through the first communication module of the fixed-point monitoring module; A5, generating fixed-point abnormal target detection frame data through the recognition control module with a pre-trained abnormal recognition model according to the fixed-point video data; A6, determining first abnormal target orientation data according to the fixed-point abnormal target detection frame data, the fixed-point video direction data and the fixed-point video positioning data using a preset target orientation estimation method; A7, transmitting the first abnormal target position data to the mobile monitoring module; A8, the drone module of the mobile monitoring module moves according to the first abnormal target position data and obtains corresponding mobile video data through the second camera module; A9, acquiring corresponding mobile video direction data through the second direction sensing module of the mobile monitoring module; A10, acquiring corresponding mobile video positioning data through the second positioning module of the mobile monitoring module; A11, transmitting the mobile video data, the mobile video direction data and the mobile video positioning data to the recognition control module through the second communication module of the mobile monitoring module; The target position estimation method comprises the following steps: B1, determining the corresponding midpoint pixel position data of the first detection frame according to the fixed-point abnormal target detection frame data; B2, determining a corresponding fixed-point abnormal target offset vector according to the preset fixed-point video midpoint pixel position data and the first detection frame midpoint pixel position data; B3, adjusting the orientation of the first camera module through the direction adjustment module according to the fixed-point abnormal target offset vector until the modulus of the fixed-point abnormal target offset vector is less than a preset calibration pixel threshold; B4, acquiring corresponding fixed-point video direction data through the first direction sensing module and defining it as first calibration direction data; B5, determining corresponding first abnormal target orientation data according to the first calibration direction data and the fixed-point video positioning data; The artificial intelligence-based video surveillance system further includes an abnormal target positioning strategy, which includes the following steps: C1, generating moving abnormal target detection frame data through the recognition control module using an abnormal recognition model according to the moving video data; C2, determining the corresponding second detection frame midpoint pixel position data according to the moving abnormal target detection frame data; C3, determining the corresponding abnormal moving target offset vector according to the preset moving video midpoint pixel position data and the second detection frame midpoint pixel position data; C4, adjusting the orientation of the second camera module through the drone module according to the abnormal moving target offset vector until the modulus of the abnormal moving target offset vector is less than the calibration pixel threshold; C5, acquiring corresponding mobile video direction data through the second direction sensing module and defining it as second calibration direction data, and acquiring corresponding mobile video positioning data through the second positioning module and defining it as second calibration positioning data; C6, determining corresponding second abnormal target azimuth data according to the second calibration direction data and the second calibration positioning data; C7, calculating corresponding intersection coordinate data according to the first abnormal target orientation data and the second abnormal target orientation data and defining them as abnormal target positioning data.

2. The video surveillance system based on artificial intelligence according to claim 1, characterized in that: The fixed-point monitoring module further includes a landing platform and a wireless charging transmitting module, wherein the wireless charging transmitting module is arranged on the landing platform, and the mobile monitoring module further includes a wireless charging receiving module, wherein the wireless charging receiving module is arranged on the drone module; Wherein, the drone module can selectively land on the landing platform, and the wireless charging transmitting module and the wireless charging receiving module can selectively form a power supply connection.

3. The video surveillance system based on artificial intelligence according to claim 1, characterized in that: Further includes target size estimation strategy: D1, calculates the corresponding target distance based on the abnormal target positioning data and the fixed-point video positioning data; D2, determine the corresponding horizontal pixel length and vertical pixel length of the detection frame according to the fixed-point abnormal target detection frame data; D3, calculating a corresponding horizontal pixel size ratio and a vertical pixel size ratio according to a preset horizontal field of view angle, a preset vertical field of view angle, a preset video image pixel height, a preset video image pixel width and a target distance; D4, calculate the corresponding estimated horizontal length of the target according to the horizontal pixel length of the detection frame and the horizontal pixel size ratio; D5, calculates the corresponding estimated vertical length of the target based on the vertical pixel length of the detection box and the vertical pixel size ratio.

4. The video surveillance system based on artificial intelligence according to claim 3 is characterized in that: Further including surround camera strategy: E1, taking the abnormal target positioning data as the center of the circle, circling the abnormal target through the mobile monitoring module according to the preset flight altitude and the preset circling distance and obtaining the corresponding abnormal target circling video data; E2, determining the multi-angle abnormal target detection frame data corresponding to each angle through the abnormal recognition model according to the abnormal target surrounding video data; E3, respectively determining the minimum horizontal pixel length and the maximum horizontal pixel length in all multi-angle abnormal target detection frame data; E4, calculate the estimated area of ​​the corresponding abnormal target according to the product of the minimum horizontal pixel length and the maximum horizontal pixel length; E5, calculates the average vertical pixel length based on all multi-angle abnormal target detection box data; E6, calculate the estimated area occupied by the abnormal target by multiplying the estimated area occupied by the abnormal target and the average vertical pixel length.