Embedded multi-lens panoramic video intelligent monitoring system and construction method

Abstract

The invention relates to the technical field of intelligent monitoring, in particular to an embedded multi-lens panoramic video intelligent monitoring system and a construction method thereof, a computing energy SE5 micro server and an NVIDIA Jetson board card construction system are integrated, a deep learning neural network is adapted to the computing energy SE5 micro server, and the NVIDIA Jetson board card construction system is adapted to the NVIDIA Jetson board card construction system. The functions of network model weight conversion, sample generation, deployment training, test analysis and the like are completed; embedded spherical panoramic splicing software for lookup table calculation and GPU parallel acceleration is adapted on a Jetson board card platform to complete lookup table import, pixel mapping, suture line fusion and CUDA deployment and optimization functions, and meanwhile, a YOLO-based vehicle illegal parking detection, retrograde driving detection and human-vehicle statistics module is designed according to the parking lot monitoring requirements of the highway service area, so that the parking lot monitoring requirements of the highway service area are met. According to the method, the real-time splicing and intelligent analysis processing functions of the high-resolution panoramic video are realized on a heterogeneous embedded platform through collaborative optimization, and the application bottleneck of OMS panoramic video splicing and intelligent analysis processing on the embedded platform at present is solved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent monitoring technology, and in particular to an embedded multi-lens panoramic video intelligent monitoring system and a construction method thereof. Background Art

[0002] Compared with traditional "pinhole" surveillance cameras that can only capture limited positions and viewing angles, the multi-lens panoramic imaging system OMS (Omnidirectional Multi-Camera System) with a 360 or 180-degree field of view can greatly reduce the cost of monitoring projects such as wiring and personnel construction in a wide-area monitoring range (such as urban stations, airports, ports, etc.), and help to grasp the overall security situation of the monitoring area at a macro level, and has broad market application prospects.

[0003] However, currently OMS relies on high-performance PCs or graphics workstations to implement panoramic stitching calculations for multiple videos. This has the problems of large device size, high power consumption, and poor mobility, which limits its application in embedded and mobile scenarios. In addition, the panoramic video obtained by OMS through expanding the viewing angle through the spherical projection model has serious distortion, which poses a challenge to subsequent video intelligent analysis tasks.

[0004] In recent years, video intelligent processing based on deep learning neural networks (such as YOLO) has shined in the construction and application of security systems, but the existing video intelligent processing network is mainly aimed at traditional "pinhole" videos, which is difficult to be directly applied to the spherical panoramic video output by OMS. The video intelligent processing process and algorithm under the existing network model are difficult to accurately obtain the monitoring target feature information from the spherical panoramic video, thereby increasing the risk of target missed detection and false detection. In addition, the traditional serial processing process of "panoramic stitching first, then intelligent analysis" is undoubtedly a greater challenge for embedded computing devices and platforms with limited computing resources and power consumption constraints. Summary of the invention

[0005] The purpose of the present invention is to provide an embedded multi-lens panoramic video intelligent monitoring system and a construction method, to construct a multi-lens spherical panoramic video intelligent monitoring system on an embedded platform, and to realize violation behavior detection and intelligent analysis and processing.

[0006] To achieve the above-mentioned object, the present invention provides an embedded multi-lens panoramic video intelligent monitoring system, including a video decoding module, a lookup table processing module, a panoramic image stitching and fusion module, an AI intelligent analysis module, a video encoding module and a video stream pushing module;

[0007] The video information collected by each network camera flows into the panoramic image stitching and fusion module and the AI ​​intelligent analysis module through the video decoding module, and the images are stitched in combination with the lookup table of the lookup table processing module. At the same time, the AI ​​intelligent analysis module analyzes each image to obtain the analysis results. The video encoding module encodes the panoramic image video stream that contains the analysis results and is stitched, and the video stream push module pushes the video stream to the video server as an RTSP video stream.

[0008] Among them, after reading the video stream data of multiple cameras and the corresponding lookup table data, the panoramic image stitching and fusion module obtains a complete panoramic image through bilinear interpolation, grayscale assignment and weighted fusion processing. The lookup table in this process is a lookup table from the panoramic image to the original image.

[0009] Among them, the AI ​​intelligent analysis module reads the lookup table while decoding the input multi-channel camera video streams, and divides the input camera video streams and the lookup table into blocks, and then performs AI intelligent algorithm detection on each image block separately and in parallel to obtain the coordinates, probability and category information of the target on the original image, and finally maps the pixel coordinate information of the original image block to the lookup table to obtain the panoramic image coordinate information corresponding to the coordinate points on each original image, and performs coordinate mapping on the four corner points of a detection box respectively to obtain the coordinate information corresponding to the detection box on the panoramic image, and display the detection box, detection target probability and category information on the panoramic image. The lookup table in this process is a lookup table from the original image to the panoramic image.

[0010] Among them, the AI ​​intelligent analysis module includes a parking violation detection module, a vehicle reverse detection module and a person and vehicle statistics module;

[0011] The illegal parking detection module is used to detect illegal parking in parking spaces and lanes. The vehicle reversing detection module is used to identify whether the vehicle is traveling in the specified direction of the lane and to determine whether reversing occurs. The person and vehicle statistics module is used for comprehensive statistics of information in the service area and real-time monitoring.

[0012] Among them, the lookup table processing module eliminates the ghosting phenomenon by redesigning the lookup table. Specifically, when a person appears in multiple original videos at the same time, a ghosting phenomenon of multiple detection frames will appear in the panoramic image. By designing the lookup table, the lookup table coordinates of each original video do not overlap with each other, and the overlapping area of ​​multiple videos only belongs to one lookup table, so that there is only one detection frame when projecting to the panoramic video, eliminating the ghosting phenomenon.

[0013] Furthermore, the present invention also proposes a method for constructing an embedded multi-lens panoramic video intelligent monitoring system, which is used to construct the embedded multi-lens panoramic video intelligent monitoring system, and includes the following steps:

[0014] Step 1: Build on the Suanneng SE5 microserver and NVIDIA Jetson board to form an architecture with ARM+GPU+TPU / NPU co-processing;

[0015] Step 2: Adapt the deep learning neural network on the Suanneng SE5 microserver and the embedded spherical panorama stitching software with lookup table calculation and GPU parallel acceleration on the NVIDIA Jetson board;

[0016] Step 3: Based on the monitoring requirements of the service area parking lot, the AI ​​intelligent analysis module is used to design the illegal parking detection module, the vehicle reverse detection module and the pedestrian and vehicle statistics module through the deep learning neural network.

[0017] The present invention provides an embedded multi-lens panoramic video intelligent monitoring system and a construction method. The embedded multi-lens spherical panoramic video intelligent monitoring system is constructed by integrating a Suanneng SE5 microserver and an NVIDIA Jetson board. A deep learning neural network is adapted on the Suanneng SE5 microserver to complete functions such as network model weight conversion, sample generation, deployment training, and test analysis. On the Jetson board platform, embedded spherical panoramic stitching software with lookup table calculation and GPU parallel acceleration is adapted to complete lookup table import, pixel mapping, seam line fusion, and CUDA deployment and optimization functions. At the same time, in view of the monitoring needs of parking lots in highway service areas, a vehicle illegal parking detection, reverse driving detection, and human and vehicle statistics module based on YOLO are designed. The present invention realizes the real-time stitching and intelligent analysis and processing functions of high-resolution panoramic videos on heterogeneous embedded platforms through deep collaborative optimization of software and hardware, solving the current application bottleneck of OMS panoramic video stitching and intelligent analysis and processing on embedded platforms. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 The invention discloses a principle block diagram of an embedded multi-lens panoramic video intelligent monitoring system.

[0020] Figure 2 It is a structural schematic diagram of the panoramic image stitching and fusion module of the present invention.

[0021] Figure 3 It is a schematic diagram of the image splicing lookup table mapping process of the present invention.

[0022] Figure 4It is a structural diagram of the AI ​​intelligent analysis module of the present invention.

[0023] Figure 5 It is a schematic diagram of the AI ​​intelligent analysis coordinate mapping process of the present invention.

[0024] Figure 6 It is the present invention to search for representation intent.

[0025] Figure 7 It is a schematic diagram of a system flow in a specific embodiment of the present invention.

[0026] Figure 8 It is a schematic diagram of the computing energy micro-server system architecture of a specific embodiment of the present invention.

[0027] Fig. 9 It is a schematic diagram of five-channel original video images according to a specific embodiment of the present invention.

[0028] Fig.10 It is a final operation effect diagram formed by splicing specific embodiments of the present invention.

[0029] Fig.11 It is a comparison diagram of ghost removal effects of a specific embodiment of the present invention.

[0030] Fig.12 It is a diagram showing the effect of vehicle reverse driving detection in a specific embodiment of the present invention.

[0031] Fig.13 It is a rendering of the vehicle illegal parking detection effect in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0032] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0033] See also Figure 1 , the present invention provides an embedded multi-lens panoramic video intelligent monitoring system, including a video decoding module, a lookup table processing module, a panoramic image stitching and fusion module, an AI intelligent analysis module, a video encoding module and a video stream pushing module;

[0034] The video information collected by each network camera flows into the panoramic image stitching and fusion module and the AI ​​intelligent analysis module through the video decoding module, and the images are stitched in combination with the lookup table of the lookup table processing module. At the same time, the AI ​​intelligent analysis module analyzes each image to obtain the analysis results. The video encoding module encodes the panoramic image video stream that contains the analysis results and is stitched, and the video stream push module pushes the video stream to the video server as an RTSP video stream.

[0035] The following is a further explanation based on specific modules:

[0036] like Figure 2 As shown in the figure, it is the principle block diagram of the panoramic image stitching and fusion module. After reading the video stream data of multiple cameras and the corresponding lookup table data, the lookup table here is the lookup table from the panoramic image to the original image. First, block processing is performed. ID[*] represents the lookup table number corresponding to each original video. Taking the original video 0 as an example, assuming that there is an integer coordinate (u, v) point on the panoramic image, the floating-point coordinate of the original video 0 corresponding to it is (x, y) according to the lookup table. Figure 3 As shown in the figure, the weight of the array weight corresponding to the coordinate point (x, y) is w. Rounding down (x, y) gives (x 1 ,y 1 ), and perform bilinear interpolation sampling on it. Let the coordinate (x 1 ,y 1 ) is the gray value f(x 1 ,y 1 ), from which we can get the gray value at (x, y):

[0037] f(x,y)=(1-σ x )(1-σ y )f(x 1 ,y 1 )+σ x (1-σ y )f(x 1 +1,y 1 )+σ x σ y f(x 1 +1,y 1 +1)+(1-σ x )σ y f(x 1 ,y 1 +1)

[0038] Assume that the initial pixel value of the interpolation point (u, v) on the panoramic image is I 0 (u,v), then use the formula

[0039] I(u,v)=[I 0(u,v)(1-w)+f(x,y)w]C

[0040] The pixel value I(u,v) at the coordinate point (u,v) in the panoramic image can be obtained, and C is the calculated uniformity coefficient of the original image 0. According to the above process, all the data in the lookup table can be traversed to obtain a complete panoramic image.

[0041] like Figure 4 The figure shows the principle block diagram of the AI ​​intelligent analysis module. The lookup table is read while decoding the input multi-channel camera video streams. Unlike the previous lookup table, this lookup table is a lookup table from the original image to the panoramic image. The input camera video stream and the lookup table are divided into blocks, which are divided into image blocks and lookup tables corresponding to each camera. First, each image block is separately and parallelly detected by AI intelligent algorithms, such as YOLO and other algorithms, to obtain the coordinates, probability and category information of the target on the original image, and the pixel coordinate information of the original image block is mapped to the lookup table to obtain the panoramic image coordinate information corresponding to the coordinate points on each original image. The four corner points of a detection frame are mapped to coordinates (the AI ​​intelligent analysis coordinate mapping process is shown in the figure). Figure 5 As shown in the figure, the coordinate information corresponding to the detection frame on the panoramic image is obtained, and the detection frame, detection target probability, and category information are displayed on the panoramic image. Because the lookup table is designed, the lookup table from each original video to the panoramic image does not overlap with each other, so when the same target appears in multiple original videos at the same time, the ghosting phenomenon of multiple detection frames in the panoramic image can be eliminated. When multiple intelligent analysis algorithms are used to detect multiple original videos, multiple threads can be used, each thread corresponds to an intelligent analysis algorithm, to maximize parallelization.

[0042] The AI ​​intelligent analysis module is mainly divided into three parts, namely core application modules such as illegal parking detection, vehicle reverse detection and pedestrian and vehicle statistics.

[0043] 1) Illegal parking detection module

[0044] The illegal parking detection module is divided into illegal parking in parking spaces and illegal parking in lanes. The logical process of illegal parking detection in parking spaces is as follows:

[0045] ① Traverse all parking spaces and check whether the center point of each car is within the corresponding parking space area.

[0046] ②If the vehicle does not match the parking space type (such as a large truck parked in a small car space), it will be marked as illegal parking.

[0047] ③For each parking space marked as illegally parked, record the start time of the illegal parking and calculate the duration of the illegal parking.

[0048] ④When the illegal parking duration exceeds the preset threshold, an alarm is triggered and the relevant information is saved in the database.

[0049] ⑤ After the alarm is triggered, the illegal parking space will be marked, the polygonal frame of the parking space will be marked in yellow, and the illegal parking time will be displayed next to the parking space. After the alarm, the vehicle record will be updated to prevent repeated alarms.

[0050] ⑥ If a parking space is empty, reset the status of the parking space and clear the illegal parking sign and timing data.

[0051] ⑦Finally, the system counts the number of vacant parking spaces in real time and displays them according to vehicle type.

[0052] Lane parking violation detection is designed to detect illegal parking of vehicles in the lane. The logical process of lane parking violation detection is as follows:

[0053] ① Detection area drawing: On the input image, first draw the illegal parking detection areas of all lanes monitored by the current camera to facilitate intuitive viewing of the monitoring range of each lane.

[0054] ② Vehicle detection and area determination: Traverse all detected vehicles, calculate the coordinates of the center point of each vehicle, and determine whether the point falls into the illegal parking area of ​​a certain lane.

[0055] ③ Vehicle status update: For vehicles that already exist in the database record, check whether the vehicle has moved. If the vehicle has moved, reset the parking violation related information, update the vehicle border and status, and mark it as "moved" on the image. If the vehicle has not moved, record the current time as the parking violation start time (only record it for the first time), and continue to count the parking violation.

[0056] ④ Remove invalid vehicle information: For vehicles that no longer appear in the test results, remove their related records from the database to ensure data synchronization and accuracy.

[0057] ⑤ Calculation of illegal parking time and alarm triggering: Traverse the vehicle information in all lanes and calculate the illegal parking time of each vehicle (if the vehicle has not moved, calculate the difference between the current time and the start time of illegal parking). When the illegal parking time exceeds 10 seconds, the image shows the illegal parking status of the vehicle and its illegal parking time. If the illegal parking time of the vehicle exceeds the set threshold (timeThreshold) and the alarm has not been triggered, set the alarm flag to true and issue an alarm signal immediately.

[0058] ⑥Return result: If any vehicle triggers the alarm, it returns true; if no vehicle triggers the alarm, it returns false.

[0059] 2) Vehicle reverse detection module

[0060] The core purpose of vehicle reverse detection is to identify whether the vehicle is driving in the specified direction of the lane and determine whether reverse driving occurs. The specific detection logic process is as follows:

[0061] ① Detect vehicle position: In each frame, traverse all detected vehicle bounding boxes and determine whether the center point of the vehicle is within the bounding box of the lane. If the vehicle is in the lane, obtain the vehicle ID and record its current rectangular box.

[0062] ② Determine whether the vehicle is moving: Determine whether the vehicle is moving by calculating the displacement between the center point of each vehicle and the center point of the previous frame. If the distance exceeds the set threshold, the vehicle is considered to be moving.

[0063] ③Reverse driving judgment: By comparing the rectangular frame of the vehicle in the current frame with the rectangular frame of the previous frame, combined with the driving direction of the lane, it is judged whether the vehicle is in reverse driving behavior. If the position of the vehicle's rectangular frame changes in the opposite direction and the reverse driving behavior lasts for a certain period of time, it is judged that the vehicle is in reverse driving behavior.

[0064] ④ Reverse counting and alarm: When reverse behavior is detected, the reverse count is increased. If the reverse behavior lasts for more than a certain number of frames (such as 40 frames), the reverse rectangle of the vehicle and the start time are recorded. If the reverse duration exceeds the set threshold (such as 5 seconds or other time standards), an alarm is triggered and the vehicle information is returned.

[0065] ⑤ Alarm and display: When a vehicle is driving against traffic, the system will draw a red rectangle in the image and display a prompt text of "driving against traffic" or "suspected of driving against traffic" above the vehicle. If the driving against traffic continues, the system will trigger an alarm according to the set conditions and return relevant alarm information.

[0066] 3) People and Vehicle Statistics Module

[0067] The service area vehicle and passenger information statistics module integrates multiple intelligent analysis functions to realize the statistics of the number of security personnel, tourists, cars, trucks, tankers, and buses, as well as real-time monitoring of vacant parking spaces and total parking spaces. Its logical structure includes the following steps:

[0068] ① Initialization and data loading: Load the parking space configuration file, parse the parking space configuration information, and obtain detailed data of various parking spaces, including quantity and type. Initialize the lane range and vehicle information related to reverse detection to provide data support for subsequent detection and processing.

[0069] ② Target detection and classification: Use the YOLOv7 model to detect all targets in the picture. According to the classification labels, different types of vehicles and people are counted separately, and target tracking is performed on the detection results.

[0070] ③ Parking space management and calculation: Initialize parking space information, obtain all parking space information, and identify parking space types. According to the vehicle detection results, determine the occupancy of each parking space. If a parking space is occupied by a vehicle, it is marked as occupied; otherwise, it is marked as vacant. By subtracting the number of occupied parking spaces from the total number of parking spaces, the current number of vacant parking spaces is counted in real time.

[0071] ④ Reverse detection and processing: Call the reverse detection module to detect the reverse behavior of the vehicle. If the vehicle is detected to be in reverse, the system will update the relevant alarm information and record it in the corresponding statistical module.

[0072] ⑤ Comprehensive statistics and output: Calculate and summarize the number of security personnel, tourists, vehicles of all types, and vacant and total parking spaces in the current screen. The statistical results are presented in a visual way on the interface or saved in a log file. At the same time, the statistical information can be sent to other systems through the interface.

[0073] Furthermore, the present invention also proposes a method for constructing an embedded multi-lens panoramic video intelligent monitoring system, which is used to construct the embedded multi-lens panoramic video intelligent monitoring system, and includes the following steps:

[0074] Step 1: Build on the Suanneng SE5 microserver and NVIDIA Jetson board to form an architecture with ARM+GPU+TPU / NPU co-processing;

[0075] Step 2: Adapt the deep learning neural network on the Suanneng SE5 microserver and the embedded spherical panorama stitching software with lookup table calculation and GPU parallel acceleration on the NVIDIA Jetson board;

[0076] Step 3: Based on the monitoring requirements of the service area parking lot, the AI ​​intelligent analysis module is used to design the illegal parking detection module, the vehicle reverse detection module and the pedestrian and vehicle statistics module through the deep learning neural network.

[0077] In the present invention, by combining CPU, ARM, GPU and TPU / NPU, the CPU side mainly completes the calculation of the homography transformation matrix of the reference image and the image to be spliced, that is, image registration. Then, the interpolation sampling coordinate position of each pixel is calculated by the homography matrix, and a lookup table is generated. The embedded ARM side is specifically responsible for the acquisition, decoding and buffering of video images, and transmits the video stream data to the splicing board. Subsequently, the interpolation sampling of the image is directly performed through the lookup table to achieve efficient fusion of video images. Finally, the spliced ​​video image is encoded and pushed to the video server through the RTSP video stream. When realizing image splicing and intelligent analysis, the processing unit of the embedded board can be used to accelerate the algorithm, and then the AI ​​edge computing module can be deployed on the embedded device according to specific needs to achieve a relatively balanced state and improve the overall video surveillance system performance.

[0078] The following is described in conjunction with specific embodiments:

[0079] like Figure 7 As shown, the basic process of the present invention is: start to power the ARM architecture microserver and the splicing board, the board initializes the Linux operating system and hardware structure, uses the computing microserver VPU video processing unit to decode multiple video streams, and stores the decoded video streams to the eMMC on the ARM side, the splicing board reads the lookup table 1 (the lookup table of the panoramic image to the original image) and divides the lookup table 1 into blocks according to each original video, and obtains the splicing image interpolation sampling floating-point horizontal coordinate MapX corresponding to each video image, the splicing image interpolation sampling floating-point vertical coordinate MapY corresponding to each video image, and the pixel grayscale value obtained after each video image is interpolated and sampled by the lookup table The weight size MapW in the spliced ​​image, and then weighted fusion is performed to obtain a panoramic spliced ​​image. While performing panoramic image stitching, the AI ​​microserver board performs AI algorithm detection on each video stream separately, and uses lookup table 2 (the lookup table from original image to panoramic image) to map the detection frame detected on each original image to the panoramic image to obtain the final panoramic stitching image, which is then pushed to the NVR video server via the RTSP network, where the AI ​​microserver board and the stitching board communicate through the network port board.

[0080] Figure 8 This is the system architecture diagram of the computing energy micro server. Fig. 9 These are the original images of five channels of video at a certain moment. The effect after the system is running is as follows: Fig.10 As shown, we can see that the system has achieved panoramic stitching of five-way videos, implemented the detection of AI intelligent analysis algorithm, and eliminated ghosting. Fig.11 This is a comparison of the effects before and after removing ghosting. Fig.12 and Fig.13This is a schematic diagram of the vehicle reversing and illegal parking detection effect. Fig.12 Reverse in the figure represents the reverse sign, and the value following it is the reverse time. The upper left corner represents the various statistical information collected by the human and vehicle information statistics module. Fig.13 NO_P stands for illegal parking sign, followed by the illegal parking time.

[0081] What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. An embedded multi-lens panoramic video intelligent monitoring system, characterized in that: It includes video decoding module, lookup table processing module, panoramic image stitching and fusion module, AI intelligent analysis module, video encoding module and video stream push module; The video information collected by each network camera flows into the panoramic image stitching and fusion module and the AI ​​intelligent analysis module through the video decoding module, and the images are stitched in combination with the lookup table of the lookup table processing module. At the same time, the AI ​​intelligent analysis module analyzes each image to obtain the analysis results. The video encoding module encodes the panoramic image video stream that contains the analysis results and is stitched, and the video stream push module pushes the video stream to the video server as an RTSP video stream.

2. The embedded multi-lens panoramic video intelligent monitoring system according to claim 1, characterized in that: After reading the video stream data of multiple cameras and the corresponding lookup table data, the panoramic image stitching and fusion module obtains a complete panoramic image through bilinear interpolation, grayscale assignment and weighted fusion processing. The lookup table in this process is a lookup table from the panoramic image to the original image.

3. The embedded multi-lens panoramic video intelligent monitoring system according to claim 2, characterized in that: The AI ​​intelligent analysis module decodes the input multi-channel camera video streams and reads the lookup table at the same time, divides the input camera video streams and the lookup table into blocks, and then performs AI intelligent algorithm detection on each image block separately and in parallel to obtain the coordinates, probability and category information of the target on the original image. Finally, the pixel coordinate information of the original image block is mapped to the lookup table to obtain the panoramic image coordinate information corresponding to the coordinate points on each original image. The four corner points of a detection frame are mapped to obtain the coordinate information corresponding to the detection frame on the panoramic image, and the detection frame, detection target probability and category information are displayed on the panoramic image. The lookup table in this process is a lookup table from the original image to the panoramic image.

4. The embedded multi-lens panoramic video intelligent monitoring system according to claim 3, characterized in that: The AI ​​intelligent analysis module includes a parking violation detection module, a vehicle reverse detection module and a vehicle and pedestrian statistics module; The illegal parking detection module is used to detect illegal parking in parking spaces and lanes. The vehicle reversing detection module is used to identify whether the vehicle is traveling in the specified direction of the lane and to determine whether reversing occurs. The person and vehicle statistics module is used for comprehensive statistics of information in the service area and real-time monitoring.

5. The embedded multi-lens panoramic video intelligent monitoring system according to claim 4, characterized in that: The lookup table processing module eliminates the ghosting phenomenon by redesigning the lookup table. Specifically, when a person appears in multiple original videos at the same time, a ghosting phenomenon of multiple detection frames will appear in the panoramic image. By designing the lookup table, the lookup table coordinates of each original video do not overlap with each other, and the overlapping area of ​​the multiple videos only belongs to one of the lookup tables, so that there is only one detection frame when projecting to the panoramic video, eliminating the ghosting phenomenon.

6. A method for constructing an embedded multi-lens panoramic video intelligent monitoring system, used for constructing the embedded multi-lens panoramic video intelligent monitoring system as claimed in any one of claims 1 to 5, characterized in that: The following steps are involved: Step 1: Build on the Suanneng SE5 microserver and NVIDIA Jetson board to form an architecture with ARM+GPU+TPU / NPU co-processing; Step 2: Adapt the deep learning neural network on the Suanneng SE5 microserver and the embedded spherical panorama stitching software with lookup table calculation and GPU parallel acceleration on the NVIDIA Jetson board; Step 3: Based on the monitoring needs of the service area parking lot, the AI ​​intelligent analysis module is used to design the illegal parking detection module, the vehicle reverse detection module and the pedestrian and vehicle statistics module through the deep learning neural network.

Images