A video analysis system and method based on edge computing

Abstract

This invention discloses a video analysis system and method based on edge computing, belonging to the field of computer vision. The video analysis system includes multiple video analysis devices, a data scheduling and management module, a big data center collection module, and a terminal device module. The method is as follows: after device initialization, the interface adaptive matching module automatically identifies the connected interface devices and assigns them addresses; the edge computing module receives video streams from the upper-layer video analysis devices, extracts frames of the real-time image, uses an optimized deep learning algorithm to obtain frame target features, and determines whether a set target exists in the current frame; in conjunction with the data scheduling and management module, it adjusts the edge computing resources according to a strategy; the calculated information is uploaded to the big data center collection module, which processes it centrally and then pushes the recognition results to the user terminal. This invention can quickly respond to on-site situations and has significant implications for improving the intelligence of industrial vision.

Description

Technical Field

[0001] This invention belongs to the field of video target and behavior analysis and computer vision artificial intelligence, and in particular relates to a video analysis system and method based on edge computing. Background Technology

[0002] With urbanization, traditional retrieval-based monitoring methods can no longer meet people's growing demands for security and efficiency. On the one hand, traditional monitoring methods mainly involve simple training for supervisors, who then rely on reminders and recording of violators' information, finally collecting evidence by retrieving surveillance footage. This approach has drawbacks such as unstable monitoring efficiency and significant susceptibility to the energy and subjective will of supervisors. On the other hand, with the development of artificial intelligence, visual recognition is being used in modern monitoring technology. This technology has advantages such as fast response speed, continuous operation, and low monitoring costs. However, existing video detection technologies mostly rely on centralized servers. This approach is not suitable for areas with a wide detection range, where the number of network nodes and the effective network length are large, resulting in high network latency and an inability to provide timely reminders. Furthermore, it struggles to meet the needs of complex and open scenarios in terms of detection speed and accuracy.

[0003] Therefore, in order to achieve modern intelligent video detection and analysis and improve video analysis speed, this invention proposes an intelligent video analysis method and device based on edge computing. By employing edge computing, the transmission and computation latency from scene changes to result generation is reduced. The edge terminal performs decoding, transmission, acquisition, target detection inference analysis, encoding, and forwarding of the video on-site, directly transmitting the results to the data center for feedback recording, thus structurally improving computation speed.

[0004] Currently, some patents for video analysis methods based on edge terminals have emerged. For example, Chinese patent application number 202210741957.6, with a publication date of September 30, 2022, discloses an intelligent video analysis method and system based on edge terminals. This method includes acquiring video data; acquiring image data to be analyzed based on the video data; constructing an AI model; optimizing the AI ​​model through inter-layer fusion technology; using the optimized AI model to infer the sample data to be analyzed to obtain target analysis results; and post-processing the target analysis results to obtain target feature information, thereby achieving intelligent video analysis. The drawback of this method is that it only uses edge terminals for simple analysis of video information and does not consider situations where computational pressure increases sharply. Generally, the computing power of edge terminals is limited, requiring hardware resource scheduling or increased hardware budget. Given a fixed hardware budget, using algorithms to schedule resources is undoubtedly a better choice. Clearly, this method lacks flexibility, has a high single-point device failure rate, and is prone to downtime.

[0005] For example, Chinese patent application No. 202010315164.9, published on July 28, 2020, discloses a smart terminal video analysis algorithm combining edge computing and deep learning. This method integrates edge computing and deep learning into a smart terminal video analysis algorithm, equipping edge devices with a moving target detection model to process raw video data in real time and analyze moving targets within the video, effectively reducing network latency. However, this method has a drawback: because it doesn't employ time-sharing processing for the algorithm, real-time detection of video frames keeps the machine running at full load, wasting significant schedulable resources, increasing power consumption, and shortening hardware lifespan. To ensure real-time target monitoring without missing any targets, time-sharing detection should be implemented, increasing the amount of schedulable hardware resources.

[0006] Therefore, there is an urgent need to develop a video analytics system and method based on edge computing that can quickly respond to on-site situations. Summary of the Invention

[0007] 1. The problem to be solved

[0008] To address the issues of high equipment failure rates and insufficient response speeds in existing video analytics systems, this invention provides a video analytics system based on edge computing, which can improve the time and cost of intelligent transformation for enterprises and increase transformation efficiency.

[0009] 2. Technical Solution

[0010] To solve the above problems, the technical solution adopted by the present invention is as follows:

[0011] A video analytics system based on edge computing includes multiple video analytics devices, a data scheduling and management module, a big data center collection module, and a terminal device module. Each video analytics device integrates a video information acquisition module, an interface adaptive matching module, and an edge computing module connected sequentially. The interface adaptive matching module can be an RJ45 interface adapter, capable of automatically identifying the device connected to the corresponding interface, i.e., automatically identifying the video information acquisition module's device interface and assigning an IP address. The video information acquisition module acquires video data and transmits it to the edge computing module. The edge computing module can be an edge computing core board, which processes the video data, identifies target information, and transmits it to the data scheduling and management module. The data scheduling and management module dynamically adjusts resource strategies for each edge computing module, allocating anchor frames for calculating the target information to the appropriate edge computing module based on the size of the target information. Finally, the calculated target information is uploaded to the big data center collection module, which filters the calculated target information and transmits it to the terminal device module for viewing.

[0012] A video analysis method based on edge computing includes multiple video analysis devices, a data scheduling and management module, a big data center collection module, and a terminal device module. Each video analysis device internally integrates a video information acquisition module, an interface adaptive matching module, and an edge computing module connected sequentially. The video analysis method steps of the video analysis system are as follows:

[0013] (1) Upon initial power-on, the interface adaptive matching module initializes the device. This means the module automatically identifies the type of connected device in the video acquisition module—whether it's a camera, IP speaker, or other RJ45 interface device—and assigns it an IP address. All interfaces use RJ45, allowing for easy switching and modification of the device's interface and number. The interface adaptive matching module is positioned between the video acquisition module and the edge computing module, adapting and converting the communication between them in real time.

[0014] (2) After initialization, the video information acquisition module acquires on-site video in real time and performs RTSP streaming on the video images, forwarding them to the edge computing module as video streams;

[0015] (3) The edge computing module is a core edge computing board capable of running deep learning algorithms and communicating via the network. After receiving video information, the edge computing module automatically performs frame-by-frame processing on the video stream (2 frames / s), uses deep learning algorithms to perform light convolution calculations on the captured frames, and determines whether a target exists in the current video scene. If not, the current convolution is stopped, and the module enters a micro-standby state to wait for the next calculation. If a target exists, deep convolution continues to locate the target position, further calculates the target category, and transmits the result to the data scheduling and management module. Light convolution involves performing convolution operations on two frames per second of the video stream, while selecting a convolution kernel half the size of the image during the convolution process (a larger convolution kernel reduces computational pressure but lowers accuracy), thus achieving lightweight convolution. Deep convolution involves real-time convolution of all 25 frames of the video stream, with the convolution kernel size being one-quarter the size of the image. This increases the accuracy of target detection.

[0016] Furthermore, the deep learning algorithms include SSD algorithm, YOLO series algorithms, R-CNN series algorithms, SSP algorithm, and other algorithms based on Tensorflow or PyTorch.

[0017] (4) The data scheduling and management module dynamically adjusts the resource policies for each edge computing module. It calculates the comprehensive pressure value P based on the current comprehensive utilization rates of the GPU and CPU in the edge computing module, and conducts weight and resource scheduling and allocation based on the value of P. It automatically selects the computing model weights S and L and combines with the resources of other devices for computing. That is, it selects the edge computing module running the L weight or the S weight to calculate the video stream according to the value of P. Among them, the L weight is larger than the S weight, the speed of the L weight is less than that of the S weight, and the computing accuracy of the L weight is higher than that of the S weight.

[0018] If the computing pressure P of the current edge computing module on the video stream: P < p1, then the precise mode is adopted, and the edge computing module running the L weight is used.

[0019] If the computing pressure P of the current edge computing module on the video stream: p1 < P < p2, then the fast recognition mode is adopted, and the edge computing module running the S weight is used.

[0020] If a large number of targets to be detected suddenly merge into a single edge computing module scenario, that is, P > p2, to prevent downtime, the video information acquisition module synchronously forwards the video stream at this point to the data scheduling and management module. The data scheduling and management module forwards this video stream to the other edge computing modules running the L weight, and changes the weight of the edge computing module running the L weight to S. After the calculation is completed, the calculation result is returned and forwarded to the original edge computing module through the data scheduling and management module. After the original edge computing module integrates the results, it completes the subsequent operations, and finally uploads the relevant target information to the big data center collection module for subsequent viewing and management.

[0021] For example, in the scenario of edge computing module A, a large number of targets to be detected suddenly merge (P > p2). To prevent machine downtime, the video stream at point A is synchronously forwarded to the data scheduling and management module. The data scheduling module forwards the video stream to the other computing cores running the L weight (B cluster), and changes the core weight of the B cluster to S, saving more resources for sharing the calculation of A. After the calculation is completed, the B cluster returns and forwards the calculation result to A through the data scheduling and management module. After A integrates the results, it completes the subsequent operations. Finally, the data exchange module uploads the relevant target information to the big data processing module for subsequent viewing and management.

[0022] Among them, p1 and p2 are set thresholds, and p1 < p2.

[0023] (5) The big data center collection module retains all received information. Based on user needs, it filters the information and transmits the information that the user is interested in to the terminal device module for the user to view. At the same time, it can work with the self-developed management system to view the current on-site identification screen, on-site intercom, historical violation information verification, traffic statistics, automatic data export and image download in real time, and transmit the results to the terminal device module.

[0024] Furthermore, the video information acquisition module includes a camera and an IP speaker for on-site playback, which are connected to each other through an interface adaptive matching module to achieve full-duplex communication between uplink and downlink; wherein, the camera uses the RTSP protocol for streaming, and the video uses the H.265 video encoding format.

[0025] Furthermore, the interface adaptive matching module includes four RJ45 interfaces connected to the edge computing module. The interface adaptive matching module will use readable medium information to send calibration information to the video information acquisition module connected to the interface via the UDP communication protocol, and determine the device type and assign an IP address based on the device's response information.

[0026] Furthermore, the readable medium information is stored in the ROM of the edge computing module. The readable medium information stores identification files and related computer instructions, which are transparent to the user and cannot be changed. Only the system is allowed to read it automatically.

[0027] Furthermore, the edge computing module is equipped with an ARM architecture 64-bit CPU, an Nvidia GPU for CUDA acceleration, an onboard ROM with 4GB of memory, and a 64GB storage card for running deep learning algorithms.

[0028] Furthermore, in step (3), the micro standby state is not a standby state in the traditional sense, but rather maintains the lowest power and minimum CPU and GPU resources. Once a target is detected in the scene, it can resume full power operation within 0.5s, thus getting rid of the traditional full-time full-power operation strategy.

[0029] Furthermore, in step (4), the method for calculating the comprehensive pressure value P is as follows:

[0030] P = Usage cpu *50%+Usage gpu *50%, Usage cpu For CPU utilization, Usage gpu GPU utilization;

[0031] Usage cpu =(RunTime) cpu / TotalTime cpu)*(Use_imf cpu / Cap cpu RunTime cpu TotalTime is the CPU execution time. cpu Use_imf represents the total CPU time. cpu For CPU usage information, Cap cpu This represents the CPU's maximum processing power, where:

[0032] RunTime cpu =T now -T last

[0033] TotalTime cpu =(T now -T last )*Num core

[0034] Use_imf cpu =∑ i∈core ∑ f∈Freq f*RunTime i *Weight i

[0035] Cap cpu =∑ i∈core Mas_F(i)*TotalTime cpu *Weight i ;

[0036] T now At the current sampling time, T last The most recent sampling time, Num core Let i be the number of available cores, i be the core index, core be the set of all cores, f be the frequency of the i-th frequency point, and RunTime be the frequency of the i-th frequency point. i Weight is the runtime of the i-th frequency point. i Let be the core weight of i, and Mas_F(i) be the maximum frequency of the core of i;

[0037] Usage gpu =(RunTime) gpu / TotalTime gpu )*(Use_imf gpu / Cap gpu RunTime gpu TotalTime represents the GPU execution time. gpu Use_imf represents the total GPU time. gpu For GPU usage information, Cap gpuThis represents the GPU's maximum processing power, where:

[0038] RunTime gpu =T now ′-T last ′

[0039] TotalTime gpu =(T now ′-T last ′)*Num core ′

[0040] Use_imf gpu =∑ j∈core ∑ f′∈Freq f*RunTime j *Weight j

[0041] Cap gpu =∑ j∈core Mas_F(j)*TotalTime gpu *Weight j ;

[0042] T now ′ represents the current sampling time, T last ′ represents the most recent sampling time, Num core f' represents the number of available cores, j is the core index, core is the set of all cores, f' is the frequency of the j-th frequency point, and RunTime is... j The runtime of the j-th frequency point, Weight j Let J be the core weight of j, and Mas_F(j) be the maximum frequency of the core of j.

[0043] Furthermore, the big data center collection module is a high-performance server cluster that saves the data before and after processing and transmits the processed data in encrypted form.

[0044] Furthermore, the terminal device module receives data information via HTTP, TCP, FTP, or RTSP protocols and displays it on various terminal devices, which can be implemented in various forms. For example, the various terminals mentioned in this invention may include mobile phones, car phones, laptops, broadcast receivers, PADs (tablet computers), and computers, etc.

[0045] Furthermore, the deep learning algorithm is an optimized version that uses dynamic anchor box technology to adaptively adjust the target anchor box size. The optimized deep learning algorithm ported the traditional x86 CPU-based algorithm to a 64-bit ARM architecture CPU through lightweight adjustments. Traditional deep learning training anchor box generation is top-down and fixed, unable to be dynamically adjusted, easily leading to prior information mismatch problems. Using dynamic anchor box technology to adaptively adjust the target anchor box size effectively solves the slow classification convergence problem caused by traditional fixed anchor boxes. For any non-terminal feature map point, the data related to anchor point generation comes from both the upper and lower layers simultaneously. By receiving information and calculating the velocity loss value each time in a timely manner, it can adaptively adjust the anchor box best suited for the corresponding target based on the corresponding loss value, thus increasing the target detection convergence speed and improving accuracy, especially for targets of varying sizes within the same image frame.

[0046] The speed loss value of the optimized deep learning algorithm The calculation is as follows:

[0047]

[0048] Where K represents the number of target categories actually output in the current frame image, m is its index number, m ranges from 1 to K, N represents the number of target value categories labeled in the training set, n represents the ground truth bounding box index of the manually annotated single frame image, n ranges from 1 to N, and area mn This represents the difference in anchor box area between the corresponding detection m and the manually labeled n. F is the adjustment function. Flag indicates the comparison between the predicted box and the ground truth box size. When Flag is -1, the anchor box is shrunk, and when Flag is 1, the anchor box is enlarged. IoU represents the intersection-union ratio: the ratio of the intersection to the union of the areas of the ground truth box and the predicted box.

[0049] Furthermore, each side of the outer casing of the video analysis device is connected to a tray, on which an IP speaker and a camera are mounted. The video analysis device also includes a display screen and a cooling fan.

[0050] The device includes a dustproof and heat-dissipating metal cover; a display screen for showing network signals, device connection status, and the latest output information, preferably a 4.5-inch display screen; a cooling fan for heat dissipation, preferably a 4W cooling fan; openings on both sides of the video analysis device, each opening covered by a dustproof and heat-dissipating metal cover, with vent holes on the surface of the cover, each vent hole having a diameter of 0.1mm and a 1mm interval between every two openings, the inside of the vent holes being coated with a hydrophobic material to improve their hydrophobicity; simultaneously, a waterproof and breathable membrane is embedded in the middle of the dustproof and heat-dissipating metal cover, which can ensure heat dissipation while also providing dustproof effect.

[0051] Furthermore, the dustproof and heat dissipation metal cover plate consists of an upper waterproof plate and a lower waterproof plate that cover the openings on both sides of the video analysis device. The upper waterproof plate is a bent plate with ventilation holes on its surface and an embedded waterproof and breathable membrane. The lower waterproof plate is solid, with one end connected to the outer shell of the video analysis device and connected to the upper waterproof plate to cover the openings on both sides of the video analysis device.

[0052] 3. Beneficial effects

[0053] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0054] The video analytics system and method based on edge computing of this invention can reduce the cost of intelligent transformation for enterprises, save transformation time, and at the same time have low latency and high hardware resource utilization. Attached Figure Description

[0055] Figure 1 This is a schematic diagram of the overall system flow of the intelligent video analysis method of the present invention;

[0056] Figure 2 This is a schematic diagram of the network structure of the intelligent video analysis method of the present invention;

[0057] Figure 3 This is a flowchart illustrating the resource allocation process in the intelligent video analysis method of the present invention, which uses a data exchange module to dynamically schedule resources.

[0058] Figure 4 This is a schematic diagram illustrating the optimization process of the dynamic anchor box of the optimized deep learning algorithm in the intelligent video analysis method of the present invention.

[0059] Figure 5 This is a front view of the intelligent video analysis device of the present invention;

[0060] Figure 6 The system of this invention automatically generates a spliced ​​training set based on an existing dataset to enrich the dataset;

[0061] Figure 7 The diagram shows the detection results of the optimized deep learning algorithm under various conditions according to the present invention;

[0062] Figure 8 This is a schematic diagram of the detection effect of the deep learning algorithm before optimization in this invention;

[0063] Figure 9 This is a schematic diagram comparing the detection performance of the deep learning algorithm before and after optimization.

[0064] In the picture:

[0065] 1. IP speaker; 2. Camera; 3. Support plate; 4. Display screen; 5. Housing; 6. Upper waterproof plate; 7. Waterproof and breathable membrane; 8. Ventilation holes; 9. Lower waterproof plate. Detailed Implementation

[0066] To make the objectives and technical solutions of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the described embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] like Figure 1 and Figure 2 As shown, a video analytics system based on edge computing includes multiple video analytics devices, a data scheduling and management module, a big data center collection module, and a terminal device module. Each video analytics device integrates a video information acquisition module, an interface adaptive matching module, and an edge computing module connected sequentially. The interface adaptive matching module can be an RJ45 interface adapter, capable of automatically identifying the device connected to the corresponding interface, i.e., automatically identifying the video information acquisition module's device interface and assigning an IP address. The video information acquisition module acquires video data and transmits it to the edge computing module. The edge computing module can be an edge computing core board, which processes the video data, identifies target information, and transmits it to the data scheduling and management module. The data scheduling and management module dynamically adjusts resource strategies for each edge computing module, allocating anchor frames for calculating target information to the appropriate edge computing module based on the size of the target information. Finally, the calculated target information is uploaded to the big data center collection module, which filters the calculated target information and transmits it to the terminal device module for viewing.

[0068] Example 1

[0069] This invention provides an intelligent video analysis method based on edge devices. For ease of understanding, this embodiment will explain the system operation process from a system perspective.

[0070] Upon initial power-on, the interface adaptive matching module initializes the device by sending UDP commands to each IP speaker / camera. Each recipient replies with its IP address, and the module then sends device verification commands to each IP address to determine the device type. After initialization, the cameras receive video feeds in real-time and synchronously forward them to the corresponding edge computing modules via the RTSP protocol. The edge computing modules calculate a comprehensive stress value P to determine whether to perform the calculation themselves or synchronize the task to other edge computing modules. When calculating the comprehensive stress value P, the formula RunTime is first used... cpu =T now -T lastCalculate the execution time (RunTime) of this computation task. cpu Then based on TotalTime cpu =(Tal now -Tal last )*Num core Calculate the total CPU execution time (TotalTime). cpu The two are then discussed to arrive at the proportion; secondly, according to the formula Use_imf cpu =∑ i∈core ∑ f∈Freq f*RunTime i *Weight i The CPU usage status information Use_imf is calculated by multiplying the computation time, corresponding weight, and frequency of each CPU core, and then summing the products twice. cpu Then, according to the formula Use_imf cpu =∑ i∈core ∑ f∈Freq f*RunTime i *Weight i Input the CPU's factory rated information to calculate the CPU's theoretical maximum processing capacity (Cap). cpu Finally, after calculating the individual values ​​for each item, according to the formula:

[0071] Usage cpu =(RunTime) cpu / TotalTime cpu )*(Use_imf cpu / Cap cpu ),

[0072] The total CPU utilization was calculated.

[0073] The calculation method for GPU single-task utilization is the same as that for CPU, and you can refer to the CPU calculation method to calculate it.

[0074] Finally, according to the formula P = Usage cpu *50%+Usage gpu *50% Calculate the overall pressure value P.

[0075] According to the policy settings, the weight L should be 40% when a single device performs a single task, and the weight S should be 20% under the same conditions. Therefore, an edge device can execute three tasks simultaneously: two L-weighted tasks and one S-weighted task.

[0076] This method uses three different resource scheduling schemes.

[0077] include:

[0078] Strategy 1: The calculated weight of the edge computing device is switched to the L weight. Without considering the situation of other edge devices, accuracy is prioritized and processing speed is secondary. The setting of the L weight should be such that when the device is exclusive, the P occupancy rate is 40%.

[0079] Strategy 2: The calculated weight of the edge computing device is switched to the S weight. Without considering the situation of other edge devices, accuracy is prioritized and processing speed is secondary. The setting of the S weight should be such that when the device is exclusive, the P occupancy rate is 20%.

[0080] Strategy 3: This device transfers the computing task to other devices to share the computing pressure. The other devices should be in the weight L state at this time, indicating that their computing pressure is relatively small and they can undertake the task. After sharing, the assigned device will select the S weight to save its own resources for the computing of other tasks.

[0081] Furthermore, the dynamic allocation of the resources switches the strategy according to the value of the comprehensive pressure value P. The specific strategy switching process is as Figure 3 shown. The usage rates of the CPU and GPU are affected by the size and quantity of the targets in the current scenario. According to the formula P = Usage cpu *50% + Usage gpu *50%. After comprehensively calculating the pressure value P, the value of P is obtained. By setting two thresholds p1 and p2 of the third order, the range of P is determined. If P < p1, Strategy 1 is used; if p1 < P < p2, it is switched to Strategy 2; otherwise, when P > p2, the device is switched to Strategy 3 to share the task pressure through other devices.

[0082] According to the setting of the relationship between the above P and the weight, p1 is set to 0.4(Weight L ), and p2 should be 0.6(Weight L + Weight S ). Therefore:

[0083] (1) When the comprehensive pressure value P calculated by the current edge device is < 0.4, Strategy 1 is used.

[0084] (2) When the comprehensive pressure value calculated by the current edge device is 0.4 < P < 0.6, Strategy 2 is used.

[0085] (3) When the comprehensive pressure value calculated by the current edge device is 0.6 < P, Strategy 3 is used.

[0086] It should be noted that Strategy 1 includes a special case: when the video frame corresponding to the current edge computing module is empty of targets, the device is in a light convolution state. In this state, the device processes two frames per second, using a larger convolution kernel (half the size of the frame) to further reduce computational load. Furthermore, after detecting a target, the device in light convolution can react within 0.5 seconds, quickly reverting to deep convolution (detecting all 25 frames and using a quarter-sized kernel) to improve detection accuracy. Once the computing module returns to deep convolution, it selects Strategy 1, 2, or 3 based on the P-value.

[0087] If the device ultimately executes strategies one or two, it can complete the corresponding task using its own edge computing devices. If strategy three is executed, resources need to be scheduled through the data scheduling management module. First, the source IP address of the forwarded RTSP video stream and the source edge device IP address are recorded. After recording the information, the stream is packaged. After packaging, the number of remaining currently available edge devices, n, is calculated. After the calculation, the source RTSP stream task is divided into n segments every 0.5 seconds (for example, if n=5, then device 1 executes the time segment 0-0.1 seconds, device 2 executes the time segment 0.1-0.2 seconds, and so on). Finally, the processing time segments are recorded and calculated separately. When performing single-point or segmented calculations, the algorithm will use dynamic adaptive anchor frame technology to improve the detection rate. The dynamic adaptive anchor frame technology is as follows: Figure 4 As shown, the initial anchor boxes for the input image are set to the mean of the previously learned anchor boxes, using the formula... Calculate the velocity loss value for each target box. The size of the anchor frame is adjusted based on the velocity loss value; if the loss value is small... No adjustment is needed if the anchor frame is not large enough; otherwise, the anchor frame size needs to be adjusted. The direction of the anchor frame adjustment (expanding or shrinking) depends on the IoU. If IoU < 0.5 and Flag = 1, it means the overlap area is small, and the prediction frame needs to be expanded. Otherwise, similarly, the prediction frame needs to be shrunk, with a decay of 0.00002. The direction adjustment function... By continuously calculating the loss rate and adjusting the anchor frame size, and through repeated iterations, it was finally adjusted to... This means that the anchor frame size is appropriate, which can reduce the convergence speed of the final target recognition. For example... Figure 6 To improve the recognition accuracy of the algorithm, the algorithm selects images from the original dataset one at a time, and selects nine images from these images each time, performs lighting changes, scaling and cropping on each image to generate a new image for training, increasing the complexity of the training set and improving the detection accuracy.

[0088] The optimized detection algorithm, employing segmented calculation and dynamic anchor frame technology, identifies the scene and transmits the results to the data scheduling and management module. This module integrates the identified segments based on multiple data packets containing information such as processing time, processing point IP address, and processing result. It then sends processing completion information back to the source device and transmits the processing result for that point to the big data platform for storage. The self-developed management system allows users to view historical data, including but not limited to daily violation statistics, browsing and downloading violation screenshots, real-time on-site warnings, and violation message pushes, meeting user needs for review and statistics. A comparison of the algorithm before and after optimization is shown below. Figure 8 , 9 As shown in the figure, the optimized algorithm demonstrates a significant improvement in detection accuracy and generalization performance. Further testing of the optimized algorithm's detection performance in various scenarios is as follows: Figure 7 As shown, (a) shows the detection of violators at night; (b) shows the detection under rainy weather when people are using umbrellas to cover their heads; (c) shows that the detected person is wearing a hat without protective function, but the system still detects them correctly; (d) shows the detection of people blocking each other's view during rush hour, and all detections are correct; (e) shows the detection of people riding electric bikes into the work area without permission; (f) shows that the worker is riding a work tricycle but is not wearing a safety helmet correctly; (g) shows that the safety helmet is not detected when no one is wearing it, because in the actual scenario, no scenario of no one wearing a safety helmet was set up and no training was performed, so the algorithm did not have any false recognition phenomena; (h) shows that the violator intentionally covered his head to try to interfere with the detection results, but the system still detected that he was not wearing a safety helmet, indicating that the algorithm has good generalization; (i) shows the detection results of people crossing the detection area at night when they are far away.

[0089] Example 2:

[0090] This embodiment provides an intelligent video analysis device based on edge computing, such as... Figure 5As shown, the video analysis device has two support plates 3 connected to its outer casing 5 on each side. IP speaker columns 1 and cameras 2 are mounted on the support plates 3. The video analysis device also includes a display screen 4, a cooling fan, and a dustproof and heat-dissipating metal cover. The display screen 4 is used to display network signals, device connection status, and the latest output information; a 4.5-inch display screen is preferred. The cooling fan is used for heat dissipation; a 4W cooling fan is preferred. The video analysis device has openings on both sides, covered by dustproof and heat-dissipating metal covers. Ventilation holes 8 are formed on the surface of the dustproof and heat-dissipating metal covers. The diameter of the ventilation holes 8 can be 0.1mm, with a 1mm interval between every two openings. The inside of the ventilation holes 8 is coated with a hydrophobic material to improve their hydrophobicity. Simultaneously, a waterproof and breathable membrane 7 is embedded in the middle of the dustproof and heat-dissipating metal cover; this membrane ensures both heat dissipation and dust prevention. The internal edge computing module receives data from... Figure 5 After the RTSP video stream transmitted by the camera 2 shown in the figure is analyzed using a deep learning algorithm, the latest analysis results are displayed on the display screen 4.

[0091] To prevent dust and rainwater from affecting the machine's performance, the dustproof and heat-dissipating metal cover is designed with dustproof and waterproof features, achieving both heat dissipation and dust and waterproof protection. The dustproof and heat-dissipating metal cover consists of an upper waterproof plate 6 and a lower waterproof plate 9, covering the openings on both sides of the video analysis device. The upper waterproof plate 6 is a bent plate with ventilation holes 8 on its surface and an embedded waterproof and breathable membrane 7 to prevent dust from entering the device and affecting its performance, while not hindering the internal fan and natural airflow for heat dissipation. The lower waterproof plate 9 is solid, primarily serving a ventilation and heat dissipation function, ensuring an IP56 dustproof and waterproof rating. One end of the lower waterproof plate 9 connects to the outer shell 5 of the video analysis device and is connected to the upper waterproof plate 6 to cover the openings on both sides of the video analysis device.

[0092] The present invention has been described in detail above with reference to specific exemplary embodiments. However, it should be understood that various modifications and variations can be made without departing from the scope of the invention as defined by the appended claims. The detailed description and drawings should be considered illustrative only and not restrictive, and any such modifications and variations shall fall within the scope of the invention described herein. Furthermore, the background art is intended to illustrate the current state of development and significance of the technology and is not intended to limit the present invention or the scope of application of the present application.

Claims

1. An edge computing-based video analysis method, characterized in that: The video analysis system comprises a plurality of video analysis devices, a data scheduling management module, a big data center collection module and a terminal equipment module, and the video analysis device is internally integrated with a video information acquisition module, an interface adaptive matching module and an edge computing module connected in sequence; wherein the interface adaptive matching module automatically identifies the equipment interface of the video information acquisition module and assigns an IP address, the video information acquisition module acquires video data and transmits it to the edge computing module, the edge computing module calculates and processes the video data, identifies target information and transmits it to the data scheduling management module, the data scheduling management module dynamically adjusts the resource strategy of each edge computing module, assigns the corresponding edge computing module to calculate the anchor frame of the target information according to the size of the target information, and finally uploads the calculated target information to the big data center collection module; the big data center collection module filters the calculated target information and transmits it to the terminal equipment module for reference; The video analysis method comprises the following steps: (1) The interface adaptive matching module performs device initialization, identifies whether the connected interface is a camera, an IP sound column or other RJ45 interface equipment, and assigns an IP address to it; (2) After initialization, the video information acquisition module acquires live video in real time and forwards it to the edge computing module in the form of a video stream; (3) The edge computing module processes the video stream by taking frames, uses a deep learning algorithm to perform light convolution calculation on the taken frame images, judges whether there is a target in the current video scene, if not, stops this convolution and enters a micro standby state to wait for the next calculation; if yes, continues to perform deep convolution, locates the target position, further calculates the target category and transmits the result to the data scheduling management module; wherein: The deep learning algorithm is an optimized deep learning algorithm, and a dynamic anchor frame technology is used to adaptively determine a target anchor frame size; the optimized deep learning algorithm has a speed loss value The calculation is: = F ( ) F ( ) = Flag = Wherein, K represents the actual output target class number in the frame image, m is its subscript number, m takes value from 1 to K, N represents the target value kind number identified in the training set, n represents the real frame image of artificial labeling real frame subscript, n takes value from 1 to N, It represents the anchor frame area difference corresponding to the detection m and artificial labeling n, if the speed loss value Then it is not necessary to adjust, otherwise it needs to adjust the anchor frame size, F is the adjustment function, Flag represents the predicted frame in the real frame size contrast situation, Flag is-1, the anchor frame is reduced, Flag is 1, the anchor frame is enlarged, IoU represents the intersection ratio: the ratio of the intersection and the union of the real frame and the predicted frame, e is the base of natural logarithm; (4) The data scheduling management module dynamically adjusts the resource strategy of each edge computing module, calculates a comprehensive stress value P according to the comprehensive use rate of the current GPU and CPU of the edge computing module, selects an edge computing module running L weight or S weight according to the P value to calculate the video stream, wherein the L weight is greater than the S weight, the speed of the L weight is less than that of the S weight, and the calculation accuracy of the L weight is higher than that of the S weight; If P If p1 If P > p2, to prevent downtime, the video information acquisition module synchronously forwards the video stream at this point to the data scheduling management module, the data scheduling management module forwards the video stream to the remaining edge computing modules running L weight, and converts the weight of the edge computing module running L weight to S; after the calculation is completed, the calculation result is returned to the original edge computing module through the data scheduling management module, the original edge computing module integrates the result and then performs subsequent operations, and finally uploads the relevant target information to the big data center collection module for subsequent viewing and management; Wherein, p1 and p2 are set threshold values, and p1 < p2. (5) The big data center collection module adopts a reserved attitude towards all received information, and after screening the information, transmits the information of interest to the user to the terminal device module for the user to consult, and at the same time, cooperates with the self-developed management system to realize the functions of real-time viewing of the current on-site identification picture, on-site intercom, historical violation information checking, flow statistics, data automatic export and image download.

2. The edge computing based video analysis method of claim 1, wherein: The video information collection module includes a camera and an IP sound column for on-site playing, and is connected to each other through an interface adaptive matching module, and full-duplex communication can be realized; wherein the camera adopts an RTSP protocol push stream mode, and the video adopts an H.265 video coding format. 3.The edge computing based video analysis method of claim 1, wherein: The interface adaptive matching module includes four RJ45 interfaces, and is connected to the edge computing module; the interface adaptive matching module sends proofreading information to the device of the video information collection module connected to the interface through a UDP communication protocol using readable medium information, and determines the type of the device and allocates an IP according to the reply information of the device.

4. The edge computing based video analytics method of claim 1, wherein: The edge computing module is equipped with an ARM architecture 64-bit CPU, an Nvidia GPU for CUDA acceleration, a built-in ROM, 4G memory and a 64G storage card, and is used for running a deep learning algorithm.

5. The edge computing-based video analysis method of claim 1, wherein: In step (3), the micro standby state is to keep the minimum power and the minimum CPU and GPU resources running.

6. The edge computing-based video analysis method of claim 1, wherein: In step (4), the method for calculating the comprehensive pressure value P is: P = + 50%, CPU usage, GPU usage; ( / ) ( ), is the CPU execution duration, is the CPU total duration, is the CPU usage information, is the CPU maximum processing capacity, wherein: = - =( - ) = = ； is the current sampling time, is the last sampling time, is the number of available cores, i is the core sequence number, core is the set of all cores, and f is the frequency of the i-th frequency point, is the running time of the i-th frequency point, is the core weight of i, is the maximum frequency of the i-th core; ( / ) ( ), is the GPU execution duration, is the GPU total duration, is the GPU usage information, is the GPU maximum processing power, wherein: = - =( - ) = = ； is the current sampling time, is the last sampling time, is the number of available cores, j is the core sequence number, core is the set of all cores, f is the frequency of the jth frequency point, is the running time of the jth frequency point, is the core weight of j, is the maximum frequency of j core.

7. The edge computing-based video analysis method of claim 1, wherein: The big data center collection module is a high-performance server cluster, which saves the data before and after processing, and transmits the processed data in an encrypted manner.

8. The edge computing-based video analysis method of claim 1, wherein: The terminal device module receives data information through HTTP, TCP, FTP or RTSP protocols, and displays the data information on various terminal devices, including but not limited to mobile phones, tablet computers, PDAs and computers.

Images