Video monitoring system, method, and storage medium

Abstract

The present disclosure provides a video monitoring system, method and storage medium, wherein the system comprises: a device-side processing module, configured to perform target detection processing on a video collected by a monitoring device, determine a detected target region and obtain a corresponding target subgraph; calculate intersection over union (IOU) information of the detected target region determined at different times, and determine a target trajectory based on the IOU information; obtain channel detection information of a wireless communication channel; a decision module, configured to determine whether to transmit information to an edge-computing-side processing module based on the channel detection information and the IOU information, and send the target subgraph subjected to compression processing to the edge-computing-side processing module; and the edge-computing-side processing module, configured to obtain target trajectory information according to the target subgraph. The present disclosure can realize self-adaptation of a communication environment and autonomous allocation of a computing task, effectively save resources, reduce costs, improve the precision of intelligent monitoring, and improve the user experience.

Description

Technical Field

[0001] This invention relates to the field of video surveillance technology, and in particular to a video surveillance system, method, and storage medium. Background Technology

[0002] In modern industrial production and daily life, video surveillance systems are playing an increasingly important role. To obtain timely and effective monitoring information, not only are video image algorithms required to be more accurate and reliable, but higher demands are also placed on communication resources and computing power. The development of 5G and MEC (Multi-Access Edge Computing) has improved the maturity, stability, and reliability of video surveillance to some extent. However, existing video surveillance systems suffer from the following problems: 1. In complex scenarios and high-definition video tasks, placing all computational processing on the monitoring equipment's edge devices requires substantial computing resources and incurs high costs; 2. Placing computation on edge computing servers can improve computing power and the accuracy of recognition algorithms to some extent, but it requires video image transmission, consuming significant bandwidth and increasing bandwidth costs; 3. Unstable communication environments can affect the overall system. Summary of the Invention

[0003] In view of this, one technical problem to be solved by the present invention is to provide a video surveillance system, method and storage medium.

[0004] According to a first aspect of this disclosure, a video surveillance system is provided, comprising: a device-side processing module, configured to perform target detection processing on video collected by a monitoring device, determine a target area and obtain a corresponding target sub-image; calculate the intersection-union ratio (IOU) information of the target areas determined at different times, and determine the trajectory of the target based on the IOU information; obtain channel detection information of a wireless communication channel; a decision module, configured to determine, based on the channel detection information and the IOU information, whether to transmit information to an edge computing-side processing module; if it is determined that information should be transmitted to the edge computing-side processing module, send the compressed target sub-image to the edge computing-side processing module; the edge computing-side processing module is configured to obtain target trajectory information based on the target sub-image.

[0005] Optionally, the device-side processing module includes: a target detection unit, used to perform target detection processing on the video using a trained target detection model, determine the target region, and obtain the target sub-image; wherein the parameters of the target detection model are integer parameters; the target detection model includes: a Ghost module.

[0006] Optionally, the target detection unit is used to obtain detection information corresponding to the video using the target detection model. in, Information for the i-th target being detected. This indicates the coordinates of the top-left point of the detection box. Indicates the width of the detection box. Indicates the height of the detection frame. The detection category is indicated by t, which represents the t-th frame of the video; the detection target region is determined based on the detection information.

[0007] Optionally, the target detection unit is used to obtain the corresponding target sub-image based on the detection information.

[0008] Optionally, the device-side processing module includes: a target tracking unit, used to determine...

[0009]

[0010] in, To be with at t The corresponding detection target area For the time at t-1 The corresponding target area for detection;

[0011] The target tracking unit is also used to determine associated targets based on an IOU threshold, set the same ID for associated targets, and determine the trajectory of associated targets based on the ID.

[0012] Optionally, the channel detection information includes: wireless channel gain; the device-side processing module includes: a channel detection unit, used to acquire the wireless channel gain and perform discretization processing.

[0013] Optionally, the decision module includes: a decision calculation unit, used to determine whether to transmit information to the edge computing side processing module based on the deep Q network and according to the wireless channel gain and the IOU information;

[0014] The update equation for the deep Q-network is: Let n(S,A) represent the learning rate, n(S,A) represent the action chosen in state S, Q(S,A) represent the cumulative average reward when taking action A in state S, and Reward be the reward function; the state space is State = (h, IOU) i h represents the wireless signal gain, and IOU is the signal gain. iThe maximum IOU corresponding to the i-th target is represented; the action space Action = (N, Z), where N represents whether information is transmitted to the edge computing side processing module, and Z represents the compression level of the target subgraph; the reward function... Y represents whether tracking was successful, and T represents the total latency.

[0015] Optionally, the edge computing side processing module includes: a feature extraction module, used to process the target sub-image using a trained feature extraction model to obtain the feature information of the target; a trajectory prediction module, used to process the target sub-image using a trained first trajectory detection model to obtain the first trajectory of the target; to predict the historical predicted trajectory of the target using a trained second trajectory detection model to obtain the second trajectory of the target; and to determine the trajectory distance between the first trajectory and the second trajectory; and a matching module, used to calculate the cosine similarity between the feature information and the previous feature information as the feature distance; to calculate the matching degree as n = λD1 + (1-λ)D2, where D1 is the trajectory distance, D2 is the feature distance, and λ is a coefficient; to determine associated targets based on the matching degree, to set the same ID identifier for associated targets, and to determine the trajectory of associated targets based on the ID identifier.

[0016] According to a second aspect of this disclosure, a video surveillance method is provided, comprising: a device-side processing module performing target detection processing on video collected by a monitoring device to determine a target area and obtain a corresponding target sub-image; the device-side processing module calculating the intersection-union ratio (IOU) information of the target areas determined at different times, and determining the trajectory of the target based on the IOU information; the device-side processing module obtaining channel detection information of a wireless communication channel; a decision module determining whether to transmit information to an edge computing side processing module based on the channel detection information and the IOU information; if the decision module determines to transmit information to the edge computing side processing module, sending the compressed target sub-image to the edge computing side processing module; and the edge computing side processing module obtaining target trajectory information based on the target sub-image.

[0017] Optionally, the device-side processing module performs target detection processing on the video collected by the monitoring device, determines the target area, and obtains the corresponding target sub-image, including: the device-side processing module uses a trained target detection model to perform target detection processing on the video, determines the target area, and obtains the target sub-image; wherein, the parameters of the target detection model are integer parameters; the target detection model includes a Ghost module.

[0018] Optionally, determining the detection target region includes: the device-side processing module using the target detection model to obtain detection information corresponding to the video. in, Information for the i-th target being detected. This indicates the coordinates of the top-left point of the detection box. Indicates the width of the detection box. Indicates the height of the detection frame. The detection category is indicated by t, which represents the t-th frame of the video; the detection target region is determined based on the detection information.

[0019] Optionally, obtaining the target sub-image includes: the device-side processing module obtaining the corresponding target sub-image based on the detection information.

[0020] Optionally, the device-side processing module calculates the Intersection over Union (IOU) information of the detection target area determined at different times, and determines the target trajectory based on the IOU information, including: the device-side processing module determines...

[0021]

[0022] in, To be with at t The corresponding detection target area For the time at t-1 The corresponding target area for detection;

[0023] The device-side processing module determines associated targets based on the IOU threshold, sets the same ID for associated targets, and determines the trajectory of associated targets based on the ID.

[0024] Optionally, the channel detection information includes: wireless channel gain; the device-side processing module obtains the channel detection information of the wireless communication channel by: the device-side processing module obtains the wireless channel gain and performs discretization processing.

[0025] Optionally, the decision module determines whether to transmit information to the edge computing side processing module based on the channel detection information and the IOU information, including: the decision module determines whether to transmit information to the edge computing side processing module based on the deep Q network and according to the wireless channel gain and the IOU information;

[0026] The update equation for the deep Q-network is: Let n(S,A) represent the learning rate, n(S,A) represent the action chosen in state S, Q(S,A) represent the cumulative average reward when taking action A in state S, and Reward be the reward function; the state space is State = (h, IOU)i h represents the wireless signal gain, and IOU is the signal gain. i The maximum IOU corresponding to the i-th target is represented; the action space Action = (N, Z), where N represents whether information is transmitted to the edge computing side processing module, and Z represents the compression level of the target subgraph; the reward function... Y represents whether tracking was successful, and T represents the total latency.

[0027] Optionally, the edge computing side processing module obtains target trajectory information based on the target sub-image by: processing the target sub-image using a trained feature extraction model to obtain target feature information; processing the target sub-image using a trained first trajectory detection model to obtain the target's first trajectory; predicting the target's historical predicted trajectory using a trained second trajectory detection model to obtain the target's second trajectory; determining the trajectory distance between the first trajectory and the second trajectory; calculating the cosine similarity between the feature information and the previous feature information as the feature distance; calculating the matching degree as n = λD1 + (1-λ)D2; where D1 is the trajectory distance, D2 is the feature distance, and λ is a coefficient; determining associated targets based on the matching degree, setting the same ID for associated targets, and determining the trajectory of associated targets based on the ID.

[0028] According to a third aspect of this disclosure, a computer-readable storage medium is provided that non-transitoryly stores computer instructions which are executed by a processor using the method described above.

[0029] The video surveillance system, method, and storage medium disclosed herein can achieve adaptive communication environment and autonomous allocation of computing tasks, effectively saving bandwidth resources and computing resources, reducing costs, improving the accuracy of intelligent monitoring, and enhancing the user experience. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a schematic diagram of a module of a video surveillance system according to an embodiment of the present disclosure;

[0032] Figure 2 This is a schematic diagram of a module according to another embodiment of the video surveillance system of this disclosure;

[0033] Figure 3 This is a schematic diagram of an edge computing side processing module in one embodiment of a video surveillance system according to the present disclosure;

[0034] Figure 4 This is a flowchart illustrating an embodiment of the video surveillance method according to the present disclosure;

[0035] Figure 5 This is a flowchart illustrating another embodiment of the video surveillance method according to the present disclosure. Detailed Implementation

[0036] The present disclosure will now be described more fully with reference to the accompanying drawings, which illustrate exemplary embodiments of the present disclosure. The technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present disclosure without creative effort are within the scope of protection of the present disclosure.

[0037] In one embodiment, such as Figure 1 As shown, this disclosure provides a video surveillance system, including a device-side processing module 11, a decision-making module 12, and an edge computing-side processing module 13. The device-side processing module 11 is located on the monitoring device side and can be deployed within the monitoring device, a server connected to the monitoring device, or other locations. The monitoring device includes monitoring probes, cameras, etc. The decision-making module 12 can be deployed in the cloud, a management system, or other locations. The edge computing-side processing module 13 is located on the edge computing device side and can be deployed within the edge computing device, or other locations. The edge computing device can be an MEC device, etc., which utilizes a wireless access network to provide users with the services and cloud computing functions they need locally.

[0038] The device-side processing module 11 performs target detection processing on the video collected by the monitoring equipment, determines the target area, and obtains the corresponding target sub-image. The target can be a pedestrian, vehicle, etc., and the target sub-image can be a portion of the image in the video frame. The device-side processing module 11 calculates the Intersection over Union (IOU) information of the target area determined at different times and determines the trajectory of the target based on the IOU information. The device-side processing module 11 obtains the channel detection information of the wireless communication channel, which is the wireless channel for transmitting the target sub-image, etc.

[0039] Decision module 12 determines whether to transmit information, including a target sub-graph, to edge computing processing module 13 based on channel detection information and IOU information. If the decision module 12 determines to transmit information to edge computing processing module 13, it sends the compressed target sub-graph to edge computing processing module 13. Edge computing processing module 13 obtains target trajectory information based on the target sub-graph, including the trajectories of pedestrians, vehicles, etc.

[0040] The video surveillance system disclosed herein is an intelligent video surveillance system based on edge-end collaborative computing. It can adapt to the communication environment, effectively save bandwidth resources and computing resources, reduce costs, and achieve efficient and intelligent video surveillance.

[0041] In one embodiment, such as Figure 2 As shown, the device-side processing module 11 includes a target detection unit 111, a target tracking unit 112, and a channel detection unit 113. The target detection unit 111 uses a trained target detection model to perform target detection processing on the video, determine the target region, and obtain the target sub-image. The parameters of the target detection model are integer parameters, and the target detection model includes a Ghost module.

[0042] Object detection models can be various neural network models. For example, the object detection model can be the YOLOv5 neural network model. Improving the YOLOv5 model by adding a Ghost module can reduce redundant operations in the results and decrease computational complexity. Various existing training methods can be used to train the YOLOv5 model. The trained YOLOv5 model can be converted to an Openvino model (an existing toolkit) and quantized. Quantizing the float32 parameters of the YOLOv5 model to int8 parameters can reduce the size of the YOLOv5 model. The quantized YOLOv5 model can then be deployed on edge devices, such as monitoring equipment or servers connected to the monitoring equipment.

[0043] For example, the target detection unit 111 uses a target detection model to obtain detection information corresponding to the video. in, Information for the i-th target being detected. This indicates the coordinates of the top-left point of the detection box. Indicates the width of the detection box. Indicates the height of the detection frame. The detection category is indicated, and t represents the t-th frame of the video; the target region is determined based on the detection information. Target detection unit 111 determines the target region based on the detection information. Obtain the corresponding target subgraph, denoted as

[0044] Target tracking unit 112 determined

[0045]

[0046] in, To be with at t The corresponding detection target area For the time at t-1 The corresponding target area for detection.

[0047] The target tracking unit 112 obtains a similarity matrix, which is the IOU matrix. Based on the IOU threshold, the target tracking unit 112 determines associated targets, assigns the same ID to associated targets, and determines the trajectory of the associated targets based on the ID. For example, if the IOU threshold is set to β, associated items, i.e., associated targets, are found based on β and existing graph matching algorithms. Matched targets are marked with the same ID; if no corresponding match is found, a new ID is assigned; if an ID does not exist for a long time, it is deleted; simultaneously, the trajectory (Trace) of each ID is recorded. id The coordinates of the target's center point are determined by saving them.

[0048] Channel detection information includes information such as wireless channel gain. Channel detection unit 113 acquires the wireless channel gain and performs discretization processing. Channel detection unit 113 can acquire the wireless channel gain using various existing methods, such as obtaining the wireless channel gain through Rayleigh fading and path loss, and simultaneously discretize the wireless channel gain, denoted as h.

[0049] Decision module 121 includes a decision calculation unit 121, an image optimization unit 122, and an information transmission unit 123. The image optimization unit 122 uses existing image compression methods to process the target sub-image sent by the target detection unit 111. Compression can be achieved using CRF in x264 on the target subgraph. Compression is performed, and the compression level is discretized, denoted as Z. The information transmission unit 123 is used to transfer the target sub-graph... The compressed data is sent to the edge computing side processing module 13.

[0050] The decision computing unit 121 uses reinforcement learning to make decisions. For example, the decision computing unit 121 determines whether to transmit information to the edge computing side processing module based on a deep Q-network and wireless channel gain and IOU information. The deep Q-network is an existing DQN network. The DQN algorithm optimizes the target by maximizing the Q-value function, selects actions by evaluating policy actions, and selects the action with the largest Q-value as the optimal response policy.

[0051] The update equation for a deep Q-network is:

[0052]

[0053] in, Let n(S,A) represent the learning rate, n(S,A) represent the action chosen in state S, and Q(S,A) represent the cumulative average reward for taking action A in state S. The reward function is Reward. Ultimately, Q(S,A) converges to the cumulative average reward for taking action A in state S. After convergence, the task can be processed on the edge or the endpoint depending on the specific circumstances.

[0054] State space: State = (h, IOU1), where h is the wireless signal gain and IOU1 is the wireless signal gain. i Represents the maximum IOU corresponding to the current i-th target, and... Discretization is performed to maintain the finiteness of the state space. The action space is Action = (N, Z), where N represents whether information is transmitted to the edge computing side processing module, and Z represents the compression level of the target subgraph; for example, N = 1 indicates transmission, N = 0 indicates no transmission, and Z represents the compression level of the image when N = 1. Therefore, the overall action space is finite.

[0055] reward function Where Y represents whether tracking was successful, and T represents the total latency; where 1 indicates successful tracking and 0 indicates failed tracking, and T represents the total latency, T = T1 + T2 + T3, where T1 represents the edge processing time, T2 represents the transmission latency, and T3 represents the edge processing time. The reward function aims to minimize the overall latency T while ensuring tracking accuracy, specifically when N = 0 in the Action, T2 and T3 = 0.

[0056] In one embodiment, such as Figure 3 As shown, the edge computing side processing module 13 includes a feature extraction module 131, a trajectory prediction module 132, and a matching module 133. The feature extraction module 131 processes the target sub-image using a trained feature extraction model to obtain the target's feature information. The trajectory prediction module 132 processes the target sub-image using a trained first trajectory detection model to obtain the target's first trajectory. The trajectory prediction module 132 uses a trained second trajectory detection model to predict the target's historical predicted trajectory to obtain the target's second trajectory. The trajectory prediction module 132 determines the trajectory distance between the first and second trajectories.

[0057] Matching module 133 calculates the cosine similarity between the feature information and the previous feature information, as the feature distance. Matching module 133 calculates the matching degree as n = λD1 + (1-λ)D2, where D1 is the trajectory distance, D2 is the feature distance, and λ is the coefficient. Matching module 133 determines the associated targets based on the matching degree, sets the same ID for the associated targets, and determines the trajectory of the associated targets based on the ID.

[0058] In one embodiment, the feature extraction model can use various neural network models, such as the existing MobileNet v3 network. The MobileNet v3 network can be trained using existing training methods, and the trained MobileNet v3 network can be used to extract features. The feature information of the (target sub-image) includes the target's shape, color, etc., and is denoted as... And storage.

[0059] The first and second trajectory detection models can use various neural network models. For example, they can employ existing LSTM (Long Short-Term Memory) networks. Considering the different forms of motion in different categories, multiple LSTM networks can be built and trained using existing training methods.

[0060] A trained LSTM network (first trajectory detection model) is used to detect trajectories of different categories of objects (targets) and obtain the first trajectory of the target. id For example, the current position of the target is detected using an LSTM network (first trajectory detection model), and based on the target's current position and the stored historical actual trajectories of the target, the first trajectory (Trace) of the target is obtained. id (The actual trajectory of the target at the current moment) and store it.

[0061] Will with Trace id The corresponding historical trajectory (the target's trajectory at the previous m time points) is predicted using a trained LSTM network (the second trajectory detection model) to obtain the predicted trajectory PTrace. id (Second trajectory). Calculate the predicted trajectory PTrace. id The distance between the predicted trajectory and the actual detected trajectory of the target is denoted as D1. For example, calculating the predicted trajectory PTrace... id The coordinates of the points on the first trace id The distance between the corresponding coordinate points on the actual detection trajectory of the target is denoted as D1. D1 can be Euclidean distance, etc.

[0062] Calculation using cosine similarity as well as Noted as D2, for example, Here, cosin is the existing cosine similarity function. The overall matching degree n = λD1 + (1-λ)D2, where λ is a coefficient, 0 < λ < 1, and the associated target is determined based on the comprehensive matching. For example, when n > a preset threshold, it indicates that the target matching is successful. Matched targets are marked with the same ID. If there is no corresponding match, a new ID is assigned. If an ID does not exist for a long time, it is deleted. At the same time, the trace of each ID is recorded. id Among them, Trdce id The information saved is the coordinates of the target center point.

[0063] The video surveillance system in the above embodiments includes a device-side processing module, a decision module, and an edge computing-side processing module. For the acquired video stream, the device-side processing module first performs target detection, mainly detecting people, vehicles, etc., extracting a sub-image containing only the target, and simultaneously recording the target's trajectory. It then submits the current channel information and IOU to the decision module. The decision module decides whether the current task should be executed by the edge computing-side processing module. If it is executed by the edge computing-side processing module, it performs image compression at the appropriate level according to the decision module, obtains and records the target trajectory information based on the target sub-image, and finally summarizes, updates, and displays the results to the user.

[0064] The device-side processing module and the edge computing-side processing module are equipped with detection and tracking algorithms of different computational complexities for the identification and tracking of key targets in the video. The algorithm complexity and recognition accuracy of the device-side processing module are lower than those of the edge computing-side processing module. Meanwhile, the decision module makes decisions through reinforcement learning, achieving self-adaptation in the communication environment and autonomous task allocation. Through these three modules, an efficient intelligent video surveillance system can be effectively realized.

[0065] Different algorithms are implemented in the device-side processing module and the edge computing-side processing module according to different computing resource consumption, which can effectively reduce and save resources; the image interaction transmission with the edge computing-side processing module is carried out by transmitting compressed sub-images instead of the original video frames, which can save transmission resources; the edge computing-side processing module can freely select computing tasks through reinforcement learning, thereby saving computing resources, bandwidth resources and improving the accuracy of intelligent monitoring.

[0066] Figure 4 This is a flowchart illustrating an embodiment of the video surveillance method according to the present disclosure, as follows: Figure 4 As shown:

[0067] Step 401: The device-side processing module performs target detection processing on the video collected by the monitoring device, determines the target area, and obtains the corresponding target sub-image.

[0068] Step 402: The device-side processing module calculates the Intersection over Union (IOU) information of the detection target area determined at different times, and determines the trajectory of the target based on the IOU information.

[0069] Step 403: The device-side processing module obtains the channel detection information of the wireless communication channel.

[0070] Step 404: The decision module determines whether to transmit information to the edge computing side processing module based on channel detection information and IOU information.

[0071] Step 405: If the decision module determines that information should be transmitted to the edge computing side processing module, it sends the compressed target sub-graph to the edge computing side processing module.

[0072] Step 406: The edge computing side processing module obtains the target trajectory information based on the target sub-graph.

[0073] In one embodiment, the device-side processing module uses a trained object detection model to perform object detection processing on the video, determine the target region, and obtain the target sub-image; wherein the parameters of the object detection model are integer parameters, and the object detection model includes a Ghost module. The device-side processing module uses the object detection model to obtain detection information corresponding to the video. in, Information for the i-th target being detected. This indicates the coordinates of the top-left point of the detection box. Indicates the width of the detection box. Indicates the height of the detection frame. The detection category is represented by t, which represents the t-th frame of the video. The target region is determined based on the detection information. The device-side processing module obtains the corresponding target sub-image based on the detection information.

[0074] Equipment-side processing module determined

[0075]

[0076] in, To be with at t The corresponding detection target area For the time at t-1 The corresponding detection target area. The device-side processing module determines the associated targets based on the IOU threshold, sets the same ID for the associated targets, and determines the trajectory of the associated targets based on the ID.

[0077] Channel detection information includes wireless channel gain. The device-side processing module acquires the wireless channel gain and discretizes it. The decision module, based on a deep Q-network and considering the wireless channel gain and IOU information, determines whether to transmit information to the edge computing-side processing module.

[0078] Figure 5 This is a flowchart illustrating another embodiment of the video surveillance method according to the present disclosure, as follows: Figure 5 As shown:

[0079] Step 501: The edge computing side processing module uses the trained feature extraction model to process the target sub-image and obtain the feature information of the target.

[0080] Step 502: The edge computing side processing module uses the trained first trajectory detection model to process the target sub-image and obtain the first trajectory of the target; it uses the trained second trajectory detection model to predict the historical predicted trajectory of the target and obtain the second trajectory of the target.

[0081] Step 503: The edge computing side processing module determines the trajectory distance between the first trajectory and the second trajectory.

[0082] Step 504: The edge computing side processing module calculates the cosine similarity between the feature information and the previous feature information, which is used as the feature distance.

[0083] Step 505: The edge computing side processing module calculates the matching degree as n = λD1 + (1-λ)D2; where D1 is the trajectory distance, D2 is the feature distance, and λ is the coefficient.

[0084] Step 506: The edge computing side processing module determines the associated targets based on the matching degree, sets the same ID for the associated targets, and determines the trajectory of the associated targets based on the ID.

[0085] In one embodiment, this disclosure provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the video surveillance method as described in any of the preceding embodiments.

[0086] The video surveillance system, method, and storage medium in the above embodiments can achieve adaptive communication environment and autonomous allocation of computing tasks, effectively saving bandwidth resources, computing resources, reducing costs, improving the accuracy of intelligent monitoring, and enhancing the user experience.

[0087] The methods and systems of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0088] The description in this disclosure is provided for illustrative and descriptive purposes only and is not intended to be exhaustive or to limit the disclosure to its forms. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to better illustrate the principles and practical application of this disclosure and to enable those skilled in the art to understand this disclosure and to design various embodiments with various modifications suitable for a particular purpose.

Claims

1. A video surveillance system, comprising: The device-side processing module is used to perform target detection processing on the video collected by the monitoring equipment, determine the target area, and obtain the corresponding target sub-image. Calculate the Intersection over Union (IOU) information of the target area determined at different times, and determine the trajectory of the target based on the IOU information; obtain the channel detection information of the wireless communication channel; The decision module is used to determine whether to transmit information to the edge computing side processing module based on the channel detection information and the IOU information. If it is determined that information will be transmitted to the edge computing side processing module, the compressed target sub-graph will be sent to the edge computing side processing module. The channel detection information includes the wireless channel gain; the device-side processing module includes: A channel detection unit is used to acquire the wireless channel gain and perform discretization processing; The decision module includes: a decision calculation unit, used to determine whether to transmit information to the edge computing side processing module based on the deep Q network and according to the wireless channel gain and the IOU information; The update equation for the deep Q-network is: ; Representation learning rate, Characterized by the choice in state S ; This represents the cumulative average reward when taking action A in state S. The reward function; the state space h represents the wireless signal gain. Represents the maximum IOU corresponding to the i-th target; action space , Indicates whether information is transmitted to the edge computing side processing module. Characterizing the compression level of the target subgraph; reward function Y represents whether tracking was successful, and T represents the total latency; The edge computing side processing module is used to obtain target trajectory information based on the target sub-graph.

2. The system as claimed in claim 1, wherein, The device-side processing module includes: The object detection unit is used to perform object detection processing on the video using a trained object detection model, determine the target region, and obtain the target sub-image; wherein, the parameters of the object detection model are integer parameters; the object detection model includes: a Ghost module.

3. The system as described in claim 2, wherein, The target detection unit is used to obtain detection information corresponding to the video using the target detection model. ;in, For the first detection Information about the target , This indicates the coordinates of the top-left point of the detection box. Indicates the width of the detection box. Indicates the height of the detection frame. Indicates the category of detection. Represented as the t-th frame of the video; the detection target region is determined based on the detection information.

4. The system as described in claim 3, wherein, The target detection unit is used to obtain the corresponding target sub-image based on the detection information.

5. The system as described in claim 3, wherein, The device-side processing module includes: Target tracking unit, used to determine ； in, For the time t The corresponding detection target area For the time at t-1 The corresponding target area for detection; The target tracking unit is also used to determine associated targets based on an IOU threshold, set the same ID for associated targets, and determine the trajectory of associated targets based on the ID.

6. The system as claimed in claim 1, wherein, The edge computing side processing module includes: The feature extraction module is used to process the target sub-image using a trained feature extraction model to obtain the feature information of the target; The trajectory prediction module is used to process the target sub-image using a trained first trajectory detection model to obtain the first trajectory of the target; to predict the historical predicted trajectory of the target using a trained second trajectory detection model to obtain the second trajectory of the target; and to determine the trajectory distance between the first trajectory and the second trajectory. The matching module is used to calculate the cosine similarity between the feature information and the previous feature information, as the feature distance; and to calculate the matching degree. D1 is the trajectory distance, and D2 is the feature distance. The coefficient is used to determine the associated targets based on the matching degree. The associated targets are assigned the same ID, and the trajectory of the associated targets is determined based on the ID.

7. A video surveillance method, comprising: The device-side processing module performs target detection processing on the video collected by the monitoring equipment, determines the target area, and obtains the corresponding target sub-image. The device-side processing module calculates the intersection-union ratio (IOU) information of the detection target area determined at different times, and determines the trajectory of the target based on the IOU information; The device-side processing module acquires channel detection information of the wireless communication channel; Based on the channel detection information and the IOU information, the decision module determines whether to transmit information to the edge computing side processing module; The channel detection information includes the wireless channel gain; the device-side processing module acquires the channel detection information of the wireless communication channel, including: The device-side processing module acquires the wireless channel gain and performs discretization processing. The decision module determines whether to transmit information to the edge computing side processing module based on the channel detection information and the IOU information, including: The decision module determines whether to transmit information to the edge computing side processing module based on the deep Q network and the wireless channel gain and the IOU information. When the decision module determines that information should be transmitted to the edge computing side processing module, it sends the compressed target sub-graph to the edge computing side processing module. The update equation for the deep Q-network is: ; Representation learning rate, Characterized by the choice in state S ; This represents the cumulative average reward when taking action A in state S. The reward function; the state space h represents the wireless signal gain. Represents the maximum IOU corresponding to the i-th target; action space , Indicates whether information is transmitted to the edge computing side processing module. Characterizing the compression level of the target subgraph; reward function Y represents whether tracking was successful, and T represents the total latency; The edge computing side processing module obtains target trajectory information based on the target sub-graph.

8. The method as described in claim 7, wherein the device-side processing module performs target detection processing on the video collected by the monitoring device, determines the target area, and obtains the corresponding target sub-image, including: The device-side processing module uses a trained target detection model to perform target detection processing on the video, determine the target region, and obtain the target sub-image. The parameters of the target detection model are integer parameters; the target detection model includes a Ghost module.

9. The method of claim 8, wherein determining the detection target region comprises: The device-side processing module uses the target detection model to obtain detection information corresponding to the video. ;in, For the first detection Information about the target , This indicates the coordinates of the top-left point of the detection box. Indicates the width of the detection box. Indicates the height of the detection frame. Indicates the category of detection. Represented as the t-th frame of the video; the detection target region is determined based on the detection information.

10. The method of claim 9, wherein obtaining the target subgraph comprises: The device-side processing module obtains the corresponding target sub-image based on the detection information.

11. The method of claim 10, wherein the device-side processing module calculates the Intersection over Union (IOU) information of the detection target region determined at different times, and determines the trajectory of the target based on the IOU information, comprising: The device-side processing module determines ； in, For the time t The corresponding detection target area For the time at t-1 The corresponding target area for detection; The device-side processing module determines associated targets based on the IOU threshold, sets the same ID for associated targets, and determines the trajectory of associated targets based on the ID.

12. The method of claim 7, wherein the edge computing side processing module obtains target trajectory information based on the target subgraph, comprising: The edge computing side processing module uses a trained feature extraction model to process the target sub-image and obtain the target's feature information; The edge computing side processing module uses a trained first trajectory detection model to process the target sub-image to obtain the first trajectory of the target; and uses a trained second trajectory detection model to predict the historical predicted trajectory of the target to obtain the second trajectory of the target. The edge computing side processing module determines the trajectory distance between the first trajectory and the second trajectory; The edge computing side processing module calculates the cosine similarity between the feature information and the previous feature information, as the feature distance; The edge computing side processing module calculates the matching degree as follows: Where D1 is the trajectory distance and D2 is the feature distance. For coefficients; The edge computing side processing module determines the associated targets based on the matching degree, sets the same ID for the associated targets, and determines the trajectory of the associated targets based on the ID.

13. A computer-readable storage medium that non-transitoryly stores computer instructions, which are executed by a processor according to any one of claims 7 to 12.

Images