5G high-definition video monitoring method and system for smart city and medium

Abstract

The invention relates to the technical field of video monitoring communication, in particular to a 5G high-definition video monitoring method and system for a smart city, and a medium, and the method specifically comprises the steps: obtaining a 5G high-definition video based on the moving features of each feature point of a moving target between each frame and a previous frame image in a monitoring video, and the gray gradient change condition in each channel image of an RGB image of each video frame; constructing an information rich characteristic value of each video frame; calculating a bandwidth influence coefficient of each detection time period based on a data fluctuation abnormal condition of the network bandwidth data in each preset detection time period; constructing a video transmission quality coefficient by combining the features, optimizing a quantization parameter of a video coding algorithm in the current detection time period, carrying out coding transmission on the monitoring video in the current detection time period, and carrying out intelligent analysis on the transmitted monitoring video; the problem of frame loss or low video quality of video content with relatively high potential information value is reduced; and the interference on subsequent video monitoring intelligent analysis and target detection can be reduced.

Description

Technical Field

[0001] This application relates to the technical field of video surveillance communication, and particularly to a 5G high-definition video surveillance method, system and medium for smart cities. Background Art

[0002] In today's digital age, the construction of smart cities is becoming an important trend in global urban development. As an important technical means, the video surveillance system plays a crucial role in the construction of smart cities. It can not only improve the security of the city, but also provide strong support for the management and operation of the city. Through cameras installed in public places, transportation hubs, commercial areas, etc., the video surveillance system can timely detect and handle various security hazards and ensure the safe operation of the city.

[0003] City surveillance videos are often restricted during transmission due to insufficient network bandwidth. To alleviate this contradiction, data transmission needs to be optimized through compression technology to achieve efficient utilization of limited bandwidth. Among them, rate control plays an important role in video coding and compression technology, which can balance video quality and bandwidth usage and optimize video storage and transmission efficiency. However, city surveillance videos have strong dynamics and complexity. Conventional surveillance methods fail to fully consider the impact of video semantic information and bandwidth changes, lack adaptability to dynamic scenes and network bandwidth fluctuations, and may cause frame loss or low video quality during the transmission of valuable video content, thereby interfering with the accuracy of subsequent intelligent analysis and target detection of video surveillance content. Summary of the Invention

[0004] To solve the above technical problems, the purpose of this application is to provide a 5G high-definition video surveillance method, system and medium for smart cities. The specific technical solutions adopted are as follows: In the first aspect, an embodiment of this application provides a 5G high-definition video surveillance method for smart cities. The method includes the following steps: Collect surveillance videos of urban public service areas and network bandwidth data at each moment; Based on the movement characteristics of each feature point of the moving target between each video frame and its previous video frame, construct a difference significant coefficient for each feature point of the moving target, and determine the movement diversity index of each video frame based on the difference significant coefficient; Calculate the complex index of content color change in each video frame based on the gradient characteristics and the degree of gray value distribution chaos in each channel image of the RGB image of each video frame, and determine the information-rich feature value of each video frame based on the movement diversity index and the complex index of content color change; Calculate the bandwidth impact coefficient of each detection period based on the abnormal data fluctuation of the network bandwidth data within each preset detection period; Construct a sequence by enriching the eigenvalue construction sequence with the information of all video frames in the current detection period and the adjacent detection periods before it, denoted as the first sequence. Construct a sequence by the bandwidth influence coefficients of the current detection period and the adjacent detection periods before it, denoted as the second sequence. Construct the video transmission quality coefficient of the current detection period based on the similarity between the first sequence and the second sequence; Optimize the quantization parameters of the video coding algorithm in the current detection period based on the video transmission quality coefficient. Combine the video coding algorithm to encode and transmit the surveillance video in the current detection period, and perform intelligent analysis on the transmitted surveillance video.

[0005] In one embodiment, the process of obtaining the motion diversity index of each video frame is as follows: Take each video frame and its previous video frame as the input of the optical flow method, and the output is the coordinate positions and motion vectors of each feature point corresponding to each moving object in each video frame; Cluster the coordinate positions of all feature points in each video frame to obtain each cluster and the local density of each feature point; Take the mean value of the similarity between the motion vector of each feature point and the motion vectors of other feature points in the cluster as the intra-cluster motion vector similarity of each feature point; Construct the difference significant coefficient of each feature point based on the motion vector, local density, and intra-cluster motion vector similarity of each feature point; Take the sum value of the difference significant coefficients of all feature points in each video frame as the motion diversity index of each video frame.

[0006] In one embodiment, the expression of the difference significant coefficient of each feature point is: , where is the difference significant coefficient of the current feature point; B represents the modulus of the motion vector of the current feature point; C represents the local density of the current feature point; A is the intra-cluster motion vector similarity of the current feature point; represents the exponential function with the natural constant e as the base.

[0007] In one embodiment, the expression of the complex index of the content color change is: , where is the complex index of the content color change in the current video frame, is the number of channel images of the RGB image; represents the mean value of the gradients of all pixel points in the th channel image of the RGB image of the current video frame; represents the standard deviation of the gray values of all pixel points in the th channel image of the RGB image of the current video frame.

[0008] In one embodiment, the information-rich feature value of each video frame is the product of the motion diversity index of each video frame and the complexity index of the content color change.

[0009] In one embodiment, the process of determining the bandwidth influence coefficient of each detection period is as follows: Take all network bandwidth data within each detection period as the input of the anomaly detection algorithm, and the output is the anomaly points of the network bandwidth data in each detection period; calculate the absolute value of the difference between each anomaly point and the network bandwidth data at the two adjacent acquisition times before and after it respectively; take the sum of all the absolute values of the differences of all anomaly points within all detection periods as the network jitter coefficient, and take the product of the mean value of all network bandwidth data within each detection period and the network jitter coefficient as the bandwidth influence coefficient of each detection period.

[0010] In one embodiment, the process of obtaining the video transmission quality coefficient of the current detection period is as follows: Calculate the average value of the information-rich feature values of all video frames within each detection period, denoted as the information-rich feature mean value of each detection period; denote the sequence composed of the normalized values of the information-rich feature mean values of the current detection period and the preset number of adjacent detection periods before it as the first sequence, and denote the sequence composed of the normalized values of the bandwidth influence coefficients of the current detection period and the preset number of detection periods before it as the second sequence; take the DTW distance between the first sequence and the second sequence as the video transmission quality coefficient of the current detection period.

[0011] In one embodiment, the process of optimizing the quantization parameter of the video coding algorithm for the current detection period based on the video transmission quality coefficient is as follows: Obtain the normalized value of the video transmission quality coefficient of the current detection period, and take the normalized value as the input of a preset mapping function, and the output is the optimized quantization parameter of the current detection period, where the optimized quantization parameter of the current detection period is negatively correlated with the normalized value.

[0012] In a second aspect, an embodiment of the present application further provides a 5G high-definition video monitoring system for a smart city. A computer program is stored in the system, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0013] In a third aspect, an embodiment of the present application further provides a 5G high-definition video monitoring medium for a smart city, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of the method described in the first aspect are implemented.

[0014] The embodiments of the present application have at least the following beneficial effects: In the present application, by collecting the monitoring videos of urban public service areas and the network bandwidth data at each moment, the motion diversity index of each video frame is analyzed based on the moving characteristics of the feature points of the moving targets between each video frame and its previous video frame, reflecting the dynamic characteristics of the video frame and evaluating the richness of the information contained in each video frame; based on the change of the gray gradient in each channel image of the RGB image of each video frame, the complexity of the color change of the content contained in each video frame is analyzed, further evaluating the richness of the information contained in each video frame; and then the information-rich feature value of each video frame is constructed; based on the abnormal data fluctuation of the network bandwidth data within each preset detection period, the bandwidth influence coefficient of each detection period is calculated, analyzing the influence of the network bandwidth state on the video transmission quality; combining the above features to construct a video transmission quality coefficient, and based on the video transmission quality coefficient, optimizing the quantization parameter of the video coding algorithm for the current detection period, and combining the video coding algorithm to encode and transmit the monitoring video within the current detection period, which can adjust the compression degree of the transmitted video in real time according to the content of the monitoring screen and the influence of the bandwidth state, ensuring the video quality during the video monitoring process, optimizing the storage and transmission efficiency, and reducing the problem of frame loss or low video quality of video content with high potential information value; helping to reduce the interference to subsequent video monitoring intelligent analysis and target detection. Description of the Drawings

[0015] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0016] Figure 1 It is a flowchart of the steps of a 5G high-definition video monitoring method for smart cities provided by an embodiment of the present application; Figure 2 It is a schematic diagram of the acquisition process of the motion diversity index. Detailed Embodiments

[0017] In order to further elaborate on the technical means and effects adopted by the present application to achieve the predetermined invention purpose, the following, in combination with the drawings and preferred embodiments, details the specific embodiments, structures, features and effects of the 5G high-definition video monitoring method, system and medium for smart cities proposed according to the present application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application pertains.

[0019] The following specifically describes the specific solutions of the 5G high-definition video surveillance method, system, and medium for smart cities provided by this application in conjunction with the accompanying drawings.

[0020] Please refer to Figure 1 , which shows the flowchart of the steps of the 5G high-definition video surveillance method for smart cities provided by an embodiment of this application. The method includes the following steps: Step S1, collect the surveillance videos of urban public service areas and the network bandwidth data at each moment.

[0021] In the front-end of smart city surveillance, high-definition video collection and optimized transmission of the city are mainly carried out, while the terminal realizes intelligent analysis and target detection of video content. The surveillance content includes face recognition, license plate recognition, and pedestrian flow prediction. The front-end consists of video collection devices, 5G base stations, and edge servers. To improve the image quality, high-resolution cameras are selected for shooting in smart city surveillance. The cameras are installed at various key positions in urban public service areas, including intersections, squares, and commercial areas. And 5G wireless sensing technology is used for video signal transmission. Wireless transmission has the advantages of flexible wiring and convenient expansion, and is suitable for surveillance scenarios with difficult wiring or frequent movement. The video data collected by the cameras is sent to the edge server of the processing unit in the area after compression processing and processed in the edge server, which can greatly reduce the workload and storage capacity of the video surveillance center. The base station equipped with the edge server preprocesses the collected video stream and performs bitrate control, and sends the processed video stream to the terminal to execute the understanding task.

[0022] The video transmission process consumes network bandwidth. When the bandwidth status of the 5G network is poor, frame loss may occur in the urban surveillance videos received by the terminal server. Therefore, it is necessary to monitor the network bandwidth in real time during transmission. Set the video capture frame rate to 30FPS and the collection frequency of network bandwidth data to 100HZ. Thus, the urban surveillance video image data and the network bandwidth data during 5G transmission are obtained.

[0023] It should be noted that only one setting method for the video capture frame rate and the collection frequency of network bandwidth data is provided in the embodiments of this application. Implementers can set the video capture frame rate and the collection frequency of network bandwidth data according to actual situations, and this application does not make specific restrictions.

[0024] Step S2: Based on the movement characteristics of each feature point of the moving object between each video frame and its previous video frame, construct the difference significance coefficient of each feature point of the moving object, and determine the motion diversity index of each video frame based on the difference significance coefficient.

[0025] The high-definition video surveillance scenarios in smart cities are very diverse, covering multiple fields such as public safety, traffic management, public services, and environmental monitoring. In addition to real-time monitoring, the collected videos can also be combined with intelligent analysis technologies to mine and analyze massive amounts of data to discover valuable information hidden in the data. The efficiency and quality of video transmission directly affect the accuracy of subsequent video surveillance content data mining and analysis work. This application takes the video in the public service field of urban surveillance as an example and optimizes it during the video transmission process. Before transmission, this application uses the High Efficiency Video Coding (HEVC) technology to encode the collected video data. Among them, rate control plays an important role in high-efficiency video coding, and the quantization parameter is an important parameter affecting the video bit rate, and this value reflects the compression degree of the video. Usually, the higher the quality requirement for the surveillance video, the lower the value of the quantization parameter. Due to the strong dynamic and complex characteristics of the surveillance video, this application combines the dynamic and complex characteristics of the video content and the state of the network bandwidth to optimize and adjust the video coding to improve the transmission efficiency of urban video surveillance.

[0026] In the surveillance videos in the public service field, the density of people will change over time, seasons, and activity arrangements. The behavior patterns of people in different places are different, and thus different degrees of dynamic characteristics appear. The more information is contained in the area with more obvious dynamic characteristics of the video frame. And the complex state of the image corresponding to the video frame may change at any time. For example, in a crowded state of people, the complexity of the video frame will increase, and the more important information needs to be transmitted. When encoding, the video quality needs to be correspondingly improved for the area with more information to be transmitted. Therefore, take a certain video frame as an example for the following processing.

[0027] Since different video frames contain different amounts of information, to improve the coding efficiency, this application sets different quantization parameters for each video frame during coding. The method for obtaining the dynamic characteristics and complexity characteristics corresponding to each video frame is as follows: First, use the current video frame and its previous video frame as the input of the Lucas-Kanade optical flow method to obtain the coordinate positions and motion vectors of each feature point corresponding to each moving object in the current video frame. Among them, the feature points in this application are corner points.

[0028] Furthermore, the behavior patterns of people in public service places are diverse, and there are certain differences in the directions and magnitudes of the motion vectors of different feature points. When performing semantic analysis on the subsequent surveillance videos in the public service field, areas with fast movement and concentrated moving targets may require higher video quality. Therefore, the coordinate positions of all moving target feature points in the current video frame are used as the input of the density peak clustering algorithm, and all moving feature points are clustered to obtain the local density corresponding to each feature point and each cluster.

[0029] It should be noted that for the clustering of the coordinate positions of all moving feature points, this application only provides one clustering method. There are many existing clustering methods, and implementers can also use other clustering algorithms for the clustering of the coordinate positions of all moving feature points. This application does not make specific restrictions.

[0030] Furthermore, for any feature point, calculate the cosine similarity between the motion vector corresponding to this feature point and the motion vectors of each other feature point within its cluster, and calculate the mean value of all the cosine similarities of this feature point, which is denoted as the intra-cluster motion vector similarity of the current feature point. The smaller the intra-cluster motion vector similarity, the greater the difference in the motion directions between this feature point and other feature points within the cluster.

[0031] It should be noted that for the calculation of the similarity between the motion vectors of each feature point and other feature points within the cluster, this application only provides one similarity calculation method. There are many existing similarity calculation methods, and implementers can also use other similarity algorithms to calculate the similarity between the motion vectors of each feature point and other feature points within the cluster. This application does not make specific restrictions.

[0032] Furthermore, construct a significant difference coefficient for each feature point based on the motion vector, local density, and intra-cluster motion vector similarity of each feature point. Among them, the significant difference coefficients of each feature point are positively correlated with the modulus of the motion vector and the local density of each feature point, and negatively correlated with the intra-cluster motion vector similarity of each feature point. Preferably, in the embodiments of this application, the expression of the significant difference coefficient of each feature point can be: , where is the significant difference coefficient of the current feature point; B represents the modulus of the motion vector of the current feature point; C represents the local density of the current feature point; A is the intra-cluster motion vector similarity of the current feature point; represents the exponential function with the natural constant e as the base.

[0033] The magnitude of the motion vector of a feature point reflects the magnitude of the target motion, and the local density of the feature points reflects the degree of aggregation of the moving feature points within a local range; the greater the motion amplitude of the feature points, the denser the distribution of the moving feature points, the richer the motion information contained in the image, and the higher the diversity; the larger D is, the greater the degree of motion difference of the target at the position of the feature point.

[0034] Furthermore, the sum of the difference significant coefficients of all feature points in the current video frame is used as the motion diversity index of the current video frame, and this value reflects the dynamic characteristics of the video frame.

[0035] Step S3: Calculate the complexity index of the content color change in each video frame based on the gradient features and the degree of chaos in the gray value distribution in each channel image of the RGB image of each video frame, and determine the information-rich feature value of each video frame based on the motion diversity index and the complexity index of the content color change.

[0036] The complex characteristics of a video frame can be reflected by the richness of the colors contained in the image and the degree of their change. For example, in a library, the behavior patterns of people tend to be static or slow-moving. However, the colors and textures of its video frames are relatively rich, thus showing a relatively high complexity of the video content, and a high video quality is required for monitoring transmission. To obtain this feature, first, obtain each channel image of a single video frame in the RGB color space, and use the Laplacian of Gaussian operator to calculate the gradient of each pixel point in each channel image respectively; among them, the Laplacian of Gaussian operator is a well-known technology, and the specific process will not be elaborated here. Among them, RGB represents the colors of the three channels of red, green, and blue.

[0037] It should be noted that for the gradient detection of pixel points, this application only provides a gradient detection method. There are many existing gradient detection methods, and implementers can also use other gradient detection algorithms to obtain the gradient of pixel points. This application does not make specific restrictions.

[0038] Then, calculate the complexity index of the content color change in each video frame. The expression is: , where is the complexity index of the content color change in the current video frame; is the number of channel images of the RGB image. In this application, n = 3, that is, the three channels of R, G, and B; represents the th channel image of the current video frame RGB image, which is the mean value of the gradients of all pixel points; represents the th channel image of the current video frame RGB image, which is the standard deviation of the gray values of all pixel points.

[0039] The average value of all gradients in each image reflects the overall change intensity of the image pixels. The larger this value is, the more complex the color changes in the video frame; the larger the standard deviation of grayscale, the more complex the colors the image may contain; the obtained The larger it is, the greater the complexity of the color changes in the content contained in the video frame.

[0040] The motion diversity index and the complexity index of content color changes of each video frame respectively reflect the dynamic characteristics and complex characteristics of the content of each video frame. The two together reflect the richness of the information contained in each video frame. For example, the target motion characteristics are more prominent in some public service surveillance videos, while the color change characteristics are more prominent in some surveillance videos.

[0041] Take the product of the motion diversity index and the complexity index of content color changes of each video frame as the information richness characteristic value of each video frame. The larger this value is, the higher the requirement for image quality during video transmission.

[0042] Step S4, calculate the bandwidth impact coefficient of each detection period based on the data fluctuation anomaly of the network bandwidth data within each preset detection period.

[0043] The transmission of urban surveillance videos requires network bandwidth. In addition to real-time video transmission, the 5G network also needs to perform other urban data transmission tasks, which easily causes unstable fluctuations in network bandwidth. When the quality requirement for the transmitted video is relatively high, and the available network bandwidth is small or unstable, frame loss may occur. Therefore, it is necessary to perform real-time optimization of video coding by combining the richness characteristics of video content information and the network bandwidth status.

[0044] When the network bandwidth status is more unstable, it is more likely to have outliers due to local jitter. To obtain the local short-term change of the network bandwidth status, first, set t seconds as the duration of a detection period. Preferably, in the embodiments of the present application, the value of t is set to 5 seconds. As other embodiments of the present application, the implementer can set the value of t according to the actual situation. Then, take all the network bandwidth data collected within each detection period as the input of the SOS (Stochastic Outlier Selection Algorithm) detection algorithm, and the output is the outlier of the network bandwidth data in this detection period. Among them, the SOS detection algorithm is a well-known technology, and the specific process will not be elaborated.

[0045] It should be noted that for the outlier detection of all network bandwidth data within the detection period, the present application only provides an outlier detection method. There are many existing outlier detection methods, and the implementer can also use other outlier detection algorithms to obtain the outliers of all network bandwidth data within the detection period. The present application does not make specific restrictions.

[0046] Calculate the absolute value of the difference between the network bandwidth data of each anomaly point and the network bandwidth data at the two adjacent acquisition times before and after it respectively. Finally, denote the set composed of all anomaly points within all detection periods as the anomaly point set, and take the sum of all the absolute values of the differences of all anomaly points in the anomaly point set as the network jitter coefficient of the network bandwidth state. This value reflects the jitter degree of the network bandwidth. The higher the occupancy of other transmission tasks, the greater the delay of urban surveillance video transmission and the greater the impact on video quality. Furthermore, take the product of the mean value of all network bandwidth data within each detection period and the network jitter coefficient as the bandwidth impact coefficient of this detection period. The higher the mean value of network bandwidth data, the smaller the available bandwidth resources and the greater the impact on video transmission; the greater the network jitter coefficient, the more unstable the video transmission may be and the greater the impact; thus, the greater the bandwidth impact coefficient, the greater the impact degree of the network bandwidth state on the video transmission quality.

[0047] Step S5: Construct a sequence by using the information-rich feature values of all video frames within the current detection period and the adjacent detection periods before it, denoted as the first sequence, and construct a sequence by using the bandwidth impact coefficients of the current detection period and the adjacent detection periods before it, denoted as the second sequence. Construct the video transmission quality coefficient of the current detection period based on the similarity between the first sequence and the second sequence.

[0048] When the content richness of the video frame is higher and the bandwidth impact coefficient is smaller, it is more conducive to the efficient transmission of urban surveillance videos. Obtain the first N detection periods before the current detection period, where N is an integer within [7, 11]. Preferably, in the embodiments of the present application, the value of N is set to 7. Calculate the average value of the information-rich feature values of all video frames within each detection period, denoted as the information-rich feature mean value of each detection period. Denote the sequence composed of the normalized values of the information-rich feature mean values of the current detection period and its first N detection periods arranged in ascending order of time as the first sequence, and denote the sequence composed of the normalized values of the bandwidth impact coefficients of the current detection period and its first N detection periods arranged in ascending order of time as the second sequence; calculate the DTW (Dynamic Time Warping) distance between the first sequence and the second sequence, and take this DTW distance as the video transmission quality coefficient of the current detection period. This value reflects the level of video quality requirements during the transmission process. Furthermore, it affects the accuracy of video surveillance content analysis.

[0049] Normalize the video transmission quality coefficients of each detection period. Preferably, in the embodiments of the present application, the tanh function is used to normalize the video transmission quality coefficients of each detection period. It should be noted that for the normalization of the video transmission quality coefficients of each detection period, the present application only provides one normalization method. There are many existing normalization methods, and implementers can also use other normalization algorithms to normalize the video transmission quality coefficients of each detection period. The present application does not make specific restrictions.

[0050] Step S6: Optimize the quantization parameter of the video coding algorithm for the current detection period based on the video transmission quality coefficient. Combine with the video coding algorithm to encode and transmit the surveillance video within the current detection period, and perform intelligent analysis on the transmitted surveillance video.

[0051] If the video transmission quality coefficient is larger, the degree of compression required in the high-efficiency video coding process is lower, and the corresponding value of the quantization parameter is set smaller. If the video transmission quality coefficient is smaller, the degree of compression required in the high-efficiency video coding process is larger, and the corresponding value of the quantization parameter is set larger. The range of the quantization parameter set in this application is [15, 35], and the effect of the algorithm is relatively good within this range. Based on the above analysis, the quantization parameter is optimized in real time. Specifically: obtain the normalized value of the video transmission quality coefficient for the current detection period, use the normalized value as the input of a preset mapping function, and the output is the optimized quantization parameter for the current detection period. Among them, the optimized quantization parameter for the current detection period is negatively correlated with the normalized value, and the role of the mapping function is to limit the value of the optimized quantization parameter within the quantization parameter range [15, 35].

[0052] Preferably, in the embodiment of this application, the expression of the mapping function can be: , where H is the optimized quantization parameter for the current detection period; L is the normalized value of the video transmission quality coefficient for the current detection period.

[0053] As other embodiments of this application, implementers can also use other mapping functions to calculate the optimized quantization parameter for the current detection period.

[0054] Furthermore, for the surveillance video within the current detection period, perform encoding processing using the high-efficiency video coding algorithm with the optimized quantization parameter for this period and transmit it to the monitoring terminal. Among them, the high-efficiency video coding algorithm and the video transmission process are both well-known technologies, and the specific process will not be elaborated here.

[0055] Efficient video compression processing can significantly reduce bandwidth occupancy, optimize storage and transmission efficiency, and enhance the detail performance of images while ensuring video quality. Finally, perform object detection and intelligent analysis on the video at the monitoring terminal, including face recognition and abnormal behavior detection. The process is as follows: First, the terminal is equipped with a high-performance GPU for training and running deep learning models. Obtain the surveillance video stream from the video surveillance access and aggregation platform through the API (Application Programming Interface) interface. Before intelligent analysis, first decode, extract frames, and preprocess the video stream. The preprocessing in this embodiment includes resizing the image and normalizing to meet the model input requirements.

[0056] Then, a deep learning model is constructed based on the PyTorch framework, and the model is trained using the labeled dataset to enable it to identify and monitor the targets in the video. Among them, the target identified in this application is a person, and the labeled dataset in this application is as follows: Each image frame of the urban surveillance video within the past month is obtained, and the people in the image are marked with rectangular boxes in each image frame. All the marked image frames within the past month are used as the labeled dataset. In addition, in the embodiments of this application, the YOLO (You Only Look Once) model is used for target detection and intelligent analysis of the video. It should be noted that for target detection and intelligent analysis of the video using a deep learning model, this application only provides one deep learning model. There are many existing deep learning models, and implementers can also use other deep learning models such as YOLO, Faster R-CNN, SSD (Single Shot Multibox Detector) for target detection and intelligent analysis of the video, and this application does not make specific restrictions.

[0057] Finally, the preprocessed video frames are input into the trained deep learning model for real-time analysis. The model outputs the recognition results, including the target position information, and can further track the targets in the surveillance video while detecting abnormal behaviors.

[0058] It should be noted that the process of target detection and intelligent analysis of the video is a well-known technology, and the specific process will not be elaborated here.

[0059] The schematic diagram of the acquisition process of the motion diversity index is as Figure 2 shown.

[0060] Based on the same inventive concept as the above method, the embodiments of this application also provide a 5G high-definition video surveillance system for smart cities. The system stores a computer program, and when the computer program is executed by a processor, it implements the steps of any one of the above methods for the 5G high-definition video surveillance method for smart cities.

[0061] Based on the same inventive concept as the above method, the embodiments of this application also provide a 5G high-definition video surveillance medium for smart cities, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above methods for the 5G high-definition video surveillance method for smart cities.

[0062] In summary, the embodiment of the present application provides a 5G high-definition video monitoring method for a smart city. By collecting the monitoring videos of urban public service areas and the network bandwidth data at each moment, the motion diversity index of each video frame is analyzed based on the moving characteristics of the feature points of the moving targets between each video frame and its previous video frame, which reflects the dynamic characteristics of the video frame and evaluates the richness of the information contained in each video frame. The complexity of the color change of the content contained in each video frame is analyzed based on the change of the gray level gradient in each channel image of the RGB image of each video frame, and further evaluates the richness of the information contained in each video frame. Then, the information-rich feature value of each video frame is constructed. The bandwidth impact coefficient of each detection period is calculated based on the abnormal data fluctuation of the network bandwidth data within each preset detection period, and the impact of the network bandwidth state on the video transmission quality is analyzed. Combining the above features, a video transmission quality coefficient is constructed, and based on the video transmission quality coefficient, the quantization parameter of the video coding algorithm in the current detection period is optimized. Combining with the video coding algorithm, the monitoring video in the current detection period is encoded and transmitted, which can adjust the compression degree of the transmitted video in real time according to the content of the monitoring screen and the influence of the bandwidth state. While ensuring the video quality during the video monitoring process, it optimizes the storage and transmission efficiency, and reduces the problem of frame loss of video content with high potential information value or low video quality. It helps to reduce the interference to subsequent video monitoring intelligent analysis and target detection.

[0063] It should be noted that the above sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above specific embodiments of the present application have been described. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0064] Each embodiment in the present application is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and the key points of each embodiment are the differences from other embodiments.

[0065] The above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the principle of the present application shall be included in the protection scope of the present application.

Claims

1. A 5G high-definition video surveillance method for smart cities, characterized in that: The method comprises the following steps: Collect surveillance videos and network bandwidth data at all times in urban public service areas; Based on the movement characteristics of each feature point of the moving target between each video frame and its previous video frame, construct a difference significance coefficient of each feature point of the moving target, and determine the motion diversity index of each video frame based on the difference significance coefficient; Calculating the complexity index of content color change in each video frame based on the gradient features and gray value distribution disorder degree in each channel image of the RGB image of each video frame, and determining the information richness feature value of each video frame based on the motion diversity index and the complexity index of content color change; Calculate the bandwidth impact coefficient of each detection period based on the data fluctuation anomaly of the network bandwidth data within each preset detection period; A sequence is constructed by using information-rich feature values ​​of all video frames in the current detection period and the adjacent detection period before it, which is recorded as a first sequence; a sequence is constructed by using bandwidth impact coefficients of the current detection period and the adjacent detection period before it, which is recorded as a second sequence; and a video transmission quality coefficient of the current detection period is constructed based on the similarity between the first sequence and the second sequence; The quantization parameters of the video encoding algorithm of the current detection period are optimized based on the video transmission quality coefficient. The surveillance video in the current detection period is encoded and transmitted in combination with the video encoding algorithm, and the transmitted surveillance video is intelligently analyzed.

2. The 5G high-definition video surveillance method for smart cities according to claim 1, characterized in that: The process of obtaining the motion diversity index of each video frame is as follows: Each video frame and its previous video frame are used as the input of the optical flow method, and the output is the coordinate position and motion vector of each feature point corresponding to each moving target in each video frame; Cluster the coordinate positions of all feature points in each video frame to obtain the local density of each cluster and each feature point; The mean of the similarities between the motion vectors of each feature point and other feature points in the cluster is taken as the similarity of the motion vectors within the cluster of each feature point; The difference significance coefficient of each feature point is constructed based on the motion vector, local density and similarity of the motion vector within the cluster of each feature point; The sum of the difference significance coefficients of all feature points in each video frame is taken as the motion diversity index of each video frame.

3. The 5G high-definition video surveillance method for smart cities as claimed in claim 2, characterized in that: The expression of the difference significance coefficient of each feature point is: , where is the difference significance coefficient of the current feature point; B is the modulus of the motion vector of the current feature point; C is the local density of the current feature point; A is the similarity of the intra-cluster motion vector of the current feature point; Represents an exponential function with the natural constant e as the base.

4. The 5G high-definition video surveillance method for smart cities according to claim 1, characterized in that: The expression of the complex index of the content color change is: , where is the complex index of the content color change in the current video frame, is the number of channel images of RGB images; Indicates the RGB image of the current video frame. The mean value of the gradient of all pixels in the channel image; Indicates the RGB image of the current video frame. The standard deviation of the grayscale values ​​of all pixels in the channel image.

5. The 5G high-definition video surveillance method for smart cities according to claim 1, characterized in that: The information rich feature value of each video frame is the product of the motion diversity index of each video frame and the complexity index of content color change.

6. The 5G high-definition video surveillance method for smart cities according to claim 1, characterized in that: The process of determining the bandwidth influence coefficient of each detection period is as follows: All network bandwidth data in each detection period is used as the input of the anomaly detection algorithm, and the output is the anomaly point of the network bandwidth data in each detection period; the absolute value of the difference between each anomaly point and the network bandwidth data of the two adjacent collection moments before and after it is calculated respectively; the sum of all the absolute values ​​of the difference of all the anomaly points in all detection periods is used as the network jitter coefficient, and the product of the mean value of all network bandwidth data in each detection period and the network jitter coefficient is used as the bandwidth impact coefficient of each detection period.

7. The 5G high-definition video surveillance method for smart cities according to claim 1, characterized in that: The process of obtaining the video transmission quality coefficient of the current detection period is as follows: Calculate the average value of the information-rich feature values ​​of all video frames in each detection period, and record it as the information-rich feature mean of each detection period; A sequence consisting of normalized values ​​of the mean values ​​of information-rich features of a current detection period and a preset number of adjacent detection periods before it is recorded as a first sequence, and a sequence consisting of normalized values ​​of bandwidth impact coefficients of the current detection period and a preset number of detection periods before it is recorded as a second sequence; the DTW distance between the first sequence and the second sequence is used as a video transmission quality coefficient of the current detection period.

8. The 5G high-definition video surveillance method for smart cities according to claim 1, characterized in that: The process of optimizing the quantization parameters of the video encoding algorithm in the current detection period based on the video transmission quality coefficient is as follows: Obtain a normalized value of the video transmission quality coefficient of the current detection period, use the normalized value as an input of a preset mapping function, and output the optimized quantization parameter of the current detection period, wherein the optimized quantization parameter of the current detection period is negatively correlated with the normalized value.

9. A 5G high-definition video surveillance system for smart cities, wherein a computer program is stored in the system, characterized in that: When the computer program is executed by the processor, the steps of the 5G high-definition video surveillance method for smart cities as described in any one of claims 1 to 8 are implemented.

10. A 5G high-definition video surveillance medium for smart cities, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the 5G high-definition video surveillance method for smart cities as described in any one of claims 1 to 8 are implemented.

Images