An optimization method for adaptive playback support of video surveillance platform

Abstract

The present invention provides an optimization method for adaptive playback support of a video surveillance platform, the steps of which include: the server collects the device information and user habits of the terminal, and configures multiple types of video stream preprocessing strategies accordingly; the server selects the video stream preprocessing strategy based on the prediction operation, and pre-pushes the preprocessed video stream to the corresponding terminal. Among them, when the video screen ratio of the server push stream is not compatible with the screen ratio of the terminal player, the server crops and scales the pushed video stream based on the black edge removal algorithm. The present invention predicts user habits and combines preset strategies to achieve efficient adaptive streaming. On the one hand, it can push adaptive coded video streams and bit rates according to demand to reduce bandwidth occupancy, reduce playback delays, and expand terminal types; on the other hand, the server side uses feedback to push video streams with adapted screen ratios using the black edge removal algorithm, which can maximize the retention of effective information on the screen while reducing the pressure on terminal playback.

Description

Technical Field

[0001] The present invention relates to the technical field of digital information transmission, and in particular to an optimization method for adaptive playback support of a video monitoring platform. Background Art

[0002] With the development of technology in the digital age, video surveillance has become an indispensable part of modern life. Among them, common video surveillance refers to the use of image acquisition devices such as cameras to collect real-time images of specific areas, and transmit image information to monitoring centers or other terminal devices through network transmission technology to achieve remote monitoring, image recording and event warning of the target area.

[0003] Furthermore, there are many reasons why video surveillance is widely used in various fields. For example, in the construction of smart cities, video surveillance can help city managers grasp traffic flow, public safety and other situations in real time, and improve the city's operating efficiency and safety; in the field of smart transportation, video surveillance can monitor road conditions and vehicle driving conditions, providing strong support for traffic diversion and accident handling; in the field of electricity and energy, video surveillance can be used to monitor the operating status of facilities such as power stations and substations to ensure the stability and safety of energy supply; in the field of security, video surveillance is one of the important application technologies that can effectively prevent or record unauthorized behavior.

[0004] At present, in the existing technical field, the main body of the video surveillance platform is still the server. The connection between the server and multiple terminals is usually obtained through standard connection or Onvif protocol docking to obtain the video stream of the camera or monitoring device. These standards or protocols provide standardized solutions for the interconnection of video surveillance systems, allowing devices from different manufacturers to work together. However, the existing technology has some obvious drawbacks, as follows:

[0005] First, with the decline in technology and hardware costs, as well as the actual demand for monitoring image quality, the cameras connected to the video surveillance platform have captured clearer images in recent years. This is reflected in the technical level that the camera resolution obtained by the platform is usually 1920×1080 or above. However, since the bit rate and resolution of the background of the traditional platform will not change with the changes in the front-end window, this leads to excessively large bit rate and resolution of the push stream, which occupies a large amount of bandwidth. In order to adapt to such high-resolution video streams, the equipment needs to be equipped with high-performance equipment or graphics cards, etc., which undoubtedly increases the project cost. Moreover, in many actual application scenarios, such a large resolution is not always required. For example, in some occasions where simple monitoring is required, high resolution may be a waste of resources.

[0006] Secondly, the display needs to go through signaling interaction, which requires a long wait. When switching channels, it is also necessary to re-sess, which affects the response speed of video surveillance. In some scenarios with high real-time requirements, such as event processing with high timeliness requirements, this delay may have serious consequences.

[0007] Furthermore, the problem of mismatch between the terminal player ratio and resolution is also prominent, which easily leads to a large number of black edges. This not only affects the viewing experience, but also may lead to incomplete display of picture information. Especially when switching between multi-screen playback or single-screen display, different devices and window sizes may aggravate this mismatch phenomenon.

[0008] Finally, the mismatch between the camera encoding and the player decoding is also a common problem. Different cameras may use different encoding methods, and the player may not be fully compatible with all encoding formats. This may cause the video to not play properly or the playback quality to deteriorate, affecting the stability and reliability of the video surveillance system.

[0009] In summary, although video surveillance technology has played an important role in various fields, there are still some problems that need to be solved in the existing technology. In order to better meet the needs of practical applications, it is also necessary to improve the existing video surveillance-playback technology to improve its performance and adaptability. Summary of the invention

[0010] In view of the problems existing in the prior art, the present invention provides an optimization method for adaptively adjusting the picture quality of monitoring video stream push according to user habits.

[0011] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0012] The present invention provides a method for optimizing adaptive playback support of a video surveillance platform, which comprises the following steps:

[0013] The server collects terminal device information and user habits;

[0014] The server is configured with multiple types of video stream preprocessing strategies based on the device information and metadata of the original video stream; the multiple types of video stream preprocessing strategies are at least provided with a low bit rate video stream preprocessing strategy and a high bit rate video stream preprocessing strategy;

[0015] The server extracts multidimensional features based on the user habits, trains a machine learning model through the results of the multidimensional feature extraction, and obtains a user habit analysis model; the input of the user habit analysis model is the result of the multidimensional feature extraction, the label is the user's past operation, and the target is the user's predicted operation;

[0016] In the multi-dimensional feature extraction, the extracted features are classified into time dimension features, space dimension features and event dimension features; the time dimension features are at least provided with the user viewing time and the user viewing duration; the space dimension features are at least provided with the location information of the original video stream and the distance between the shooting location and the user; the event dimension features are at least provided with calendar information;

[0017] The server obtains the user's predicted operation in the next period by substituting the result of the current multi-dimensional feature extraction into the user habit analysis model;

[0018] The server selects a video stream preprocessing strategy based on the user predicted operation in the next time period, and pre-pushes the preprocessed video stream to the corresponding terminal based on a resolution adaptive algorithm;

[0019] The server also establishes a feedback connection with each terminal, and a black edge removal algorithm is configured in the server;

[0020] When the video screen ratio of the server pushes the video stream does not match the screen ratio of the terminal player, the terminal where the terminal player is located feeds back at least one frame of the video playback screen with black edges to the server; the server crops and scales the pushed video stream based on the black edge removal algorithm and the video playback screen with black edges.

[0021] Optionally, the black edge removal algorithm includes Canny edge detection and Hough transform straight line detection;

[0022] When the player window ratio of the terminal changes, the server performs Canny edge detection on the fed-back video playback screen with black edges to obtain an edge image of the video frame;

[0023] The server performs Hough transform straight line detection on the edge image of the video frame to obtain a black edge range;

[0024] The server processes the pushed video stream by a cropping method or a padding method based on the black border range.

[0025] Optionally, the device information includes a video playback support format, a video resolution support list, and video window parameters of the terminal device.

[0026] Optionally, the metadata of the original video stream includes the resolution and frame rate of the video shooting.

[0027] Optionally, the multi-type video stream preprocessing strategy includes:

[0028] The server compares the device information with metadata of the original video stream, and sets video stream transcoding parameters based on the comparison result;

[0029] The video stream transcoding parameters include video encoding format, resolution and frame rate;

[0030] The server also divides the multiple types of video stream preprocessing strategies into at least the low bit rate video stream preprocessing strategy and the high bit rate video stream preprocessing strategy based on the video resolution support list and the video window parameter in the device information.

[0031] Optionally, the multi-type video stream preprocessing strategy also includes video cropping and scaling.

[0032] Optionally, the multi-type video stream preprocessing strategy further includes: in the low-bitrate video stream preprocessing strategy, the video stream resolution is adapted to the terminal player small window or the terminal splicing window;

[0033] In the high-bitrate video stream preprocessing strategy, the video stream resolution is adapted to the maximized window of the terminal player or the full-screen playback of the terminal.

[0034] Optionally, the server includes an algorithm server and an edge server;

[0035] The video stream that has been pre-streamed and pre-processed is distributed and cached in the edge server.

[0036] Optionally, the machine learning model is a decision tree model, a naive Bayes model or a support vector machine model.

[0037] Optionally, the user habits include video stream selection, video window scaling, opening and closing.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] The present invention predicts user habits and realizes efficient adaptive streaming in combination with preset strategies. On the one hand, it can push adaptive encoded video streams and bit rates according to demand to reduce bandwidth occupancy, reduce playback delay, and expand terminal types. On the other hand, the server side uses feedback to push video streams adapted to the picture ratio using a black edge removal algorithm, which maximizes the retention of effective information on the picture while reducing the terminal playback pressure. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0041] Figure 1A schematic diagram of system connection relationships in a specific embodiment of the present invention;

[0042] Figure 2 The figure is a flow chart of video stream pushing in a specific embodiment of the present invention.

[0043] In the figure: 1. Central server, 2. Edge server, 3. Monitoring and shooting equipment, 4. Large screen display, 5. Web page, 6. Mobile terminal, 7. PC desktop. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0046] It is worth noting that the methods used in the present invention are all conventional methods unless otherwise specified; the raw materials and devices used are all conventional commercially available products, and their sources are not specifically limited unless otherwise specified.

[0047] like Figure 1 As shown, this embodiment provides an optimization method for adaptive playback support of a video surveillance platform, including the following steps:

[0048] First, the video monitoring platform is composed of servers, including a central server 1 and an edge server 2. The central server of this embodiment also serves as an algorithm server and is deployed with an algorithm service to coordinate the work of the edge server 2. The media service can be deployed in the central server 1 and / or the edge server 2 according to actual needs. This embodiment adopts a distributed deployment method to deploy the media service in the edge server 2. The monitoring and shooting equipment 3 distributed in various areas is connected to the server through the network. The server is connected to each authorized terminal through the network, wherein the terminals of this embodiment include but are not limited to a large-screen display 4, a web page terminal 5, a mobile terminal 6, and a PC desktop 7.

[0049] Among them, in the server construction link, the central server 1 uses a server model that meets the computing power requirements and is equipped with a multi-core processor to ensure the efficient operation of the algorithm service. Its memory capacity is adapted according to the number of edge servers to be coordinated and the estimated data processing volume. The network interface uses a 10 Gigabit Ethernet interface to ensure high-speed and stable communication with the edge server 2 and each expected terminal. The edge server 2 is planned and deployed according to the geographical distribution of the monitoring area. The monitoring camera 3 close to the corresponding area uses a network interface that matches the central server 1. Its storage capacity is combined with the amount of data generated by the monitoring camera 3 in the covered area and the terminal access frequency to ensure that it can meet the local data cache requirements. For network connection, the monitoring camera 3 and the server are connected by wired or wireless connection, and a reliable transmission network is established using an industrial Ethernet switch to ensure the stable upload of the original video data; among the terminal devices, fixed-position terminals such as large-screen displays 4 and PC desktops 7 are preferentially connected to the wired network and connected to the server via Gigabit Ethernet; the mobile terminal 6 and the web terminal 5 are adapted to the wireless Wi-Fi 6 or 5G network, and automatically switch connections according to the network signal strength and bandwidth to ensure the smoothness of video streaming transmission.

[0050] Secondly, combined Figure 2 As shown, in actual application, the application scenario of this embodiment follows the conventional design of monitoring and viewing. The server pushes one or more video streams to a single terminal based on the terminal subscription. When it is multiple video streams, the terminal player displays the monitoring screen in each terminal in the form of a spliced ​​window.

[0051] Specifically, the terminal subscription process is as follows: after the terminal user logs in to the platform, he completes the identity authentication through the standardized permission authentication module and enters the video stream subscription interface. The interface is designed based on the structured query language (SQL) to design the database query function. The user enters the screening conditions, such as area code, time range, monitoring device identification, etc. The system converts it into an SQL query statement, initiates a query to the server-side database, obtains a list of video streams that meet the conditions, and submits a subscription request after the user selects the required monitoring screen. The request information is encrypted by the transport layer security protocol (TLS) and transmitted to the central server 1. After receiving the request, the central server 1 performs user permission verification according to the role-based access control (RBAC) model. After the verification is passed, the subscription information is recorded and distributed to the corresponding edge server 2 using the message queue telemetry transmission (MQTT) protocol to prepare for streaming.

[0052] When multiple video streams are spliced ​​and displayed, the method is as follows: an image splicing module based on the OpenCV library can be set in the player of each terminal. This module obtains the resolution information of each video by parsing the metadata of the video stream according to the terminal screen resolution, and uses the dynamic programming algorithm to calculate the optimal splicing window layout so that each window is evenly distributed on the screen and the screen space is fully utilized. Then, the multiple video streams pulled from the edge server using the FFmpeg library are decoded and played in real time in their respective windows in a predetermined order. During the playback process, a synchronization algorithm based on timestamps is used to ensure the synchronous update of the video screens of each window to avoid freezes or tearing.

[0053] The server collects the device information and user habits of the terminal; the device information includes the video playback support format, video resolution support list and video window parameters of the terminal device. The metadata of the original video stream includes the resolution and frame rate of the video shooting. User habits include video stream selection, video window scaling, opening and closing.

[0054] Among them, the device information collection is realized through the system information detection tool built into the terminal. Taking the system based on the X86 kernel as an example, in the Windows system, the tool runs on the Windows system terminal and similar system management frameworks on other operating system terminals based on the Windows Management Instrumentation (WMI). It regularly collects information such as the video playback support format, video resolution support list, and video window parameters of the terminal device, and sends it to the server after being encrypted by the Hypertext Transfer Protocol (HTTP) secure version. For the metadata of the original video stream, when encoding the video stream, the monitoring and shooting equipment embeds the resolution and frame rate of the video shooting into the video stream header in the Extensible Markup Language (XML) format. When the server receives the video stream, it can obtain it by parsing XML. User habits are collected by embedding a behavior log recording module in the terminal player. The module is written in JavaScript language. Whenever the user selects a video stream, scales the video window, opens and closes it, it records the operation type, operation time, window size change and other detailed information in real time, and is also encrypted by HTTPS and transmitted to the server.

[0055] The server is configured with multiple types of video stream preprocessing strategies based on device information and metadata of the original video stream, wherein the multiple types of video stream preprocessing strategies include at least a low bit rate video stream preprocessing strategy and a high bit rate video stream preprocessing strategy.

[0056] Therefore, in this embodiment, the multiple video stream preprocessing strategies include:

[0057] The server compares the device information with the metadata of the original video stream, and sets the video stream transcoding parameters based on the comparison result; the video stream transcoding parameters include the video encoding format, resolution and frame rate.

[0058] Specifically, the server further divides the multiple types of video stream preprocessing strategies into at least a low bit rate video stream preprocessing strategy and a high bit rate video stream preprocessing strategy based on the video resolution support list and the video window parameters in the device information.

[0059] Furthermore, the server uses a rule-based inference engine to compare device information with metadata of the original video stream. The processing strategies are:

[0060] When the video stream resolution is compatible with the terminal player small window or terminal splicing window, the stream is pushed based on the resolution of the small window or splicing window;

[0061] When the video stream resolution is incompatible with the terminal player small window or the terminal splicing window, the low-bitrate video stream preprocessing strategy is triggered; when setting the video stream transcoding parameters, for the low-bitrate video stream, the H.264 encoding format is selected, and according to the terminal screen resolution, the original video stream resolution is reduced to the size of the terminal player small window or the terminal splicing window through the linear scaling algorithm, and the frame rate is adjusted to 30fps or below to reduce the amount of data;

[0062] When the terminal player maximizes the window or sets full-screen playback separately, and the network bandwidth is sufficient, the high-bitrate video stream preprocessing strategy is triggered. For high-bitrate video streams, the MainProfile of the H.265 encoding format is selected to increase the original video stream resolution to a size that is compatible with the terminal player's maximized window or terminal full-screen playback, and the frame rate is maintained above 30fps to ensure smooth high-definition playback.

[0063] Furthermore, with user authorization, the adaptive bitrate streaming (ABR) technology can be used to collect system resource usage. Specifically, the software on the terminal device calls the system interface to obtain the system resource usage, such as CPU and memory occupancy, and sends the obtained data to the server. After that, the server receives the system resource usage data of the terminal, evaluates the remaining performance of the terminal, and pushes the appropriate resolution video stream based on the remaining performance to enhance the playback experience.

[0064] Based on the above preset strategies, the server also performs multi-dimensional feature extraction based on user habits, and trains the machine learning model through the results of the multi-dimensional feature extraction to obtain a user habit analysis model; the input of the user habit analysis model is the result of the multi-dimensional feature extraction, the label is the user's past operations, and the target is the user's predicted operations; wherein the machine learning model is one of a decision tree model, a naive Bayes model or a support vector machine model.

[0065] In this embodiment, for multi-dimensional feature extraction, the extracted features are classified into time dimension features, space dimension features and event dimension features; the time dimension features are at least provided with the user viewing time and the user viewing duration; the space dimension features are at least provided with the location information of the original video stream and the distance between the shooting location and the user; the event dimension features are at least provided with calendar information.

[0066] Among them, the server pre-processes the user habit data through a Python-based data analysis library (such as Pandas and Numpy). When extracting time dimension features, the server uses a time series analysis algorithm to obtain the user viewing time and user viewing duration from the operation time data collected by the behavior log recording module; when extracting spatial dimension features, the server combines geographic information system (GIS) technology to query the geographic location database of the monitoring camera and the IP address location when the user logs in to calculate the location information of the original video stream and the user distance; when extracting event dimension features, the server obtains calendar information such as holidays and working days based on the calendar API interface. The extracted multi-dimensional feature data is input into the selected machine learning model (such as the decision tree model trained using the CART algorithm). After multiple iterations of training, the model performance is evaluated using cross-validation technology. When the model accuracy reaches a predetermined threshold (such as 80%), the user habit analysis model is obtained.

[0067] Therefore, the server substitutes the current multi-dimensional feature extraction results into the user habit analysis model to obtain the user's predicted operation in the next time period. For example, it is predicted that the user will log in to the monitoring platform at 8:15 in the morning, and watch the video of monitoring channel A in full screen on the large-screen terminal 4 during the time period of 8:20-8:25, and continue to watch it for 30 minutes.

[0068] The server selects a video stream preprocessing strategy based on the user's predicted operation in the next time period, and pre-pushes the pre-processed video stream to the corresponding terminal based on the resolution adaptive algorithm; the pre-pull pre-processed video stream is distributed and cached in the edge server 2.

[0069] The server predicts the user's operation. If it is predicted that the user will switch to a small window to watch or a multi-channel video stream splicing window mode, the server selects a low-bitrate video stream preprocessing strategy. Otherwise, based on the prediction result, a high-bitrate video stream preprocessing strategy is selected for monitoring channel A. In the process of pre-pull streams to the corresponding terminal based on the resolution adaptive algorithm, the HTTP-based dynamic adaptive streaming (DASH) technology is used to adjust the stream pull rate in real time according to the terminal network status to ensure stable transmission of the video stream. After receiving the stream pull request, the edge server 2 uses the local cache mechanism. If there is a pre-processed video stream that meets the requirements in the cache, it will be returned directly; if not, it will request transcoding processing from the central server 1 and cache the results. The cache replacement strategy uses the least recently used (LRU) algorithm to ensure the effective use of cache space.

[0070] The server also establishes a feedback connection with each terminal, and a black-edge removal algorithm is configured in the server; when the video screen ratio of the server push stream does not match the screen ratio of the terminal player, such as the user does not maximize the window or play it in full screen, but only manually adjusts the size of a certain video stream playback window; thus, the terminal where the terminal player is located feeds back at least one frame of the video playback screen with black edges to the server; the server crops and scales the pushed video stream based on the black-edge removal algorithm and the video playback screen with black edges.

[0071] Among them, since the surveillance shooting picture is different from the conventional video picture, its shooting angle is relatively fixed, so the content presented in the picture is also basically fixed. If the conventional black edge removal algorithm is used, it is easy to cause the key area to be cropped, thereby affecting the viewing experience. In this embodiment, an optimization method is also proposed. First, based on a typical day scene, multiple typical images in each video stream are selected, and when the server is idle, the system resources are used to perform identification density analysis on the typical images, including the following steps:

[0072] First, computer vision technology, such as target detection algorithms (such as YOLO, FasterR-CNN, etc.), is used to extract features from typical images of surveillance videos, and identify target objects such as vehicles, people, billboards, and intersections, and then add a unique identifier to each target; then, a two-dimensional information identifier map is constructed: according to the location information of the target object, an information identifier map is constructed on the typical image. The typical image is divided into multiple grids, each grid corresponds to a location coordinate, and based on the location information of the target object, the target identifier is recorded in the corresponding grid.

[0073] Afterwards, a clustering algorithm (such as DBSCAN, K-Means, etc.) is used to perform cluster analysis on the information identification two-dimensional map. By setting appropriate clustering parameters, similar targets are clustered together to form information-intensive areas. These areas usually contain key information in the monitoring scene, such as busy intersections, densely populated areas, etc. Specifically, in this embodiment, for DBSCAN (Density-Based Spatial Clustering of Applications with Noise, density-based spatial clustering algorithm): it divides clusters according to the density of data points. In the information identification two-dimensional map, the data point corresponding to each grid is the feature vector of the target object (including position, size, color and other information). The algorithm first determines a core point, that is, the number of neighbor points contained in the point in its neighborhood (a circular area with a certain distance threshold ε as a radius) exceeds the preset minimum number of points MinPts. These core points and their density-connected neighbor points constitute a cluster cluster, and multiple clusters are identified by continuously iterating this process. For example, in the surveillance footage, if the area where people gather is taken as the research object, places with high population density (such as shopping mall entrances and exits, event sites, etc.) meet the core point conditions of the DBSCAN algorithm, and the grid points corresponding to the people in their neighborhood are classified into the same cluster, intuitively presenting the scope of the densely populated area on the two-dimensional graph, and the boundary is determined by the point where the density drops sharply.

[0074] For data points in low-density areas, if the number of their neighbor points is less than MinPts, they are marked as noise points and do not participate in clustering. This also helps to clearly outline the true boundaries of information-intensive areas and eliminate the interference of some isolated or misjudged targets in scope definition. The information-intensive area has clear boundaries and a fixed area on the two-dimensional map, forming a corresponding information domain.

[0075] Thus, a two-dimensional information domain map containing information density based on typical images in typical day scenes is obtained. After the information domain two-dimensional map is stored, it can be used continuously and can be called up in the same typical day scene even when the monitoring shooting angle is inconvenient, without repeated calculation, thus avoiding occupying the resources of the central server 1.

[0076] Furthermore, the black edge removal algorithm can be optimized based on the above information domain two-dimensional graph. Specifically, the black edge removal algorithm includes Canny edge detection and Hough transform line detection;

[0077] When the player window ratio of the terminal changes, the server performs Canny edge detection on the feedback video playback screen with black edges, uses the Canny function in the OpenCV library, sets appropriate threshold parameters (such as a low threshold of 50 and a high threshold of 150), and obtains the edge image of the video frame;

[0078] The server performs Hough transform straight line detection on the edge image of the video frame, and also uses the HoughLines function in the OpenCV library to obtain the black edge range.

[0079] Furthermore, the screen ratio is recalculated according to the black border range, and the maximum display area of ​​the image is drawn based on the screen ratio, that is, the retained area after cropping occupies the largest proportion of the image. This step can be performed through automated image processing software, and rules are added: take the cropping strategy with the largest information domain area in the retained area after cropping, and re-stream based on this strategy, so that the video stream can fill the entire window, so that it no longer has black borders, and retains important information in the surveillance video as much as possible.

[0080] Finally, it should be noted that the above content is only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Simple modifications or equivalent substitutions of the technical solution of the present invention by ordinary technicians in this field do not deviate from the essence and scope of the technical solution of the present invention.

Claims

1. A method for optimizing adaptive playback support of a video surveillance platform, characterized in that: The steps include: The server collects terminal device information and user habits; The server is configured with multiple types of video stream preprocessing strategies based on the device information and metadata of the original video stream; The multi-type video stream preprocessing strategy at least includes a low bit rate video stream preprocessing strategy and a high bit rate video stream preprocessing strategy; The server extracts multidimensional features based on the user habits, trains a machine learning model through the results of the multidimensional feature extraction, and obtains a user habit analysis model; the input of the user habit analysis model is the result of the multidimensional feature extraction, the label is the user's past operation, and the target is the user's predicted operation; In the multi-dimensional feature extraction, the extracted features are classified into time dimension features, space dimension features and event dimension features; the time dimension features are at least provided with the user viewing time and the user viewing duration; the space dimension features are at least provided with the location information of the original video stream and the distance between the shooting location and the user; the event dimension features are at least provided with calendar information; The server obtains the user's predicted operation in the next period by substituting the result of the current multi-dimensional feature extraction into the user habit analysis model; The server selects a video stream preprocessing strategy based on the user predicted operation in the next time period, and pre-pushes the preprocessed video stream to the corresponding terminal based on a resolution adaptive algorithm; The server also establishes a feedback connection with each terminal, and a black edge removal algorithm is configured in the server; When the video screen ratio of the server pushes the video stream does not match the screen ratio of the terminal player, the terminal where the terminal player is located feeds back at least one frame of the video playback screen with black edges to the server; the server crops and scales the pushed video stream based on the black edge removal algorithm and the video playback screen with black edges.

2. The optimization method for adaptive playback support of a video surveillance platform according to claim 1, characterized in that: The black edge removal algorithm includes Canny edge detection and Hough transform straight line detection; When the player window ratio of the terminal changes, the server performs Canny edge detection on the fed-back video playback screen with black edges to obtain an edge image of the video frame; The server performs Hough transform straight line detection on the edge image of the video frame to obtain a black edge range; The server processes the pushed video stream by a cropping method or a padding method based on the black border range.

3. The optimization method for adaptive playback support of a video surveillance platform according to claim 1, characterized in that: The device information includes the video playback support format, video resolution support list and video window parameters of the terminal device.

4. The optimization method for adaptive playback support of a video surveillance platform according to claim 3, characterized in that: The metadata of the original video stream includes the resolution and frame rate of the video.

5. The optimization method for adaptive playback support of a video surveillance platform according to claim 4, characterized in that: The multi-class video stream preprocessing strategy includes: The server compares the device information with metadata of the original video stream, and sets video stream transcoding parameters based on the comparison result; The video stream transcoding parameters include video encoding format, resolution and frame rate; The server also divides the multiple types of video stream preprocessing strategies into at least the low bit rate video stream preprocessing strategy and the high bit rate video stream preprocessing strategy based on the video resolution support list and the video window parameter in the device information.

6. The optimization method for adaptive playback support of a video surveillance platform according to claim 5, characterized in that: The multi-type video stream preprocessing strategy also includes video cropping and scaling.

7. The method for optimizing adaptive playback support of a video surveillance platform according to claim 5 or 6, characterized in that: The multi-type video stream preprocessing strategy also includes: in the low-bitrate video stream preprocessing strategy, the video stream resolution is adapted to the terminal player small window or the terminal splicing window; In the high-bitrate video stream preprocessing strategy, the video stream resolution is adapted to the maximized window of the terminal player or the full-screen playback of the terminal.

8. The method for optimizing adaptive playback support of a video surveillance platform according to claim 1, characterized in that: The server includes an algorithm server and an edge server; The video stream that has been pre-streamed and pre-processed is distributed and cached in the edge server.

9. The method for optimizing adaptive playback support of a video surveillance platform according to claim 1, characterized in that: The machine learning model is a decision tree model, a naive Bayes model or a support vector machine model.

10. The method for optimizing adaptive playback support of a video surveillance platform according to claim 1, characterized in that: The user habits include video stream selection, video window scaling, opening and closing.

Images