Video information processing method based on video information processing model

Abstract

The invention relates to the technical field of video processing, in particular to a video information processing method based on a video information processing model, which comprises the following steps of: firstly, providing the video information processing model, and obtaining preset target information in a single-area monitoring video by combining a single-frame detection result and inter-frame target motion information; the video information processing speed is further accelerated; then, the next action region of the target is determined by judging the action track of the preset target, so that cross-region monitoring video switching based on target information is realized, and information processing on monitoring videos of irrelevant regions is further reduced; and finally, the cross-region target video set is compressed by combining the foreground information and background information similarity coefficients of the adjacent frames, so that a cross-region compressed video containing a preset target action track is obtained, and the efficiency of multi-region synchronous monitoring video information processing is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of video processing, and in particular to a video information processing method based on a video information processing model. Background Art

[0002] As an important part of the modern security system, surveillance video has been widely used in many fields such as urban management, public safety, traffic monitoring, and commercial operations. By deploying high-definition smart cameras, real-time monitoring and event recording can be achieved, effectively improving security prevention and emergency response capabilities. However, under the long-term recording of high-definition cameras, the amount of surveillance video data has increased sharply. How to efficiently process this huge amount of surveillance data has become a key challenge that needs to be solved urgently. As a result, video information processing technology has emerged. This technology can perform intelligent analysis on surveillance video content, extract key information, support application scenarios such as intelligent monitoring and security warning, enhance the practicality and value of surveillance video data, and thus greatly reduce the burden of manual monitoring.

[0003] At present, the existing technology often processes key information of surveillance videos in a single area. When applied to multi-area synchronous surveillance videos in scenes such as shopping malls, office buildings, hospitals and traffic sections, redundant information processing will be caused, thereby reducing the efficiency of surveillance video information processing.

[0004] In order to solve the above problems, the present invention proposes a video information processing method based on a video information processing model. Summary of the invention

[0005] The purpose of the present invention is to provide a video information processing method based on a video information processing model, which is used for target information processing of multi-region synchronous surveillance videos, and improves the efficiency of surveillance video information processing. In order to solve the problems existing in the prior art, the present invention introduces a transfer learning training target detection module to divide the first region surveillance video and the non-first region video set. The module performs target detection on the video frames in chronological order to obtain the first region surveillance video containing the preset target, thereby reducing the processing of redundant video information; secondly, a video information processing model is constructed based on the target detection module, and the preset target information in the video is obtained by combining the single-frame detection result and the frame difference, further accelerating the speed of model information processing; then, the next region of the target action is determined by judging the action trajectory of the preset target, realizing cross-region video switching based on target information, further reducing the information processing of irrelevant videos; finally, the similarity coefficients of foreground information and background information are calculated respectively, so as to compress the target videos of different regions, obtain cross-region compressed videos, and further realize cross-region information processing of multi-region synchronous surveillance videos.

[0006] A video information processing method based on a video information processing model, the specific implementation steps include:

[0007] Obtain surveillance videos of n areas in a preset scene in the same time period, and perform preprocessing operations on the surveillance videos of all areas to obtain standard surveillance videos; wherein n represents the number of cameras in the preset scene;

[0008] Using the target detection module, according to the image of the preset target and the standard surveillance video, determine the first area surveillance video and the non-first area video set; wherein the non-first area video set is a set of n-1 area surveillance videos except the first area surveillance video;

[0009] Based on the target detection module, a video information processing model is constructed to perform target information processing on the surveillance video of the first area, and a target video and target disappearance time are obtained; wherein the target video is a video that has been processed with target information and contains a target frame;

[0010] According to the disappearance time of the target and the non-first area video set, the movement trajectory of the preset target is judged, so as to determine the next movement area of ​​the preset target and update the first area surveillance video and the non-first area video set, and further integrate the target videos of different areas in chronological order to obtain a cross-regional target video set;

[0011] The cross-region target video set is compressed according to the foreground information and background information of the video frame, so as to obtain the cross-region compressed video of the preset target.

[0012] Preferably, the specific implementation process of the preprocessing includes:

[0013] Use filters to remove noise interference from surveillance videos in different areas to obtain denoised surveillance videos;

[0014] The video frames of the denoised surveillance video are enhanced by an image enhancement algorithm to obtain a clear surveillance video; wherein the image enhancement algorithm includes: low light enhancement, rain removal, fog removal and blur removal algorithms;

[0015] A grayscale operation is performed on each video frame of the clear surveillance video, and the pixel values ​​of all video frame images are restricted to [0, 1], so as to obtain a standard surveillance video.

[0016] Preferably, the specific implementation process of the target detection module includes:

[0017] Training phase: First, sample images of the preset target class are collected to construct a target data set, and the preset target samples are used as positive samples, and samples other than the preset targets are used as negative samples; then, a target detection model pre-trained on an open source data set is obtained; then, data enhancement operations are performed on the target data set to obtain a target expansion data set; finally, a transfer learning method is introduced, and the pre-trained target detection model is used to train on the target expansion data set, and the model weights are fine-tuned through the cross entropy loss function and the DIOU loss function to obtain a trained target detection model; wherein, the open source data sets include: Image Net, COCO and Pascal VOC data sets; the data enhancement operations include: rotation, flipping and translation;

[0018] Inference stage: According to the trained target detection model, the standard surveillance video is detected frame by frame, and the video in which the preset target is first detected is regarded as the first area surveillance video.

[0019] Preferably, the implementation process of the reasoning stage includes:

[0020] According to the trained target detection model, the video frames of the standard surveillance video in different areas at the first time point are detected one by one to determine whether the preset target exists in the video frame of the first time; if the preset target does not exist, the next video frame of the first time is updated to the video frame of the first time for re-detection and judgment operations; if the preset target exists, the video frame with the highest target confidence is regarded as the first action position of the preset target, the standard surveillance video of the corresponding area is used as the first area surveillance video, the standard surveillance videos of n-1 areas except the first area surveillance video are integrated into a non-first area video set, and the target detection operation is stopped.

[0021] Preferably, the video information processing model includes:

[0022] In the first stage, the information processing result of the first video frame is obtained by using the trained target detection model; the difference map of the first video frame and the second video frame is calculated, and the information processing result of the first video frame and the difference map are respectively input into the feature extraction module to obtain a first information feature map and a difference feature map respectively; the cosine similarity of the first video frame and the second video frame is calculated, and a weight is generated according to the similarity to fuse the first information feature map and the difference feature map to obtain a second video fusion feature; wherein, the first video frame is the first video frame in the first area monitoring video that contains the preset target; the second video frame is an unprocessed video frame in the first area monitoring video; the specific calculation formula of the second video fusion feature is:

[0023]

[0024] F=(1 c -cos)⊙F dif +F dec ;

[0025] Among them, cos i A represents the cosine similarity of the i-th channel; i and B i Represent the image vectors of the i-th channel of the first video frame and the second video frame respectively; · represents the dot product operation of the vector; |||| represents the operation of taking the modulus of the vector; F represents the second video fusion feature; 1 c represents a vector with dimension c; cos represents the cosine similarity; F dif and F dec denote the difference feature map and the first information feature map respectively; ⊙ denotes a channel multiplication operation;

[0026] In the second stage, the set of processed video frames in the first area surveillance video is regarded as the first area processed video set; the second video fusion features of each video frame in the first area processed video set and the second video frame are calculated respectively, and the learning weights are used to combine them to obtain the second video dense fusion features. The specific formula is expressed as follows:

[0027]

[0028] in, represents the second video dense fusion feature of the kth video frame; w j represents the learning weight; F j represents the second fusion feature of the jth video frame; note that k ≥ 2 and

[0029] Then, the dense fusion features of the second video are input into the fully connected layer to obtain the information processing result of the second video frame; further, the next frame of the second video frame is updated to the second video frame to repeat the target information processing operation to obtain the target video of the first area monitoring video;

[0030] When the preset target is not in the second video frame, determine whether the preset target disappears according to the target frame coordinates in the previous frame of the second video frame and the size of the video frame; if the target frame area is smaller than the predetermined disappearance threshold or any edge of the target frame coincides with the edge of the corresponding video frame, it means that the preset target is about to leave the shooting range of the first area monitoring video, so the time point corresponding to the second video frame is the target disappearance time; otherwise, update the next frame of the second video frame to the second video frame, and continue to process the video information.

[0031] Preferably, the process of determining the action trajectory of the preset target includes:

[0032] The m-th frame video of the surveillance video of different regions in the non-first region video set after the target disappearance time is intercepted to obtain a cross-region matching video set; the semantic difference between the n-1 region matching video frames and the corresponding m-th frame video before the target disappearance time is calculated respectively, and the specific calculation formula is:

[0033]

[0034] in, represents semantic difference; VGG() represents the pre-trained VGG network; X t-m represents the mth frame of video image before the target disappears; X t+m represents the mth frame of video image after the target disappears; || ||2 represents the two-norm function;

[0035] If the semantic difference is less than a predetermined semantic threshold, it means that the preset target is not in the surveillance video of the corresponding area; if the semantic difference is greater than or equal to the predetermined semantic threshold, the target detection module is used to detect the matching video frame of the corresponding area; if the semantic difference corresponding to the cross-region matching video set is less than the predetermined semantic threshold, it means that the preset target has left the preset scene, and the target videos in different areas are integrated into the cross-region target video set in chronological order;

[0036] Further, based on the detection results of the matching video frames in different areas, the surveillance video of the next action area of ​​the preset target is determined, and the video information is processed; if the preset target is only detected in one matching video frame, the surveillance video of the corresponding area after the disappearance time of the target is obtained, and it is updated as the surveillance video of the first area, and the video set of the non-first area is updated at the same time; if the preset target exists in the matching video frames in more than one area, the area where the matching video frame with the highest target confidence is located is determined, and the surveillance video of the area after the disappearance time of the target is updated to the surveillance video of the first area, and the video set of the non-first area is updated at the same time; otherwise, it means that the preset target has left the preset scene, and the target videos of different areas are integrated into the cross-regional target video set in chronological order.

[0037] Preferably, the implementation process of the compression processing includes:

[0038] The cross-regional target video set is preliminarily compressed according to the frame rate of the surveillance video to obtain a first compressed video; the first compressed video is decomposed into a foreground video and a background video using a target frame, and the foreground similarity coefficient and the background similarity coefficient of adjacent video frames are respectively calculated by a structural similarity index, thereby obtaining the adjacent frame similarity coefficient of the first compressed video; wherein, the foreground video sets the background pixel value to 0; and the background video sets the foreground pixel value to 0; the specific calculation formula of the adjacent frame similarity coefficient is:

[0039]

[0040] here, represents the similarity coefficient between the jth and j+1th frame video images; SSIM() represents the structural similarity function; Respectively represent the j-th and j+1-th frame video images of the foreground video; denote the jth and j+1th frame video images of the background video respectively; α, β denote adjustment parameters of the foreground similarity coefficient and the background similarity coefficient respectively, and α>β>0;

[0041] According to a predetermined inter-frame threshold, determining whether to perform secondary compression processing; if the adjacent frame similarity coefficient is greater than the predetermined inter-frame threshold, removing the j-th frame of video; otherwise, not removing the video frame; repeatedly performing secondary compression processing on the first compressed video until the adjacent frame similarity coefficients of all retained video frames are greater than the predetermined inter-frame threshold, thereby obtaining a second compressed video;

[0042] The maximum width and height of the target frame in the second compressed video are further obtained, and the second compressed video is further cropped and corrected. The specific calculation formula is:

[0043]

[0044] w1=(1+γ1)W, h1=(1+γ2)H;

[0045] Wherein, x1, y1 represent the horizontal and vertical coordinates of the upper left corner of the target area to be cropped respectively; w1, h1 represent the width and height of the target area to be cropped respectively; [x1, y1, w1, h1] represent the coordinates of the target area to be cropped; x, y represent the horizontal and vertical coordinates of the upper left corner of the target frame respectively; h, w represent the height and width of the target frame respectively; [x, y, w, h] represent the coordinates of the target frame; H, E represent the maximum height and width of the target frame in the second compressed video respectively; γ1, γ2 represent the adjustment parameters of cropping respectively;

[0046] The second compressed video is cropped according to the coordinates of the target area to be cropped to obtain a cross-region compressed video.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] 1. The present invention constructs a video information processing model based on a trained target detection model, combining single-frame detection results and inter-frame difference information. The model calculates the difference map between video frames and combines the inter-frame similarity coefficient and the single video frame detection results to obtain the processing result of the next video frame. It can quickly obtain the preset target information in the single-region surveillance video, greatly improving the speed of single-region surveillance video information processing, thereby improving the efficiency of overall surveillance video information processing.

[0049] 2. The present invention determines the target action trajectory based on preset target information to determine the next area of ​​the target action. The method first calculates the semantic difference between each video frame in the non-first area video set before and after the target disappears, quickly filters out the area where the preset target does not exist, and effectively reduces the scope of target information processing; at the same time, it combines the target detection module to accurately determine whether the preset target exists in a certain area monitoring range, and determines the optimal monitoring video of the target's cross-regional action based on the target confidence, thereby avoiding invalid processing in the monitoring video area where the target does not appear or has left, and further improving the efficiency of the overall monitoring video processing.

[0050] 3. The present invention designs a double compression process based on the efficiently acquired cross-regional surveillance video set. The number of target video frames is greatly reduced through the initial compression process, and the target video frames are further decomposed into foreground information (preset target part) and background information (non-preset target part). The similarity coefficients of the foreground and background are calculated respectively to make a secondary compression decision, so that the compression process is more accurate and can retain more behavioral details of the preset targets, while reducing the retention of redundant information, thereby improving the efficiency of subsequent cross-regional behavior judgment of the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A flowchart of a video information processing method based on a video information processing model provided by an embodiment of the present invention;

[0052] Figure 2 A structural diagram of a video information processing module provided by an embodiment of the present invention;

[0053] Figure 3 A flow chart of target trajectory determination provided by an embodiment of the present invention;

[0054] Figure 4 A flowchart of a target video compression process provided by an embodiment of the present invention;

[0055] Figure 5 This is an example diagram of a road section surveillance video provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] With the rapid development of science and technology and the progress of society, surveillance video has become an indispensable part of modern society. However, the amount of surveillance video data is huge and complex, which poses a huge challenge to storage and transmission, and directly affects the efficient management, transmission, analysis and utilization of video data. Therefore, surveillance video information processing technology is particularly important. By efficiently analyzing surveillance video data, the practicality and value of video data can be significantly improved, and a more accurate and comprehensive monitoring field of view can be provided for monitoring personnel, thereby improving monitoring efficiency.

[0058] The present invention proposes a video information processing method based on a video information processing model, which performs joint information processing on a plurality of surveillance videos of a preset scene according to preset target information, so as to improve the efficiency of video information processing. In order to illustrate the effectiveness of the method of the present invention, the present invention will be described in detail in combination with the accompanying drawings of the embodiments and the following specific implementation details.

[0059] Embodiment 1

[0060] The embodiment of the present application discloses a video information processing method based on a video information processing model to realize information processing of a preset target on a surveillance video in a shopping mall, see Figure 1 The specific steps of the method proposed in the present invention include: S1. Acquire surveillance videos of different areas in the same time period and perform preprocessing; S2. Use the target detection module to divide the first area surveillance video and the non-first area video set; S3. Use the video information processing model to obtain the target video and the target disappearance time; S4. Calculate the semantic difference and judge the target movement trajectory. If the target leaves the preset scene, generate a cross-region target video set; otherwise, determine the next area of ​​the target movement; S5. Compress the cross-region target video set.

[0061] Furthermore, surveillance videos of n areas in a shopping mall in the same time period are obtained, and the surveillance videos of different areas are preprocessed to obtain standard surveillance videos, corresponding to the above step S1; wherein n represents the number of cameras in the shopping mall; the specific implementation process of the preprocessing operation is:

[0062] Use filters to remove noise interference from surveillance videos in different areas to obtain denoised surveillance videos;

[0063] The video frames of the denoised surveillance video are enhanced by an image enhancement algorithm to obtain a clear surveillance video; wherein the image enhancement algorithm includes: low light enhancement, spot removal, defogging and deblurring algorithms;

[0064] A grayscale operation is performed on each video frame of the clear surveillance video, and the pixel values ​​of all video frame images are restricted to [0, 1], so as to obtain a standard surveillance video.

[0065] The embodiment of the present application performs preprocessing operations on the acquired synchronous surveillance videos of multiple areas in a shopping mall, which is very important for improving the video quality and the efficiency of subsequent information processing. Among them, removing noise interference in the video through a filter can significantly improve the clarity of the video, reduce the visual interference caused by noise, and make the subsequent processing more accurate; using low-light enhancement, defogging, despeckle and deblurring algorithms to enhance the surveillance video frames can effectively improve the visual effects of the video under different lighting conditions, temperature changes and camera delays, and help to quickly analyze and process the preset target information; graying the surveillance video frames can greatly reduce the complexity of the video data and reduce the amount of calculation for subsequent processing, thereby improving the efficiency of subsequent surveillance video information processing.

[0066] Further, the target detection module is used to determine the first area surveillance video and the non-first area video set according to the image of the preset target and the standard surveillance video, corresponding to the above step S2; wherein the non-first area video set is a set of n-1 area surveillance videos except the first area surveillance video; the specific implementation process of the target detection module in the training stage includes:

[0067] First, sample images of the preset target class are collected to construct a target data set, the preset target samples are used as positive samples, and samples other than the preset targets are used as negative samples;

[0068] Next, obtain an object detection model pre-trained on an open source dataset; wherein the open source datasets include: Image Net, COCO, and Pascal VOC datasets;

[0069] Then, a data enhancement operation is performed on the target data set to obtain a target expanded data set; wherein the data enhancement operation includes: rotation, flipping and translation;

[0070] Finally, a transfer learning method is introduced, and the pre-trained target detection model is used to train on the target expansion dataset. The model weights are fine-tuned through the cross entropy loss function and the DIOU loss function to obtain a trained target detection model.

[0071] The embodiment of the present application introduces a transfer learning method to train the target detection model based on a pre-trained model. This method is based on the target detection model pre-trained with an open source data set, uses a data enhancement strategy to balance the number of positive and negative samples in the preset target data set, and supervises and trains the model through the cross entropy loss function and the DIOU loss function, which significantly shortens the model training time, effectively enhances the model's generalization ability for complex scenes, and further improves the efficiency of subsequent information processing.

[0072] Furthermore, in the model inference stage, the standard surveillance video is detected frame by frame according to the trained target detection model, and the video in which the preset target is first detected is regarded as the first area surveillance video. The specific implementation process includes:

[0073] According to the trained target detection model, the video frames of the standard surveillance video in different areas at the first time point are detected one by one to determine whether the preset target exists in the video frame of the first time; if the preset target does not exist, the next video frame of the first time is updated to the video frame of the first time for re-detection and judgment operations; if the preset target exists, the video frame with the highest target confidence is regarded as the first action position of the preset target, the standard surveillance video of the corresponding area is used as the first area surveillance video, the standard surveillance videos of n-1 areas except the first area surveillance video are integrated into a non-first area video set, and the target detection operation is stopped.

[0074] The embodiment of the present application uses a trained target detection model to efficiently detect standard surveillance videos frame by frame, and can quickly locate and identify the surveillance area where the preset target first exists. This process not only ensures the accuracy of the detection results through confidence judgment, but also stops the detection of subsequent videos in a timely manner, effectively saving computing resources, improving the efficiency of information processing, and providing timely and accurate data support for subsequent single-area surveillance video processing and analysis.

[0075] Further, according to the target detection module, a video information processing model is constructed to perform target information processing on the first area surveillance video to obtain a target video, corresponding to the above-mentioned step S3; wherein the target video is a video that has been processed with target information and contains a target frame; the implementation process of the video information processing model includes:

[0076] See also Figure 2In the first stage, the information processing result of the first video frame is obtained by using the trained target detection model; the difference map of the first video frame and the second video frame is calculated, and the information processing result of the first video frame and the difference map are respectively input into the feature extraction module to obtain the first information feature map and the difference feature map respectively; the cosine similarity of the first video frame and the second video frame is calculated, and the weight is generated according to the similarity to fuse the first information feature map and the difference feature map to obtain the second video fusion feature; wherein, the first video frame is the first video frame in the first area monitoring video that contains the preset target; the second video frame is the unprocessed video frame in the first area monitoring video; the specific calculation formula of the second video fusion feature is:

[0077]

[0078] F=(1 c -cos)⊙F dif +F dec ;

[0079] Among them, cos i A represents the cosine similarity of the i-th channel; i and B i Represent the image vectors of the i-th channel of the first video frame and the second video frame respectively; · represents the dot product operation of the vector; |||| represents the operation of taking the modulus of the vector; F represents the second video fusion feature; 1 c represents a vector with dimension c; cos represents the cosine similarity; F dif and F dec denote the difference feature map and the first information feature map respectively; ⊙ denotes a channel multiplication operation;

[0080] See also Figure 2 In the second stage, the set of processed video frames in the first area surveillance video is regarded as the first area processed video set; the second video fusion features of each video frame in the first area processed video set and the second video frame are calculated respectively, and the learning weights are used to combine them to obtain the second video dense fusion features. The specific formula is expressed as follows:

[0081]

[0082] in, represents the second video dense fusion feature of the kth video frame; w j represents the learning weight; F j represents the second fusion feature of the jth video frame; note that k ≥ 2 and

[0083] Then, the dense fusion features of the second video are input into the fully connected layer to obtain the information processing result of the second video frame; further, the next frame video image of the second video frame is updated to the second video frame to repeat the target information processing operation to obtain the target video of the first area monitoring video.

[0084] The embodiment of the present application proposes a video information processing model, which can make full use of the prior information of the first video frame and the difference information between video frames based on the trained target detection model and inter-frame motion information, and adjust the fusion features by calculating the cosine similarity and generating weights, thereby effectively improving the continuity and accuracy of the inter-frame information. At the same time, the correlation between the second video frame and the processed video frame is fused using the learned weights, further enhancing the richness and robustness of the target feature representation. Compared with the analysis and processing of a single frame one by one, this information processing process not only improves the accuracy of target information processing, but also speeds up the processing speed of single-area surveillance video information.

[0085] Further, according to the information processing result of the second video frame, it is judged whether the target leaves the first area monitoring video, so as to obtain the target disappearance time, and the specific implementation process is:

[0086] When the preset target is not in the second video frame, determine whether the preset target disappears according to the target frame coordinates in the previous frame of the second video frame and the size of the video frame; if the target frame area is smaller than the predetermined disappearance threshold or any edge of the target frame coincides with the edge of the corresponding video frame, it means that the preset target is about to leave the shooting range of the first area monitoring video, so the time point corresponding to the second video frame is the target disappearance time; otherwise, update the next frame of the second video frame to the second video frame, and continue to process the video information.

[0087] The embodiment of the present application can accurately and timely determine when the preset target leaves the current monitoring area, that is, determine the target disappearance time. According to the relative position of the target frame and the edge of the video frame in the previous frame of the second video frame in which there is no preset target in the monitoring area, as well as the size of the target frame, it is possible to effectively identify whether the target is about to leave the current monitoring area, thereby timely recording the time when the target disappears, which not only enhances the accuracy and real-time nature of monitoring the dynamic changes of the target in the video, but also improves the efficiency of video information processing.

[0088] Further, according to the disappearance time of the target and the non-first area video set, the movement trajectory of the preset target is determined, so as to determine the next movement area of ​​the preset target and update the first area surveillance video and the non-first area video set, and further integrate the target videos of different areas into a cross-region target video set in chronological order, corresponding to Figure 1 S4 step of Figure 3 The process of determining the preset target action trajectory includes:

[0089] The m-th frame video of the surveillance video of different regions in the non-first region video set after the target disappearance time is intercepted to obtain a cross-region matching video set; the semantic difference between the n-1 region matching video frames and the corresponding m-th frame video before the target disappearance time is calculated respectively, and the specific calculation formula is:

[0090]

[0091] in, represents semantic difference; VGG() represents the pre-trained VGG network; X t-m represents the mth frame of video image before the target disappears; X t+m represents the mth frame of video image after the target disappears; || ||2 represents the two-norm function;

[0092] It is worth noting that the value of m is highly correlated with the overlap of the shooting areas in a shopping mall and the distance between cameras. For example, the surveillance video of camera A in the shopping mall is the surveillance video of the first area, and the set of surveillance videos of cameras B and C is not the video set of the first area. When the preset target leaves the monitoring range of camera A, the value of m is determined according to the overlap of the shooting areas of cameras B and C with camera A. Case 1: The shooting range of camera A The shooting range of camera B There is spatial overlap. So represents the area of ​​the overlapping part of the two shots, then The larger the value, the more preset targets can appear at the same time. and Therefore, m should be larger to distinguish The semantic difference between the presence or absence of a preset target; Case 2: The shooting range of camera A Shooting range of camera C If there is no spatial overlap, the closer the distance between camera A and camera C is, the closer the preset target is. arrive The shorter the time taken, the smaller m should be to prevent the preset target from leaving the shooting range of camera C within m frames;

[0093] If the semantic difference is less than a predetermined semantic threshold, it means that the preset target is not in the surveillance video of the corresponding area; if the semantic difference is greater than or equal to the predetermined semantic threshold, the target detection module is used to detect the matching video frames of the corresponding area; if the semantic differences corresponding to the cross-regional matching video set are all less than the predetermined semantic threshold, it means that the preset target has left the preset scene, and the target videos in different areas are integrated into the cross-regional target video set in chronological order.

[0094] The embodiment of the present application combines the semantic difference calculation and the target detection module to make a judgment on the movement trajectory of the preset target. By calculating the semantic difference of the video frames before and after the target disappears, and setting a predetermined threshold for screening, the corresponding areas of the monitoring video frames that are irrelevant to the target or have little change can be quickly excluded, reducing unnecessary calculations. At the same time, for cross-region matching video frames with significant semantic differences, the target detection module is used for detailed analysis; and when the semantic difference of all cross-region matching video frames is less than the predetermined threshold, it is directly judged that the preset target has left the preset scene, and the cross-region target video set is integrated. By selectively switching surveillance videos in different areas, the efficiency of multi-region synchronous surveillance videos in processing cross-region target information is improved.

[0095] Furthermore, according to the detection results of the matching video frames in different areas, the monitoring video of the next action area of ​​the preset target is determined, and video information processing is performed on it. The specific implementation process includes:

[0096] If the preset target is detected in only one matching video frame, the surveillance video of the corresponding area after the disappearance time of the target is obtained, and it is updated as the surveillance video of the first area, and the video set of the non-first area is updated at the same time; if the preset target exists in the matching video frames of more than one area, the area where the matching video frame with the highest target confidence is located is determined, and the surveillance video of the area after the disappearance time of the target is updated to the surveillance video of the first area, and the video set of the non-first area is updated at the same time; otherwise, it means that the preset target has left the preset scene, and the target videos of different areas are integrated into the cross-regional target video set in chronological order.

[0097] The embodiment of the present application realizes a secondary judgment on the movement trajectory of the preset target by performing a detailed analysis on the detection results of the video frames to be matched. By distinguishing different situations in which the target is detected, various complex scenarios can be flexibly responded to, such as the detection of the target in a single-area surveillance video frame, the detection of the target in a multi-area surveillance video frame, and the failure to detect the target. For situations in which the target is detected in a single-area or multi-area surveillance video frame, the optimal surveillance area is selected for subsequent processing by comparing the target confidence. At the same time, when the matching video frames in different areas do not detect the target, it is directly judged that the target has left the preset scene, and a cross-regional target video set is obtained by integration, thereby improving the efficiency of multi-area synchronous surveillance video in processing target information.

[0098] Further, the cross-region target video set is compressed according to the foreground information and background information of the video frame, so as to obtain the second compressed video of the preset target, corresponding to Figure 1 S5 step of Figure 4 The implementation process of the compression processing includes:

[0099] The cross-regional target video set is preliminarily compressed according to the frame rate of the surveillance video to obtain a first compressed video; the first compressed video is decomposed into a foreground video and a background video using a target frame, and the foreground similarity coefficient and the background similarity coefficient of adjacent video frames are respectively calculated by a structural similarity index, thereby obtaining the adjacent frame similarity coefficient of the first compressed video; wherein, the foreground video sets the background pixel value to 0; and the background video sets the foreground pixel value to 0; the specific calculation formula of the adjacent frame similarity coefficient is:

[0100]

[0101] here, represents the similarity coefficient between the jth and j+1th frame video images; SSIM() represents the structural similarity function; Respectively represent the j-th and j+1-th frame video images of the foreground video; denote the jth and j+1th frame video images of the background video respectively; α, β denote adjustment parameters of the foreground similarity coefficient and the background similarity coefficient respectively, and α>β>0;

[0102] According to the predetermined inter-frame threshold, determine whether to perform secondary compression processing; if the adjacent frame similarity coefficient is greater than the predetermined inter-frame threshold, remove the j-th frame video; otherwise, do not remove the video frame; repeat the secondary compression processing on the first compressed video until the adjacent frame similarity coefficients of all retained video frames are greater than the predetermined inter-frame threshold, thereby obtaining a second compressed video.

[0103] The embodiment of the present application combines video frame rate and video frame similarity analysis to achieve efficient and precise compression processing of cross-regional target video sets. The cross-regional target video set is initially compressed by frame rate to reduce the amount of data, and the target frame is used to accurately separate the foreground information and background information, and the weights of the foreground and background similarity coefficients are further adjusted to calculate the inter-frame similarity coefficient. At the same time, according to the preset inter-frame preset and inter-frame similarity, it is decided whether to perform secondary compression, which retains the key information of the preset target and effectively removes redundant frames, thereby significantly improving the efficiency of subsequent target information processing.

[0104] Furthermore, the second compressed video is cropped and corrected according to the preset target to obtain the cross-region compressed video of the preset target, and the specific implementation process includes:

[0105] The maximum width and height of the target frame in the second compressed video are obtained, and the second compressed video is further cropped and corrected. The specific calculation formula is:

[0106]

[0107] w1=(1+γ1)W, h1=(1+γ2)H;

[0108] Wherein, x1, y1 represent the horizontal and vertical coordinates of the upper left corner of the target area to be cropped respectively; w1, h1 represent the width and height of the target area to be cropped respectively; [x1, y1, w1, h1] represent the coordinates of the target area to be cropped; x, y represent the horizontal and vertical coordinates of the upper left corner of the target frame respectively; h, w represent the height and width of the target frame respectively; [x, y, w, h] represent the coordinates of the target frame; H, W represent the maximum height and width of the target frame in the second compressed video respectively; γ1, γ2 represent the adjustment parameters of cropping respectively;

[0109] The second compressed video is cropped according to the coordinates of the target area to be cropped to obtain a cross-region compressed video.

[0110] It is worth noting that γ1 and γ2 are related to the complexity of the surveillance video background. For example, the shooting range of camera A in a shopping mall is There is a lot of traffic inside, so the preset target is The larger the adjustment parameter is, the target video will contain the contact behavior information between the preset target and the surrounding people; the shooting range of camera B If the flow of people inside is sparse, the preset target is The smaller the adjustment parameter is, the compressed video does not contain a lot of redundant background.

[0111] The embodiment of the present application realizes accurate cropping and correction of the preset target in the second compressed video, and ensures the accuracy and consistency of the cropping result according to the adjustment parameters of the area to be cropped and the maximum width and height of all target frames. This process not only removes background information irrelevant to the target and reduces the storage space occupied, but also retains the key behavior information of the preset target in the compressed video through correction processing, thereby realizing efficient video compression and further improving the efficiency of video information processing.

[0112] The embodiment of the present application realizes cross-region information processing of multi-region synchronous monitoring videos through single-region monitoring video information processing, cross-region monitoring video switching and cross-region target video compression processing. The specific implementation process includes the following steps: (a) target information processing of single-region monitoring video; (b) switching the next monitoring area according to the preset target action trajectory; (c) compressing the cross-region target video set. The present invention proposes a video information processing model, a target behavior trajectory judgment process and two compression processes for the above three processes respectively; the video information processing model can quickly obtain the preset target information in the single-region monitoring video, thereby improving the processing speed of the single-region monitoring video information; the target behavior trajectory judgment process judges the next area where the target appears according to the semantic information of the monitoring videos of different regions, avoiding invalid processing in the monitoring area where the target does not appear or has left, and further improving the efficiency of the overall video processing; the two compression processes compress the cross-region target video set according to the frame rate and the inter-frame similarity coefficient, which can retain more behavioral details of the preset target while reducing redundant information, thereby improving the efficiency of subsequent target behavior analysis.

[0113] Embodiment 2

[0114] In Example 1, the method of the present invention realizes the information processing of the preset target on the surveillance video in a shopping mall. In the embodiment of this application, the method proposed by the present invention will be described again to realize the information processing of the A car on the surveillance video of a road section. The specific implementation process is as follows:

[0115] See also Figure 5 , obtain surveillance videos of n areas in a road section in the same time period, and perform preprocessing operations on the surveillance videos of different areas to obtain standard surveillance videos; wherein n represents the number of cameras in the road section; the specific implementation process of the preprocessing operation is:

[0116] Use filters to remove noise interference from surveillance videos in different areas to obtain denoised surveillance videos;

[0117] The video frames of the denoised surveillance video are enhanced by an image enhancement algorithm to obtain a clear surveillance video; wherein the image enhancement algorithm includes: low light enhancement, rain removal, fog removal and blur removal algorithms;

[0118] A grayscale operation is performed on each video frame of the clear surveillance video, and the pixel values ​​of all video frame images are restricted to [0, 1], so as to obtain a standard surveillance video.

[0119] Specifically, considering that outdoor cameras are greatly affected by the environment, the surveillance videos taken need to be enhanced according to different situations; for example, surveillance videos of a certain road section taken on rainy and foggy days need to be derained and defogged; surveillance videos of a certain road section taken on cloudy days or at night need to be low-light enhanced or de-spotted; if a certain road section is a highway section, deblurring processing is required.

[0120] Furthermore, a target data set is constructed based on the image of car A, and is trained using a pre-trained target detection model to obtain a trained target detection model; the model is further used to perform frame-by-frame detection on the standard surveillance video, and the video that first detects the preset target is regarded as the first-area surveillance video, and the standard surveillance videos of n-1 areas except the first-area surveillance video are integrated into a non-first-area video set, and the target detection operation is stopped; wherein the target detection model is a Transformer-based target detection model.

[0121] Specifically, there may be negative sample cars similar to Car A and similar to it in the surveillance video of a certain road section, that is, cars with exactly the same brand, color and model. In order to enable the target detection model to accurately detect Car A, when constructing the target data set, the embodiment of the present application adds car pictures with the same brand, color and model as Car A as negative samples to the data set, so that the target detection model can accurately identify Car A and prevent false detection.

[0122] Further, according to the target detection module, a video information processing model is constructed to perform target information processing on the first area surveillance video to obtain the target video; wherein the implementation process of the video information processing model includes:

[0123] In the first stage, the information processing result of the first video frame is obtained by using the trained target detection model; the difference map between the first video frame and the second video frame is calculated, and the information processing result of the first video frame and the difference map are respectively input into the feature extraction module to obtain a first information feature map and a difference feature map respectively; the cosine similarity of the first video frame and the second video frame is calculated, and a weight is generated according to the similarity to fuse the first information feature map and the difference feature map to obtain a second video fusion feature; wherein the first video frame is the first video frame in the first area surveillance video that contains the preset target; the second video frame is an unprocessed video frame in the first area surveillance video;

[0124] In the second stage, the set of processed video frames in the first area surveillance video is regarded as the first area processed video set; the second video fusion features of each video frame in the first area processed video set and the second video frame are calculated respectively, and the learned weights are used to combine them to obtain the second video dense fusion features;

[0125] Then, the dense fusion features of the second video are input into the fully connected layer to obtain the information processing result of the second video frame; further, the next frame of the second video frame is updated to the second video frame to repeat the target information processing operation to obtain the target video of the first area monitoring video;

[0126] Finally, according to the information processing result of the second video frame, it is determined whether the target has left the first area monitoring video, so as to obtain the target disappearance time; specifically, considering that the road monitoring shooting range is wide and there may be overlapping shooting areas between different cameras, when the area of ​​the target frame of car A in the second video frame is smaller than the predetermined disappearance threshold or there is no car A, it means that car A has left the shooting range of the first area monitoring video, so the time point corresponding to the second video frame is the target disappearance time; otherwise, it is considered that the next frame of the second video frame is updated to the second video frame, and the video information processing is continued.

[0127] Furthermore, according to the target disappearance time and the non-first area video set, the movement trajectory of car A is determined, so as to determine the next driving area of ​​car A and update the first area monitoring video and the non-first area video set, and further integrate the target videos of different areas into a cross-regional target video set in chronological order; the implementation process of determining the preset target movement trajectory includes:

[0128] The m-th frame video of the surveillance video of different regions in the non-first region video set after the target disappearance time is intercepted to obtain a cross-region matching video set; the semantic difference between the n-1 regional matching video frames and the corresponding m-th frame video before the target disappearance time is calculated respectively; if there is a semantic difference less than a predetermined semantic threshold, it means that car A is not in the surveillance video of the corresponding region; if there is a semantic difference greater than or equal to the predetermined semantic threshold, the target detection module is used to detect the matching video frame of the corresponding region; if the semantic difference corresponding to the cross-region matching video set is less than the predetermined semantic threshold, it means that car A has left a certain road section, and the target videos of different regions are integrated into the cross-region target video set in chronological order;

[0129] More specifically, the value of m is highly correlated with the moving speed of car A on a certain road section and the distance between cameras; for example, the surveillance video of camera A on a certain road section is the first area surveillance video, the set of surveillance videos of cameras B and C is not the first area video set, and the distance between camera A and camera B is smaller than the distance between camera A and camera C; case one: the maximum speed limit of car A in the surveillance areas of cameras B and C is the same, then the value of m corresponding to camera B is smaller than the value of m corresponding to camera C, so as to prevent car A from leaving the shooting range of camera B within m frames; case two: the surveillance area of ​​camera B is a low-speed section, and the surveillance area of ​​camera C is a high-speed section, then the value of m corresponding to camera B is larger than the value of m corresponding to camera C, so as to prevent car A from leaving the shooting range of camera C within m frames;

[0130] According to the detection results of the matching video frames in different areas, the surveillance video of the next driving area of ​​car A is determined, and the video information is processed; if only one matching video frame is detected with car A, the surveillance video of the corresponding area after the target disappearance time is obtained, and it is updated as the first area surveillance video, and the non-first area video set is updated at the same time; if more than one area has car A in the matching video frames, the area where the matching video frame with the highest target confidence is located is determined, and the surveillance video of the area after the target disappearance time is updated as the first area surveillance video, and the non-first area video set is updated at the same time; otherwise, it means that car A has left a certain road section, and the target videos of different areas are integrated into the cross-regional target video set in chronological order.

[0131] Furthermore, the cross-region target video set is compressed according to the foreground information and background information of the video frame, so as to obtain the cross-region compressed video of car A; the implementation process of the compression processing includes:

[0132] Preliminarily compressing the cross-region target video set according to the frame rate of the surveillance video of a road section to obtain a first compressed video; decomposing the first compressed video into a foreground video and a background video using a target frame, and calculating the foreground similarity coefficient and the background similarity coefficient of adjacent video frames respectively by a structural similarity index, thereby obtaining the adjacent frame similarity coefficient of the first compressed video;

[0133] According to a predetermined inter-frame threshold, determining whether to perform secondary compression processing; if the adjacent frame similarity coefficient is greater than the predetermined inter-frame threshold, removing the previous frame of video; otherwise, not removing the video frame; repeatedly performing secondary compression processing on the first compressed video until the adjacent frame similarity coefficients of all retained video frames are greater than the predetermined inter-frame threshold, thereby obtaining a second compressed video;

[0134] The second compressed video is cropped and corrected according to the preset target, the maximum width and height of the target frame in the second compressed video are obtained, and the second compressed video is further cropped and corrected. The specific calculation formula is:

[0135]

[0136] w1=(1+γ1)W, h1=(1+γ2)H;

[0137] Wherein, x1, y1 represent the horizontal and vertical coordinates of the upper left corner of the target area to be cropped respectively; w1, h1 represent the width and height of the target area to be cropped respectively; [x1, y1, w1, h1] represent the coordinates of the target area to be cropped; x, y represent the horizontal and vertical coordinates of the upper left corner of the target frame respectively; h, w represent the height and width of the target frame respectively; [x, y, w, h] represent the coordinates of the target frame; H, W represent the maximum height and width of the target frame in the second compressed video respectively; γ1, γ2 represent the adjustment parameters of cropping respectively;

[0138] The second compressed video is cropped according to the coordinates of the target area to be cropped to obtain a cross-region compressed video.

[0139] More specifically, considering that the target frame of car A may overlap with the edge of the monitoring area in road monitoring, the coordinates of the area to be cropped may be negative or exceed the width and height of the video frame. Therefore, the coordinates of the area to be cropped need to be restricted, that is, 0≤x1≤W max -w1,0≤y1≤H max -h1, here W max ,H max They represent the width and height of the video frame respectively. The image super-resolution algorithm based on deep learning is further used to improve the resolution of these cropped area images, so that these cross-region compressed videos can maintain the same size.

[0140] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A video information processing method based on a video information processing model, characterized in that: include: Obtain surveillance videos of n areas in a preset scene in the same time period, and perform preprocessing operations on the surveillance videos of all areas to obtain standard surveillance videos; wherein n represents the number of cameras in the preset scene; Using the target detection module, according to the image of the preset target and the standard surveillance video, determine the first area surveillance video and the non-first area video set; wherein the non-first area video set is a set of n-1 area surveillance videos except the first area surveillance video; Based on the target detection module, a video information processing model is constructed to perform target information processing on the surveillance video of the first area, and a target video and target disappearance time are obtained; wherein the target video is a video that has been processed with target information and contains a target frame; According to the disappearance time of the target and the non-first area video set, the movement trajectory of the preset target is judged, so as to determine the next movement area of ​​the preset target and update the first area surveillance video and the non-first area video set, and further integrate the target videos of different areas in chronological order to obtain a cross-region target video set; The cross-region target video set is compressed according to the foreground information and background information of the video frame, so as to obtain the cross-region compressed video of the preset target.

2. A video information processing method based on a video information processing model according to claim 1, characterized in that: The preprocessing includes: denoising, image enhancement and grayscale operations; Use filters to remove noise interference from surveillance videos in different areas to obtain denoised surveillance videos; The video frames of the denoised surveillance video are enhanced by an image enhancement algorithm to obtain a clear surveillance video; wherein the image enhancement algorithm includes: low light enhancement, rain removal, fog removal and blur removal algorithms; A grayscale operation is performed on each video frame of the clear surveillance video, and the pixel values ​​of all video frame images are restricted to [0, 1], so as to obtain a standard surveillance video.

3. The video information processing method based on the video information processing model according to claim 1, characterized in that: The target detection module includes: a training phase and an inference phase; Training phase: First, sample images of the preset target class are collected to construct a target data set, and the preset target samples are used as positive samples, and samples other than the preset targets are used as negative samples; then, a target detection model pre-trained on an open source data set is obtained; then, data enhancement operations are performed on the target data set to obtain a target expansion data set; finally, a transfer learning method is introduced, and the pre-trained target detection model is used to train on the target expansion data set, and the model weights are fine-tuned through the cross entropy loss function and the DIOU loss function to obtain a trained target detection model; wherein, the open source data sets include: Image Net, COCO and Pascal VOC data sets; the data enhancement operations include: rotation, flipping and translation; Inference stage: According to the trained target detection model, the standard surveillance video is detected frame by frame, and the video in which the preset target is first detected is regarded as the first area surveillance video.

4. The video information processing method based on the video information processing model according to claim 3 is characterized in that: The implementation process of the reasoning stage includes: According to the trained target detection model, the video frames of the standard surveillance video in different areas at the first time point are detected one by one to determine whether the preset target exists in the video frame of the first time; if the preset target does not exist, the next video frame of the first time is updated to the video frame of the first time for re-detection and judgment operations; if the preset target exists, the video frame with the highest target confidence is regarded as the first action position of the preset target, the standard surveillance video of the corresponding area is used as the first area surveillance video, the standard surveillance videos of n-1 areas except the first area surveillance video are integrated into a non-first area video set, and the target detection operation is stopped.

5. The video information processing method based on the video information processing model according to claim 1, characterized in that: The implementation process of the video information processing model includes: In the first stage, the information processing result of the first video frame is obtained by using the trained target detection model; the difference map of the first video frame and the second video frame is calculated, and the information processing result of the first video frame and the difference map are respectively input into the feature extraction module to obtain a first information feature map and a difference feature map respectively; the cosine similarity of the first video frame and the second video frame is calculated, and a weight is generated according to the similarity to fuse the first information feature map and the difference feature map to obtain a second video fusion feature; wherein, the first video frame is the first video frame in the first area monitoring video that contains the preset target; the second video frame is an unprocessed video frame in the first area monitoring video; the specific calculation formula of the second video fusion feature is: F=(1 c -cos)⊙F dif +F dec ; Among them, cos i A represents the cosine similarity of the i-th channel; i and B i Represent the image vectors of the i-th channel of the first video frame and the second video frame respectively; · represents the dot product operation of the vector; || || represents the operation of taking the modulus of the vector; F represents the second video fusion feature; 1 c represents a vector with dimension c; cos represents the cosine similarity; F dif and F dec denote the difference feature map and the first information feature map respectively; ⊙ denotes a channel multiplication operation; In the second stage, the set of processed video frames in the first area surveillance video is regarded as the first area processed video set; the second video fusion features of each video frame in the first area processed video set and the second video frame are calculated respectively, and the learning weights are used to combine them to obtain the second video dense fusion features. The specific formula is expressed as follows: in, represents the second video dense fusion feature of the kth video frame; w j represents the learning weight; F j represents the second fusion feature of the jth video frame; note that k ≥ 2 and Then, the dense fusion features of the second video are input into the fully connected layer to obtain the information processing result of the second video frame; further, the next frame video image of the second video frame is updated to the second video frame to repeat the target information processing operation to obtain the target video of the first area monitoring video.

6. The video information processing method based on the video information processing model according to claim 5, characterized in that: The implementation process of the video information processing model also includes: When the preset target is not in the second video frame, determine whether the preset target disappears according to the target frame coordinates in the previous frame of the second video frame and the size of the video frame; if the target frame area is smaller than the predetermined disappearance threshold or any edge of the target frame coincides with the edge of the corresponding video frame, it means that the preset target is about to leave the shooting range of the first area monitoring video, so the time point corresponding to the second video frame is the target disappearance time; otherwise, update the next frame of the second video frame to the second video frame, and continue to process the video information.

7. The video information processing method based on the video information processing model according to claim 1, characterized in that: The process of determining the action trajectory of the preset target includes: The m-th frame video of the surveillance video of different regions in the non-first region video set after the target disappearance time is intercepted to obtain a cross-region matching video set; the semantic difference between the n-1 region matching video frames and the corresponding m-th frame video before the target disappearance time is calculated respectively, and the specific calculation formula is: in, represents semantic difference; VGG() represents the pre-trained VGG network; x t-m represents the mth frame of video image before the target disappears; X t+m represents the mth frame of video image after the target disappears; || ||2 represents the two-norm function; If the semantic difference is less than a predetermined semantic threshold, it means that the preset target is not in the surveillance video of the corresponding area; if the semantic difference is greater than or equal to the predetermined semantic threshold, the target detection module is used to detect the matching video frames of the corresponding area; if the semantic differences corresponding to the cross-regional matching video set are all less than the predetermined semantic threshold, it means that the preset target has left the preset scene, and the target videos in different areas are integrated into the cross-regional target video set in chronological order.

8. The video information processing method based on the video information processing model according to claim 7, characterized in that: The process of determining the action trajectory of the preset target also includes: According to the detection results of the matching video frames in different areas, the surveillance video of the next action area of ​​the preset target is determined, and the video information is processed; if the preset target is only detected in one matching video frame, the surveillance video of the corresponding area after the disappearance time of the target is obtained, and it is updated as the surveillance video of the first area, and the video set of the non-first area is updated at the same time; if the preset target exists in the matching video frames in more than one area, the area where the matching video frame with the highest target confidence is located is determined, and the surveillance video of the area after the disappearance time of the target is updated to the surveillance video of the first area, and the video set of the non-first area is updated at the same time; otherwise, it means that the preset target has left the preset scene, and the target videos of different areas are integrated into the cross-regional target video set in chronological order.

9. The video information processing method based on the video information processing model according to claim 1, characterized in that: The implementation process of the compression process includes: The cross-regional target video set is preliminarily compressed according to the frame rate of the surveillance video to obtain a first compressed video; the first compressed video is decomposed into a foreground video and a background video using a target frame, and the foreground similarity coefficient and the background similarity coefficient of adjacent video frames are respectively calculated by a structural similarity index, thereby obtaining the adjacent frame similarity coefficient of the first compressed video; wherein, the foreground video sets the background pixel value to 0; and the background video sets the foreground pixel value to 0; the specific calculation formula of the adjacent frame similarity coefficient is: here, represents the similarity coefficient between the jth and j+1th frame video images; SSIM() represents the structural similarity function; Respectively represent the j-th and j+1-th frame video images of the foreground video; represent the jth and j+1th frame video images of the background video respectively; α, β represent the adjustment parameters of the foreground similarity coefficient and the background similarity coefficient respectively, and α>β>0; According to the predetermined inter-frame threshold, determine whether to perform secondary compression processing; if the adjacent frame similarity coefficient is greater than the predetermined inter-frame threshold, remove the j-th frame video; otherwise, do not remove the video frame; repeat the secondary compression processing on the first compressed video until the adjacent frame similarity coefficients of all retained video frames are greater than the predetermined inter-frame threshold, thereby obtaining a second compressed video.

10. The video information processing method based on the video information processing model according to claim 9, characterized in that: The implementation process of the compression processing also includes: The maximum width and height of the target frame in the second compressed video are obtained, and the second compressed video is further cropped and corrected. The specific calculation formula is: w1=(1+γ1)W, h1=(1+γ2)H; Wherein, x1, y1 represent the horizontal and vertical coordinates of the upper left corner of the target area to be cropped, respectively; w1, h1 represent the width and height of the target area to be cropped, respectively; [x1, y1, w1, h1] represent the coordinates of the target area to be cropped; x, y represent the horizontal and vertical coordinates of the upper left corner of the target frame, respectively; h, w represent the height and width of the target frame, respectively; [x, y, w, h] represent the coordinates of the target frame; H, W represent the maximum height and width of the target frame in the second compressed video, respectively; γ1, γ2 represent the adjustment parameters of cropping, respectively; The second compressed video is cropped according to the coordinates of the target area to be cropped to obtain a cross-region compressed video.

Images