A method and system for optimizing the picture quality of panoramic video fusion video

Abstract

The present invention discloses a method and system for optimizing the picture quality of panoramic video fusion video, which relates to the technical field of video data processing. The method includes: integrating multiple sets of video image frame data based on the video image frame coding information, and then evaluating the picture quality of the integrated set of video sequence image frame data through a set of video picture quality evaluation indicators to obtain a set of video sequence image frame picture quality evaluation parameter sets; using a dual-channel video picture quality optimization method based on the set of video sequence image frame picture quality evaluation parameter sets to perform mapping processing on the set of video sequence image frame data, and then performing panoramic video enhancement fusion on the processed set of video sequence picture quality optimized image frame data according to the video image frame coding information to obtain a target panoramic optimized fusion video. Through the dual-channel video picture quality optimization, targeted picture quality optimization of the panoramic video is realized, the picture quality clarity and optimization processing efficiency of the panoramic fusion video are improved, and thus the panoramic video monitoring effect is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of video data processing, and in particular to a method and system for optimizing the picture quality of panoramic video fusion video. Background Art

[0002] With the rapid development of digital media and virtual reality technologies, panoramic video has become an important form of video surveillance. The panoramic video fusion technology can perform global real-time monitoring of large scenes and quickly retrieve historical events, greatly reducing the work efficiency of operation and maintenance personnel. However, during the panoramic video fusion process, due to differences in perspectives, lighting, colors, etc. between different cameras, the picture quality of the fused video often deteriorates. Therefore, how to optimize the picture quality of panoramic video fusion video has become an urgent problem to be solved. Summary of the Invention

[0003] This application provides a method and system for optimizing the picture quality of panoramic video fusion video, solves the technical problem that the panoramic video fusion in the prior art leads to a decrease in video picture quality, thereby affecting the video surveillance effect, and achieves the technical effect of realizing targeted picture quality optimization of panoramic video through dual channels of video picture quality optimization, improving the picture quality clarity and optimization processing efficiency of panoramic fusion video, and thereby ensuring the video surveillance effect of panoramic video.

[0004] In view of the above problems, the present invention provides a method and system for optimizing the picture quality of panoramic video fusion video.

[0005] In a first aspect, this application provides a method for optimizing the picture quality of panoramic video fusion video, the method comprising: acquiring video information collected by a plurality of cameras, respectively extracting image frames from the video information collected by the plurality of cameras to obtain a plurality of sets of video image frame data; performing index coding on the plurality of sets of video image frame data according to a video surveillance scene to obtain video image frame coding information, and integrating the plurality of sets of video image frame data based on the video image frame coding information to determine a set of video sequence image frame data; acquiring a set of video picture quality evaluation indicators, performing multi-dimensional evaluation extraction of the picture quality on the set of video sequence image frame data through the set of video picture quality evaluation indicators to obtain a set of video sequence image frame picture quality evaluation parameters; constructing a dual channel for video picture quality optimization, and performing mapping association processing on the set of video sequence image frame data based on the set of video sequence image frame picture quality evaluation parameters by using the dual channel for video picture quality optimization to determine a set of video sequence picture quality optimized image frame data; and performing panoramic video enhancement fusion on the set of video sequence picture quality optimized image frame data according to the video image frame coding information to obtain a target panoramic optimized fusion video.

[0006] On the other hand, the present application also provides a picture quality optimization system for panoramic video fusion video. The system includes: an image frame extraction module, configured to obtain video information collected by multiple cameras, and respectively perform image frame extraction on the video information collected by the multiple cameras to obtain a plurality of video image frame data sets; a video image integration module, configured to perform index coding on the plurality of video image frame data sets according to a video monitoring scenario to obtain video image frame coding information, and perform video image integration on the plurality of video image frame data sets based on the video image frame coding information to determine a video sequence image frame data set; a picture quality multi-dimensional evaluation module, configured to obtain a video picture quality evaluation index set, and perform picture quality multi-dimensional evaluation extraction on the video sequence image frame data set through the video picture quality evaluation index set to obtain a video sequence image frame picture quality evaluation parameter set; a video picture quality optimization module, configured to construct a dual-channel for video picture quality optimization, and perform mapping association processing on the video sequence image frame data set based on the video sequence image frame picture quality evaluation parameter set by using the dual-channel for video picture quality optimization to determine a video sequence picture quality optimized image frame data set; a panoramic video enhancement fusion module, configured to perform panoramic video enhancement fusion on the video sequence picture quality optimized image frame data set according to the video image frame coding information to obtain a target panoramic optimized fusion video.

[0007] One or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0008] Due to the technical solution of respectively performing image frame extraction on the video information collected by multiple cameras, then performing index coding integration on the extracted multiple video image frame data sets according to the video monitoring scenario to determine a video sequence image frame data set, performing picture quality multi-dimensional evaluation extraction on the video sequence image frame data set through a video picture quality evaluation index set to obtain a video sequence image frame picture quality evaluation parameter set, using a dual-channel for video picture quality optimization to perform mapping association processing on the video sequence image frame data set based on the video sequence image frame picture quality evaluation parameter set to determine a video sequence picture quality optimized image frame data set, and performing panoramic video enhancement fusion on it according to the video image frame coding information to obtain a target panoramic optimized fusion video. Furthermore, the technical effect of realizing targeted picture quality optimization of panoramic video through the dual-channel for video picture quality optimization, improving the picture quality clarity and optimization processing efficiency of the panoramic fusion video, and further ensuring the panoramic video monitoring effect is achieved.

[0009] The above description is only an overview of the technical solutions of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present application more obvious and understandable, the specific embodiments of the present application are specifically given below. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a schematic flowchart of a method for optimizing the picture quality of a panoramic video fusion video in this application;

[0011] Figure 2 It is a schematic flowchart of constructing a dual-channel for optimizing the picture quality of a panoramic video fusion video in this application;

[0012] Figure 3 It is a schematic structural diagram of a system for optimizing the picture quality of a panoramic video fusion video in this application.

[0013] Explanation of reference numerals: Image frame extraction module 11, video image integration module 12, multi-dimensional picture quality evaluation module 13, video picture quality optimization module 14, panoramic video enhancement and fusion module 15. Specific implementation mode

[0014] By providing a method and system for optimizing the picture quality of a panoramic video fusion video in this application, the technical problem in the prior art that the panoramic video fusion causes the decline of the video picture quality, thereby affecting the video monitoring effect, is solved. The technical effect of realizing the targeted picture quality optimization of the panoramic video through the dual-channel of video picture quality optimization, improving the picture quality clarity and optimization processing efficiency of the panoramic fusion video, and thereby ensuring the panoramic video monitoring effect is achieved.

[0015] In order to make the purpose, technical solution and advantages of this application clearer, the following further details this application in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain this application and are not used to limit this application.

[0016] The following describes this application in conjunction with the accompanying drawings in this application.

[0017] Embodiment 1

[0018] As Figure 1 shown, this application provides a method for optimizing the picture quality of a panoramic video fusion video, and the method includes:

[0019] Step S1: Obtain video information collected by multiple cameras, and respectively perform image frame extraction on the video information collected by the multiple cameras to obtain multiple sets of video image frame data.

[0020] Specifically, to achieve targeted picture quality optimization for panoramic fusion videos, first, video information collected by multiple cameras is obtained through multiple monitoring devices. The video information collected by the multiple cameras is monitoring video information of multiple regions and multiple angles for monitoring a monitored scene area. Image frames are extracted from the video information collected by the multiple cameras respectively through video stream decoding, and each decoded image frame is saved as a picture file to obtain corresponding multiple video image frame data sets, providing a data basis for subsequent video picture quality optimization.

[0021] Step S2: Index and encode the multiple video image frame data sets according to the video monitoring scene to obtain video image frame encoding information, and integrate the multiple video image frame data sets based on the video image frame encoding information to determine a video sequence image frame data set.

[0022] Furthermore, for obtaining the video image frame encoding information, the steps of this application further include:

[0023] Obtain the regional distribution information of the video monitoring scene, use 3D modeling technology to perform three-dimensional modeling on the regional distribution information, and construct a scene three-dimensional space model; perform regional labeling and segmentation on the scene three-dimensional space model to determine a set of labeled regions of the monitoring scene; determine image frame encoding element information according to the set of labeled regions of the monitoring scene and the video image frame timestamps; design an encoding system based on the encoding element information to determine an image frame encoding rule, where the image frame encoding rule includes an encoding order, an encoding length, and an encoding identifier; based on the image frame encoding rule, perform index encoding identification on the multiple video image frame data sets to obtain the video image frame encoding information.

[0024] Specifically, the multiple video image frame data sets are index-encoded according to the video monitoring scene. Among them, the video monitoring scene is the monitored scene area collected by camera videos, such as a power inspection area, etc. To encode the video image frames, first obtain the regional distribution information of the video monitoring scene, and the regional distribution information includes the spatial regional distribution position of the video monitoring scene and the distribution structure of the spatial regions, etc. Use 3D modeling technology to perform three-dimensional modeling on the regional distribution information to construct a scene three-dimensional space model, and the scene three-dimensional space model is used to visually display the spatial distribution information of the video monitoring scene.

[0025] Perform regional labeling segmentation on the three-dimensional space model of the scene to determine the set of labeled regions of the monitoring scene. The set of labeled regions of the monitoring scene includes various region types and spatial positions in the monitoring scene area, such as power production areas, power distribution areas, etc., as type labels. According to the set of labeled regions of the monitoring scene and the video image frame timestamps, determine the encoding element information of the image frames. The encoding element information of the image frames is the type of encoding requirements for each image frame data, including the labeling of the monitoring scene area and the timestamp of this image frame. Based on the encoding element information, design an encoding system to determine the image frame encoding rules. The image frame encoding rules are the basis for the encoding content of the encoding elements, including the encoding order, that is, the front-back order of the encoding elements; the encoding length, that is, the information length of the encoding elements; and the encoding identifier, that is, the identification symbol of the encoding elements, such as numbers, letters, etc.

[0026] Based on the image frame encoding rules, perform index encoding identification on each image frame in the set of multiple video image frame data to obtain the corresponding video image frame encoding information. The video image frame encoding information is the unique information encoding of each image frame, which is convenient for subsequent rapid indexing of the image frame data. And based on the video image frame encoding information, integrate the set of multiple video image frame data. According to the image frame encoding, the image frame data can be arranged and integrated in the order of the acquisition region type, position, and the image frame timestamps from early to late to determine the integrated set of video sequence image frame data. Realize the rapid and reasonable encoding of video image frames, improve the processing efficiency of image frame information, and thus ensure the subsequent panoramic video image quality optimization processing efficiency.

[0027] Step S3: Obtain the set of video image quality evaluation indicators, and perform multi-dimensional evaluation and extraction of the quality of the set of video sequence image frame data through the set of video image quality evaluation indicators to obtain the set of video sequence image frame quality evaluation parameters.

[0028] Specifically, formulate the set of video image quality evaluation indicators. The set of video image quality evaluation indicators is an important set of indicators for multi-dimensional evaluation of video image quality, including resolution, color depth, color restoration degree, contrast, etc. Perform multi-dimensional evaluation and extraction of the quality of the set of video sequence image frame data through the set of video image quality evaluation indicators, map and extract the quality evaluation results to obtain the corresponding set of video sequence image frame quality evaluation parameters. The set of video sequence image frame quality evaluation parameters includes the quality evaluation parameters of all video image frame data, providing a reference basis for subsequent image frame quality optimization.

[0029] Step S4: Construct a dual-channel for video image quality optimization, and use the dual-channel for video image quality optimization to perform mapping and correlation processing on the video sequence image frame data set based on the video sequence image frame quality evaluation parameter set, and determine the video sequence image frame data set with optimized image quality.

[0030] As Figure 2 shown, further, for the construction of the dual-channel for video image quality optimization, the steps of this application further include:

[0031] Acquire a data set of scene video image frames according to the video surveillance scene, and select a background image frame data set from the data set of scene video image frames according to the background step size; perform data selection on the data set of scene video image frames according to the foreground step size to obtain a foreground image frame data set, where the foreground step size is smaller than the background step size; use the background image frame data set and the foreground image frame data set as training data for image quality optimization analysis training until a preset convergence condition is reached to obtain a background optimization processing channel and a foreground optimization processing channel; connect the background optimization processing channel and the foreground optimization processing channel in parallel to construct the dual-channel for video image quality optimization.

[0032] Further, for the acquisition of the foreground image frame data set, the steps of this application further include:

[0033] Determine a standard surveillance scene grayscale image according to the data set of scene video image frames; perform grayscale distribution recognition on the standard surveillance scene grayscale image to obtain a surveillance scene grayscale interval, and use the surveillance scene grayscale interval as the scene background feature; perform foreground target recognition on the data set of scene video image frames to obtain a foreground target set, and perform grayscale distribution recognition on the foreground target set to obtain the scene foreground feature; construct an attention mechanism module based on the scene background feature and the scene foreground feature, and use the attention mechanism module to perform data selection on the data set of scene video image frames according to the foreground step size to obtain the foreground image frame data set.

[0034] Specifically, to improve the efficiency of image frame optimization processing, a dual-channel for video image quality optimization is constructed. The dual-channel for video image quality optimization is used to perform simultaneous and efficient image quality optimization processing on the foreground data and background data of the image frame. The specific construction process is as follows: First, a dataset of scene video image frames is collected according to the video surveillance scene. The dataset of scene video image frames is the historical video image frame data of the video surveillance scene. The dataset of scene video image frames is selected for image acquisition according to the background step size to obtain a corresponding dataset of background image frames. The background step size is the extraction time interval of the image frame. The background of the image frame is static background data, including scene buildings, etc., with little change. Therefore, the background step size is relatively large. For example, one frame is collected every 10 s. Then, the dataset of scene video image frames is selected for image data according to the foreground step size to obtain a corresponding dataset of foreground image frames. The foreground of the image frame is dynamic moving target data, with large changes. Therefore, the foreground step size is smaller than the background step size. For example, one frame is collected every 5 s.

[0035] The extraction process of the foreground image frame data is as follows: First, according to the dataset of scene video image frames, a standard surveillance scene grayscale image is determined. The standard surveillance scene grayscale image is the grayscale image of the surveillance scene area without moving targets. The grayscale distribution of the standard surveillance scene grayscale image is identified to obtain a corresponding grayscale interval of the surveillance scene, and the grayscale interval of the surveillance scene is used as the scene background feature. The foreground targets in the dataset of scene video image frames are identified to obtain a foreground target set. The foreground target set is the set of moving targets in the surveillance scene, such as power personnel. The grayscale distribution of the foreground target set is identified to obtain a corresponding scene foreground feature. The scene foreground feature is the set of grayscale distribution features of the foreground targets. Based on the scene background feature and the scene foreground feature, an attention mechanism module is constructed. The attention mechanism module is used to quickly identify and extract the foreground area and background area of the image in the video surveillance scene. Through the attention mechanism module, the dataset of scene video image frames is collected for image extraction according to the foreground step size, and then the foreground data of the collected image frames is selected to obtain a corresponding dataset of foreground image frames after processing.

[0036] Using the background image frame dataset and the foreground image frame dataset as training data respectively, evaluating the image quality parameters through the video image quality evaluation metric set, and then performing image quality optimization analysis training through the image quality evaluation parameters until the preset convergence condition is reached. The preset convergence condition can be that the model performance index is stable or the preset number of iterations is reached, obtaining the trained background image quality optimization model and foreground image quality optimization model, and using them as the background optimization processing channel and foreground optimization processing channel. Connect the background optimization processing channel and the foreground optimization processing channel in parallel to construct a dual-channel for video image quality optimization to perform efficient parallel optimization processing of the background data and foreground data of the image frames simultaneously. Using the dual-channel for video image quality optimization, based on the video sequence image frame quality evaluation parameter set, perform optimization channel mapping association and step size extraction optimization processing on the background image frame data and foreground image frame data of the video sequence image frame data set, and determine the video sequence image quality optimized image frame data set after the output video image frame quality optimization processing. Realize the targeted and efficient image quality optimization of the dual-channel for video image quality optimization, thereby improving the image quality clarity and image frame optimization processing efficiency of the panoramic fusion video.

[0037] Step S5: Perform panoramic video enhancement fusion on the video sequence image quality optimized image frame data set according to the video image frame coding information to obtain the target panoramic optimized fusion video.

[0038] Furthermore, for obtaining the target panoramic optimized fusion video, the steps of this application further include:

[0039] Mark the overlapping and blending regions of the video sequence image quality optimized image frame data set to obtain a video blending region set; assign weights to the video sequence image quality optimized image frame data set based on the video blending region set to determine the sequence video fusion weight factor information; based on the sequence video fusion weight factor information and according to the video image frame coding information, perform panoramic video fusion on the video sequence image quality optimized image frame data set to obtain the target panoramic optimized fusion video.

[0040] Specifically, the panoramic video enhancement fusion is performed on the video sequence picture quality optimized image frame data set according to the video image frame coding information. During the fusion process, an adaptive weighted fusion algorithm is adopted. First, the overlapping and blending regions in the video sequence picture quality optimized image frame data set are marked to obtain a corresponding video blending region set, and the video blending region set is the adjacent and overlapping regions of each collected video image frame. Based on the video blending region set, weight assignment is performed on the video sequence picture quality optimized image frame data set. The fusion weight can be assigned to the image frames with overlapping regions according to the information content of the image frames. The larger the detailed information content of an image frame, the larger its corresponding fusion weight factor. In this way, the sequence video fusion weight factor information is determined to retain more video details and color information. Based on the sequence video fusion weight factor information and according to the video image frame coding information, the video sequence picture quality optimized image frame data set is subjected to panoramic video fusion to obtain a target panoramic optimized fusion video with optimized picture quality. The picture quality clarity and optimization fusion efficiency of the panoramic fusion video are improved, ensuring that the video image frame fusion has richer information content, and further ensuring the panoramic video monitoring effect.

[0041] Furthermore, for obtaining the background optimization processing channel and the foreground optimization processing channel, the steps of this application further include:

[0042] Image frame screening is performed on the background image frame data set and the foreground image frame data set according to a preset video picture quality standard to obtain a high picture quality image frame data set, and the high picture quality image frame data set includes a background high picture quality data set and a foreground high picture quality data set; picture quality loss processing is performed on the high picture quality image frame data set to obtain a low picture quality image frame data set, and the low picture quality image frame data set is subjected to picture quality evaluation through the video picture quality evaluation index set to obtain a low picture quality image frame evaluation parameter set; the high picture quality image frame data set and the low picture quality image frame data set are subjected to mapping correlation analysis to determine an image frame picture quality sample data set; a convolutional neural network structure is used to perform picture quality enhancement training optimization on the low picture quality image frame evaluation parameter set and the image frame picture quality sample data set to obtain the background optimization processing channel and the foreground optimization processing channel.

[0043] Furthermore, for obtaining the background optimization processing channel and the foreground optimization processing channel, the steps of this application further include:

[0044] Evaluate the reasons for image quality generation and analyze the low image quality level for the set of low image quality frame evaluation parameters to determine the low image quality generation reason parameters and low image quality level parameters; based on the low image quality generation reason parameters and low image quality level parameters, configure the image quality optimization program for the set of image frame quality sample data to obtain a set of image quality optimization programs; use the convolutional neural network structure to perform image quality enhancement training and optimization on the set of image quality optimization programs and the set of image frame quality sample data to obtain the background optimization processing channel and the foreground optimization processing channel.

[0045] Specifically, use the background image frame data set and the foreground image frame data set as training data for image quality optimization analysis and training respectively. The specific process is as follows: First, screen the background image frame data set and the foreground image frame data set according to the preset video image quality standard. Among them, the preset video image quality standard is the image quality standard of panoramic fusion video, which can be set according to application requirements. Obtain a set of high-quality image frame data that meets the preset video image quality standard. The set of high-quality image frame data includes a background high-quality data set and a foreground high-quality data set. Perform image quality loss processing on the set of high-quality image frame data, including image quality loss methods such as reducing the resolution of video image frames, image frame compression, and adding noise, to obtain a set of low-quality image frame data after loss processing. Evaluate the image quality of the set of low-quality image frame data through the set of video image quality evaluation indicators to obtain a corresponding set of low-quality image frame evaluation parameters. Perform mapping correlation analysis on the set of high-quality image frame data and the set of low-quality image frame data to determine the set of image frame quality sample data. The set of image frame quality sample data includes a set of high-quality image frame data that meets the preset video image quality standard and a corresponding set of low-quality image frame data after loss processing.

[0046] Use the convolutional neural network structure to perform image quality enhancement training and optimization on the set of low-quality image frame evaluation parameters and the set of image frame quality sample data. First, evaluate the reasons for image quality generation and analyze the low image quality level for the set of low-quality image frame evaluation parameters to determine the low image quality generation reason parameters, such as noise influence, information loss, etc.; the low image quality level parameters, which are the severity levels of the low image quality at which the current image frame quality evaluation parameters are located. Based on the low image quality generation reason parameters and low image quality level parameters, configure the image quality optimization program for the set of image frame quality sample data, analyze the image quality optimization steps according to the low image quality generation reasons and their low image quality levels. Exemplarily, configure the filter parameters of the corresponding low image quality level for the noise influence on low image quality, and then perform filtering processing on it through the filter. Integrate the processing steps of each low-quality image frame to obtain a set of image quality optimization programs. The set of image quality optimization programs is used to match the corresponding optimization processing steps according to each low image quality generation reason and low image quality level.

[0047] Use a convolutional neural network structure to perform quality enhancement training and optimization on the set of video quality optimization programs and the set of image frame quality sample data. Use the set of image frame quality sample data as training data and the set of video quality optimization programs as output identification data to perform quality optimization processing training on video image frames until the preset convergence condition is reached, obtaining a trained background quality optimization model and a foreground quality optimization model, and using them as the background optimization processing channel and the foreground optimization processing channel. Implement efficient parallel quality optimization processing of image frame background data and image frame foreground data, improve the image quality clarity and image frame optimization processing efficiency of panoramic fusion videos, and thus ensure the panoramic video monitoring effect.

[0048] In summary, the video quality optimization method for a panoramic video fusion video provided by this application has the following technical effects:

[0049] Since the technical solution is adopted of separately extracting image frames from video information collected by multiple cameras, then indexing and encoding and integrating the extracted multiple sets of video image frame data according to the video monitoring scenario to determine a set of video sequence image frame data, performing multi-dimensional evaluation and extraction of the quality of the set of video sequence image frame data through a set of video quality evaluation indicators to obtain a set of video sequence image frame quality evaluation parameters, using a dual-channel video quality optimization to perform mapping and association processing on the set of video sequence image frame data based on the set of video sequence image frame quality evaluation parameters to determine a set of video sequence quality optimization image frame data, and performing panoramic video enhancement and fusion on it according to the video image frame coding information to obtain a target panoramic optimized fusion video. Thus, the technical effect of achieving targeted quality optimization of panoramic videos through a dual-channel video quality optimization, improving the image quality clarity and optimization processing efficiency of panoramic fusion videos, and thus ensuring the panoramic video monitoring effect is achieved.

[0050] Embodiment 2

[0051] Based on the same inventive concept as the video quality optimization method for a panoramic video fusion video in the foregoing embodiment, the present invention also provides a video quality optimization system for a panoramic video fusion video, as Figure 3 shown, the system includes:

[0052] The image frame extraction module 11 is used to obtain video information collected by multiple cameras, extract image frames from the video information collected by the multiple cameras respectively, and obtain a plurality of video image frame data sets; the video image integration module 12 is used to perform index coding on the plurality of video image frame data sets according to the video monitoring scenario, obtain video image frame coding information, and perform video image integration on the plurality of video image frame data sets based on the video image frame coding information to determine a video sequence image frame data set; the multi-dimensional video quality evaluation module 13 is used to obtain a video quality evaluation index set, perform multi-dimensional video quality evaluation extraction on the video sequence image frame data set through the video quality evaluation index set, and obtain a video sequence image frame quality evaluation parameter set; the video quality optimization module 14 is used to construct a dual-channel video quality optimization, and perform mapping and association processing on the video sequence image frame data set based on the video sequence image frame quality evaluation parameter set by using the dual-channel video quality optimization to determine a video sequence quality optimization image frame data set; the panoramic video enhancement and fusion module 15 is used to perform panoramic video enhancement and fusion on the video sequence quality optimization image frame data set according to the video image frame coding information to obtain a target panoramic optimized fusion video.

[0053] Further, the video image integration module 12 is further used for:

[0054] Obtain the regional distribution information of the video monitoring scenario, perform three-dimensional modeling on the regional distribution information by using three-dimensional modeling technology, and construct a scene three-dimensional space model; perform regional tagging segmentation on the scene three-dimensional space model to determine a set of monitored scene tagged regions; determine image frame coding element information according to the set of monitored scene tagged regions and the video image frame timestamp; perform coding system design based on the coding element information to determine an image frame coding rule, where the image frame coding rule includes a coding order, a coding length, and a coding identifier; based on the image frame coding rule, perform index coding identification on the plurality of video image frame data sets to obtain the video image frame coding information.

[0055] Further, the video quality optimization module 14 is further used for:

[0056] Collect and obtain a dataset of scene video image frames according to the video surveillance scenario, and select a background image frame dataset from the dataset of scene video image frames according to a background step size; perform data selection on the dataset of scene video image frames according to a foreground step size to obtain a foreground image frame dataset, where the foreground step size is smaller than the background step size; use the background image frame dataset and the foreground image frame dataset as training data respectively to perform image quality optimization analysis training until a preset convergence condition is reached, and obtain a background optimization processing channel and a foreground optimization processing channel; connect the background optimization processing channel and the foreground optimization processing channel in parallel to construct the video image quality optimization dual channel.

[0057] Further, the video image quality optimization module 14 is further configured to:

[0058] Determine a standard surveillance scene grayscale image according to the dataset of scene video image frames; perform grayscale distribution recognition on the standard surveillance scene grayscale image to obtain a surveillance scene grayscale interval, and use the surveillance scene grayscale interval as a scene background feature; perform foreground target recognition on the dataset of scene video image frames to obtain a foreground target set, perform grayscale distribution recognition on the foreground target set to obtain a scene foreground feature; based on the scene background feature and the scene foreground feature, construct an attention mechanism module, and perform data selection on the dataset of scene video image frames according to the foreground step size through the attention mechanism module to obtain the foreground image frame dataset.

[0059] Further, the video image quality optimization module 14 is further configured to:

[0060] Perform image frame screening on the background image frame dataset and the foreground image frame dataset according to a preset video image quality standard to obtain a high-quality image frame data set, where the high-quality image frame data set includes a background high-quality data set and a foreground high-quality data set; perform image quality loss processing on the high-quality image frame data set to obtain a low-quality image frame data set, perform image quality evaluation on the low-quality image frame data set through the video image quality evaluation index set to obtain a low-quality image frame evaluation parameter set; perform mapping correlation analysis on the high-quality image frame data set and the low-quality image frame data set to determine an image frame quality sample data set; use a convolutional neural network structure to perform image quality enhancement training optimization on the low-quality image frame evaluation parameter set and the image frame quality sample data set to obtain the background optimization processing channel and the foreground optimization processing channel.

[0061] Further, the video image quality optimization module 14 is further configured to:

[0062] Evaluate the parameters set of the low-quality image frames to analyze the reasons for image quality generation and the low-quality level, and determine the parameters for the reasons of low-quality generation and the parameters for the low-quality level; based on the parameters for the reasons of low-quality generation and the parameters for the low-quality level, configure the image quality optimization program for the set of image frame quality sample data to obtain a set of image quality optimization programs; use a convolutional neural network structure to perform image quality enhancement training and optimization on the set of image quality optimization programs and the set of image frame quality sample data to obtain the background optimization processing channel and the foreground optimization processing channel.

[0063] Further, the panoramic video enhancement and fusion module 15 is further configured to:

[0064] Mark the overlapping and blending regions of the set of image frames with optimized video sequence image quality to obtain a set of video blending regions; based on the set of video blending regions, assign weights to the set of image frames with optimized video sequence image quality to determine the information of the sequence video fusion weight factors; based on the information of the sequence video fusion weight factors and according to the video image frame coding information, perform panoramic video fusion on the set of image frames with optimized video sequence image quality to obtain the target panoramic optimized fusion video.

[0065] The foregoing Figure 1 All the various change modes and specific examples of the method for optimizing the image quality of a panoramic video fusion video in the first embodiment are equally applicable to the system for optimizing the image quality of a panoramic video fusion video in this embodiment. Through the foregoing detailed description of the method for optimizing the image quality of a panoramic video fusion video, those skilled in the art can clearly know the implementation method of the system for optimizing the image quality of a panoramic video fusion video in this embodiment. Therefore, for the sake of simplicity of the specification, it will not be described in detail here.

[0066] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present application, and all of them should be covered by the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for optimizing the image quality of panoramic video fusion video, characterized in that: The method comprises: Acquire video information collected by multiple cameras, extract image frames from the video information collected by the multiple cameras respectively, and obtain multiple video image frame data sets; Index encoding is performed on the multiple video image frame data sets according to the video surveillance scene to obtain video image frame encoding information, and video image integration is performed on the multiple video image frame data sets based on the video image frame encoding information to determine a video sequence image frame data set, wherein the multiple video image frame data sets are integrated with video images, including: video image integration is performed on the multiple video image frame data sets based on the video image frame encoding information, and image frame data can be arranged and integrated according to the acquisition area type and position, and the image frame timestamp is arranged in a descending order according to the image frame encoding to determine the integrated video sequence image frame data set; Acquire a set of video quality evaluation indicators, perform multi-dimensional quality evaluation extraction on the video sequence image frame data set using the video quality evaluation indicator set, and obtain a set of video sequence image frame quality evaluation parameters; The video quality evaluation index set is an important index set for multi-dimensional evaluation of video quality, including resolution, color depth, color reproduction and contrast; Constructing a dual channel for video quality optimization, and using the dual channel for video quality optimization to perform mapping and association processing on the video sequence image frame data set based on the video sequence image frame quality evaluation parameter set to determine the video sequence quality optimized image frame data set; The step of determining a video sequence quality optimized image frame data set comprises: The background image frame data set and the foreground image frame data set are respectively used as training data, and the image quality parameters are evaluated by the video image quality evaluation index set, and then the image quality optimization analysis training is performed by the image quality evaluation parameters until a preset convergence condition is reached. The preset convergence condition can be that the model performance index is stable or a preset number of iterations is reached, and the trained background image quality optimization model and foreground image quality optimization model are obtained, and they are used as the background optimization processing channel and the foreground optimization processing channel; The background optimization processing channel and the foreground optimization processing channel are connected in parallel to construct a dual-channel for video quality optimization, so as to simultaneously perform efficient parallel optimization processing of image frame background data and image frame foreground data; Using the video quality optimization dual channel based on the video sequence image frame quality evaluation parameter set, optimizing channel mapping association and step length extraction optimization processing are performed on the background image frame data and the foreground image frame data of the video sequence image frame data set, and determining the video sequence image frame data set after the output video image frame quality optimization processing; Performing panoramic video enhancement fusion on the video sequence quality optimized image frame data set according to the video image frame coding information to obtain a target panoramic optimized fused video; The step of obtaining the target panoramic optimized fusion video includes: Marking the overlapping blending regions of the video sequence quality optimization image frame data set to obtain a video blending region set; Based on the video blending area set, weighting is performed on the video sequence quality optimized image frame data set to determine sequence video fusion weight factor information; Based on the sequence video fusion weight factor information and according to the video image frame encoding information, the video sequence quality optimized image frame data set is subjected to panoramic video fusion to obtain the target panoramic optimized fused video; The construction of the dual-channel video quality optimization includes: Acquire a scene video image frame data set according to the video surveillance scene acquisition, and select a background image frame data set from the scene video image frame data set according to the background step length; Selecting data from the scene video image frame data set according to the foreground step length to obtain a foreground image frame data set, wherein the foreground step length is smaller than the background step length; The background image frame data set and the foreground image frame data set are respectively used as training data for image quality optimization analysis training until a preset convergence condition is reached to obtain a background optimization processing channel and a foreground optimization processing channel; The background optimization processing channel and the foreground optimization processing channel are connected in parallel to construct the video quality optimization dual channel; The obtaining of a foreground image frame data set comprises: Determining a standard monitoring scene grayscale image according to the scene video image frame data set; Performing grayscale distribution recognition on the grayscale image of the standard monitoring scene to obtain a grayscale interval of the monitoring scene, and using the grayscale interval of the monitoring scene as a scene background feature; Performing foreground target recognition on the scene video image frame data set to obtain a foreground target set, and performing grayscale distribution recognition on the foreground target set to obtain scene foreground features; Based on the scene background features and the scene foreground features, an attention mechanism module is constructed, and data is selected from the scene video image frame data set according to the foreground step size through the attention mechanism module to obtain the foreground image frame data set; The obtaining of the background optimization processing channel and the foreground optimization processing channel comprises: Performing image frame screening on the background image frame data set and the foreground image frame data set according to a preset video quality standard to obtain a high-quality image frame data set, wherein the high-quality image frame data set includes a background high-quality data set and a foreground high-quality data set; Performing image quality loss processing on the high-quality image frame data set to obtain a low-quality image frame data set, and performing image quality evaluation on the low-quality image frame data set using the video quality evaluation index set to obtain a low-quality image frame evaluation parameter set; Performing mapping association analysis on the high-quality image frame data set and the low-quality image frame data set to determine an image frame quality sample data set; The low-quality image frame evaluation parameter set and the image frame quality sample data set are subjected to image quality enhancement training optimization using a convolutional neural network structure to obtain the background optimization processing channel and the foreground optimization processing channel.

2. The method for optimizing the image quality of a panoramic video fusion video according to claim 1, characterized in that: The obtaining of video image frame coding information comprises: Acquire regional distribution information of the video surveillance scene, perform stereoscopic modeling on the regional distribution information using three-dimensional modeling technology, and construct a three-dimensional spatial model of the scene; Performing region labeling segmentation on the three-dimensional spatial model of the scene to determine a set of labeled regions of the monitoring scene; Determining image frame coding element information according to the monitoring scene labeled area set and the video image frame timestamp; Designing a coding system based on the coding element information to determine an image frame coding rule, wherein the image frame coding rule includes a coding order, a coding length, and a coding identifier; Based on the image frame coding rule, the multiple video image frame data sets are indexed and coded to obtain the video image frame coding information.

3. The method for optimizing the image quality of a panoramic video fusion video according to claim 1, characterized in that: The obtaining of the background optimization processing channel and the foreground optimization processing channel comprises: Analyzing the image quality generation cause and the low image quality level of the low image quality frame evaluation parameter set to determine a low image quality generation cause parameter and a low image quality level parameter; Based on the low image quality generation reason parameter and the low image quality level parameter, performing image quality optimization program configuration on the image frame image quality sample data set to obtain an image quality optimization program set; The image quality optimization program set and the image frame image quality sample data set are subjected to image quality enhancement training optimization using a convolutional neural network structure to obtain the background optimization processing channel and the foreground optimization processing channel.

4. A panoramic video fusion video quality optimization system, characterized in that: A method for optimizing the image quality of a panoramic video fusion video according to any one of claims 1 to 3, the system comprising: An image frame extraction module is used to obtain video information collected by multiple cameras, and respectively extract image frames from the video information collected by the multiple cameras to obtain multiple video image frame data sets; A video image integration module, configured to index and encode the plurality of video image frame data sets according to the video surveillance scene, obtain video image frame encoding information, and perform video image integration on the plurality of video image frame data sets based on the video image frame encoding information to determine a video sequence image frame data set; A multi-dimensional image quality evaluation module is used to obtain a set of video image quality evaluation indicators, and to perform multi-dimensional image quality evaluation extraction on the video sequence image frame data set through the video image quality evaluation indicator set to obtain a set of video sequence image frame quality evaluation parameters; A video quality optimization module, used for constructing a dual channel for video quality optimization, using the dual channel for video quality optimization to perform mapping and association processing on the video sequence image frame data set based on the video sequence image frame quality evaluation parameter set, and determining a video sequence quality optimized image frame data set; The panoramic video enhancement fusion module is used to perform panoramic video enhancement fusion on the video sequence quality optimized image frame data set according to the video image frame coding information to obtain a target panoramic optimized fused video.

Images