A method and system for integrating multi-source heterogeneous video resources based on data features
Through the methods of segmentation, format conversion and key feature factor screening, the problems of excessive load and format incompatibility during the integration of multi-source heterogeneous video resources are solved, the integration efficiency and stability are improved, and the correctness and compatibility of video resources are ensured. .
Patent Information
- Application Number
- CN202411161972.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-08-23
AI Technical Summary
The existing multi-source heterogeneous video resource integration method based on data characteristics is relatively large during integration, which is prone to integration failure and loss of video resources, and the video is easily incorrect format or incompatible during format conversion.
By obtaining task category information and video resource type information, segmenting multi-source heterogeneous video resource data, extracting video attribute parameters, performing format conversion and key feature factor extraction, filtering preferred data based on key feature index, and performing video integration and classification, setting Index information to optimize data.
It effectively avoids excessive integration load and resource bottlenecks, improves processing efficiency and stability, ensures compatibility of integrated video formats, and simplifies task management and resource allocation.
Smart Images

Figure CN118695025B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of video resource integration, and in particular to a method and system for integrating multi-source heterogeneous video resources based on data features. Background Art
[0002] The multi-source heterogeneous video resource integration system aims to improve the comprehensiveness and accuracy of information by integrating video data from different sources. The diversity of video data sources and the inconsistency of data formats make the integration and utilization of multi-source heterogeneous video resources complicated. The system can effectively manage and integrate these heterogeneous video resources through a unified data feature extraction and processing method. It can be applied to intelligent monitoring, media analysis, education and training, etc., which can improve the efficiency and accuracy of decision support and has significant practical significance.
[0003] The existing multi-source heterogeneous video resource integration method and system based on data features have a large load on the integration system when integrating multi-source heterogeneous videos due to the diversity of video data sources and inconsistency of data formats in multi-source heterogeneous video resources. There are bottlenecks in computing resources and storage resources, which are prone to integration failures and video resource loss. At the same time, in the process of format conversion of video resources, it is easy for the uploaded video to be in the wrong format or incompatible after integration. Therefore, it is necessary to propose a multi-source heterogeneous video resource integration method and system based on data features to solve the above-mentioned problems. Summary of the invention
[0004] In order to solve the above technical problems, a method and system for integrating multi-source heterogeneous video resources based on data features are provided. The technical solution solves the problem that the existing method and system for integrating multi-source heterogeneous video resources based on data features proposed in the above background technology have a heavy load when integrating multi-source heterogeneous videos, and are prone to integration failure and video resource loss. At the same time, when converting the format of video resources, video encoding and decoding are prone to errors, resulting in the uploaded video being in the wrong format or incompatible after integration.
[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is:
[0006] A method for integrating multi-source heterogeneous video resources based on data features, comprising:
[0007] Acquire task category information, collect multi-source heterogeneous video resource data, and determine collection time information and video resource type information, wherein the task category information is task feature information that needs to be processed by the integration system, and the video resource type information includes dynamic video resource information and static video resource information;
[0008] Segment the multi-source heterogeneous video resource data to obtain multi-source heterogeneous video resource sub-data, obtain video attribute parameters of each multi-source heterogeneous video resource sub-data respectively, evaluate the video attribute parameters, and obtain format conversion parameters;
[0009] Performing format conversion on the multi-source heterogeneous video resource sub-data based on the format conversion parameters to obtain the selected multi-source heterogeneous video resource sub-data, and extracting key feature factors from each of the selected multi-source heterogeneous video resource sub-data;
[0010] Each candidate multi-source heterogeneous video resource sub-data is evaluated based on the key feature factor to obtain a key feature index, and the candidate multi-source heterogeneous video resource sub-data is screened based on the key feature index to obtain the preferred multi-source heterogeneous video resource sub-data;
[0011] Perform video integration on the preferred multi-source heterogeneous video resource sub-data to obtain comprehensive video resource data, and set index information based on key feature factors of each preferred multi-source heterogeneous video resource sub-data;
[0012] The comprehensive video resource data is classified based on the index information, and the classified comprehensive video resource data is optimized.
[0013] In an optional embodiment, the segmenting of the multi-source heterogeneous video resource data to obtain the multi-source heterogeneous video resource sub-data specifically includes:
[0014] Based on the acquisition time information and the video resource type information, the multi-source heterogeneous video resource data is classified to obtain dynamic video resource data and static video resource data;
[0015] Extracting dynamic video position sequence information and static video position sequence information from dynamic video resource data and static video resource data respectively;
[0016] Compare the dynamic video position sequence information and the static video position sequence information, obtain common position sequence information, and sort the dynamic video resource data and the static video resource data;
[0017] Setting time segmentation parameters based on the common position sequence information and the sorted dynamic video resource data and static video resource data;
[0018] Segmenting the sorted dynamic video resource data and static video resource data based on the time segmentation parameter to obtain corresponding dynamic video resource segmentation data and static video resource segmentation data;
[0019] Integrate the corresponding dynamic video resource segmentation data and static video resource segmentation data to obtain multi-source heterogeneous video resource sub-data, and establish a multi-source heterogeneous video resource sub-database based on the common position sequence information;
[0020] The first column of data in the multi-source heterogeneous video resource sub-database is common position sequence information, the second column of data is dynamic video resource segmentation data, and the third column of data is static video resource segmentation data.
[0021] In an optional embodiment, the step of respectively obtaining the video attribute parameters of each multi-source heterogeneous video resource sub-data and evaluating the video attribute parameters to obtain the format conversion parameters specifically includes:
[0022] Obtaining video metadata of each multi-source heterogeneous video resource sub-data in the multi-source heterogeneous video resource sub-database, and determining video attribute parameters and video attribute parameter type information corresponding to each multi-source heterogeneous video resource sub-data;
[0023] Classifying the video attribute parameters based on the video attribute parameter type information to obtain video attribute basic parameters, video attribute encoding parameters, video attribute encoding parameters, video color attribute parameters, container format parameters and time attribute parameters;
[0024] Set format conversion requirement information, and filter the classified video attribute parameters to obtain video attribute conversion parameters;
[0025] Rounding each video attribute parameter to be converted to an integer to obtain an integer value of the video attribute conversion parameter;
[0026] The mode value of the video attribute conversion parameter integer value corresponding to each video attribute parameter is determined, and the mode value of the video attribute conversion parameter integer value is used as the format conversion parameter corresponding to each video attribute parameter.
[0027] In an optional embodiment, the format conversion of the multi-source heterogeneous video resource sub-data based on the format conversion parameter to obtain the selected multi-source heterogeneous video resource sub-data, and extracting the key feature factor from each of the selected multi-source heterogeneous video resource sub-data specifically includes:
[0028] Based on the format conversion parameters, n threads are set, and each thread only processes a single task;
[0029] Evaluate the video attribute parameters corresponding to the sub-data of multi-source heterogeneous video resources, and sequentially put the sub-data of multi-source heterogeneous video resources into the corresponding threads among n threads for format conversion;
[0030] The multi-source heterogeneous video resource sub-data after all format conversions are stored to obtain the multi-source heterogeneous video resource sub-data to be selected;
[0031] Perform frame extraction and audio extraction on the video in the selected multi-source heterogeneous video resource sub-data to obtain multi-source heterogeneous frame image data and multi-source heterogeneous audio track data;
[0032] Perform visual feature extraction and audio feature extraction on multi-source heterogeneous frame image data and multi-source heterogeneous audio track data respectively to obtain image feature factors and audio feature factors;
[0033] The image feature factors and the audio feature factors are screened based on the task category information, and the screened image feature factors and the audio feature factors are reduced in dimension to obtain the image feature key factors and the audio feature key factors;
[0034] The image feature key factors and the audio feature key factors are integrated to obtain the key feature factors.
[0035] In an optional embodiment, the step of evaluating each candidate multi-source heterogeneous video resource sub-data based on the key feature factor to obtain a key feature index, and screening the candidate multi-source heterogeneous video resource sub-data based on the key feature index to obtain the preferred multi-source heterogeneous video resource sub-data specifically includes:
[0036] The key feature factors corresponding to each of the selected multi-source heterogeneous video resource sub-data are input into the key feature evaluation model to obtain the key feature index;
[0037] Determine a key feature index threshold based on the task category information and the key feature evaluation model, and evaluate the key feature index corresponding to each candidate multi-source heterogeneous video resource sub-data based on the key feature index threshold, and record the evaluation result;
[0038] Based on the evaluation results, the multi-source heterogeneous video resource sub-data to be selected are screened to obtain the preferred multi-source heterogeneous video resource sub-data.
[0039] In an optional embodiment, the step of performing video integration on the preferred multi-source heterogeneous video resource sub-data to obtain comprehensive video resource data and setting index information based on key characteristic factors of each preferred multi-source heterogeneous video resource sub-data specifically includes:
[0040] Based on the common position sequence information, the videos in the preferred multi-source heterogeneous video resource sub-data are spatially synchronized, and the preferred multi-source heterogeneous videos are merged to obtain a comprehensive video;
[0041] Converting the image feature key factors and the audio feature key factors into text information, and associating the text information with the corresponding preferred multi-source heterogeneous videos, and displaying them in the comprehensive video;
[0042] Combine the comprehensive video and text information to obtain comprehensive video resource data;
[0043] The key characteristic factors of each preferred multi-source heterogeneous video resource sub-data are format-converted to obtain index information.
[0044] Furthermore, a multi-source heterogeneous video resource integration system based on data features is proposed, which is used to implement any of the above integration methods, including:
[0045] An acquisition module, the acquisition module is used to acquire task category information, collect multi-source heterogeneous video resource data, dynamic video resources and static video resources, and store them in a video storage unit;
[0046] A main control module, the main control module is used to obtain multi-source heterogeneous video resource data from a video storage unit, and transmit the multi-source heterogeneous video resource data to a video integration unit, segment the multi-source heterogeneous video resource data to obtain multi-source heterogeneous video resource sub-data, perform format conversion on the multi-source heterogeneous video resource sub-data based on a format conversion parameter to obtain selected multi-source heterogeneous video resource sub-data, extract key feature factors from each of the selected multi-source heterogeneous video resource sub-data, and send the key feature factors to a key feature evaluation unit, perform video integration on the preferred multi-source heterogeneous video resource sub-data to obtain comprehensive video resource data, and set index information based on the key feature factors of each preferred multi-source heterogeneous video resource sub-data, classify the comprehensive video resource data, and send the classified comprehensive video resource data to a display module for interface optimization, screen the segmented multi-source heterogeneous video resource data to obtain selected multi-source heterogeneous video resource sub-data, and screen the selected multi-source heterogeneous video resource sub-data to obtain preferred multi-source heterogeneous video resource sub-data;
[0047] A transmission module, which is used for data transmission between the acquisition module, the main control module, the evaluation module and the display module;
[0048] An evaluation module, the evaluation module is used to evaluate the video attribute parameters, obtain the format conversion parameters, and send the format conversion parameters to the judgment unit, and evaluate each to-be-selected multi-source heterogeneous video resource sub-data based on the key feature factor to obtain the key feature index, and send the key feature index to the judgment unit;
[0049] A display module is used to display a user control interface and a comprehensive video after the interface is optimized.
[0050] In an optional embodiment, the main control module includes:
[0051] A data receiving unit, the data receiving unit is used to generate key information and send it to the video storage unit, obtain multi-source heterogeneous video resource data from the video storage unit, and transmit the multi-source heterogeneous video resource data to the video integration unit;
[0052] A judgment unit, the judgment unit is used to screen the segmented multi-source heterogeneous video resource data to obtain to-be-selected multi-source heterogeneous video resource sub-data, and to screen the to-be-selected multi-source heterogeneous video resource sub-data to obtain preferred multi-source heterogeneous video resource sub-data;
[0053] A video integration unit, wherein the video integration unit is used to segment multi-source heterogeneous video resource data to obtain multi-source heterogeneous video resource sub-data, perform format conversion on the multi-source heterogeneous video resource sub-data based on format conversion parameters to obtain selected multi-source heterogeneous video resource sub-data, extract key feature factors from each of the selected multi-source heterogeneous video resource sub-data, and send the key feature factors to a key feature evaluation unit, perform video integration on the preferred multi-source heterogeneous video resource sub-data to obtain comprehensive video resource data, and set index information based on the key feature factors of each preferred multi-source heterogeneous video resource sub-data, classify the comprehensive video resource data, and send the classified comprehensive video resource data to a display module for interface optimization.
[0054] In an optional embodiment, the acquisition module includes:
[0055] A video acquisition unit, the video acquisition unit is used to collect multi-source heterogeneous video resource data, dynamic video resources and static video resources, and store them in a video storage unit;
[0056] The video storage unit is used to store the video data collected by the video collection unit, screen the key information sent by the video receiving unit and transmit the multi-source heterogeneous video resource data to the data receiving unit.
[0057] In an optional embodiment, the evaluation module includes:
[0058] A format conversion evaluation unit, the format conversion evaluation unit is used to evaluate the video attribute parameters, obtain format conversion parameters, and send the format conversion parameters to the determination unit;
[0059] A key feature evaluation unit evaluates each to-be-selected multi-source heterogeneous video resource sub-data based on the key feature factor to obtain a key feature index, and sends the key feature index to the judgment unit.
[0060] Compared with the prior art, the present invention has the following beneficial effects:
[0061] This scheme proposes a multi-source heterogeneous video resource integration method and system based on data features, which divides the sorted dynamic video resource data and static video resource data through time segmentation parameters to avoid the integration system integrating videos for a long time, affecting the integration efficiency. Through format conversion parameters, the format of multi-source heterogeneous videos is unified to avoid the large integration load and bottlenecks in computing resources and storage resources caused by the diversity of video data sources and the inconsistency of data formats during integration. At the same time, it also avoids the situation where the video uploaded after integration has misaligned formats or is incompatible. Based on the format conversion parameters, n threads are set, and each thread only processes a single task. The video attribute parameters of each multi-source heterogeneous video are processed separately by using n threads, and each thread only processes one task, which improves processing efficiency, simplifies task allocation and management, and improves processing accuracy and stability. At the same time, resources can be allocated according to the needs of each task to avoid resource waste. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] Figure 1 A flowchart of a method for integrating multi-source heterogeneous video resources based on data features proposed by the present invention;
[0063] Figure 2 It is a flowchart of obtaining sub-data of multi-source heterogeneous video resources in the present invention;
[0064] Figure 3 This is a flowchart for extracting key characteristic factors in the present invention;
[0065] Figure 4 This is a system framework diagram of a multi-source heterogeneous video resource integration system based on data features proposed by the present invention. DETAILED DESCRIPTION
[0066] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art may think of other obvious variations.
[0067] Reference Figure 1 - Figure 4 As shown, a method for integrating multi-source heterogeneous video resources based on data features includes:
[0068] Obtain task category information, collect multi-source heterogeneous video resource data, and determine the collection time information and video resource type information. The task category information is the task feature information that the integration system needs to process. The video resource type information includes dynamic video resource information and static video resource information.
[0069] Segment the multi-source heterogeneous video resource data to obtain multi-source heterogeneous video resource sub-data, obtain video attribute parameters of each multi-source heterogeneous video resource sub-data respectively, evaluate the video attribute parameters, and obtain format conversion parameters;
[0070] Performing format conversion on the multi-source heterogeneous video resource sub-data based on the format conversion parameters to obtain the selected multi-source heterogeneous video resource sub-data, and extracting key feature factors from each of the selected multi-source heterogeneous video resource sub-data;
[0071] Each candidate multi-source heterogeneous video resource sub-data is evaluated based on the key feature factor to obtain a key feature index, and the candidate multi-source heterogeneous video resource sub-data is screened based on the key feature index to obtain the preferred multi-source heterogeneous video resource sub-data;
[0072] Perform video integration on the preferred multi-source heterogeneous video resource sub-data to obtain comprehensive video resource data, and set index information based on key feature factors of each preferred multi-source heterogeneous video resource sub-data;
[0073] The comprehensive video resource data is classified based on the index information, and the classified comprehensive video resource data is optimized.
[0074] Specifically, the task category information is the task feature information that the integration system needs to process. The integration system can set different integration tasks according to different industries or different application scenarios, collect video data of the industry or the application scenario, and obtain corresponding multi-source heterogeneous video resource data. For example, if the integration system is applied to the water conservancy industry, the multi-source heterogeneous video resource data includes surveillance videos of reservoirs and other areas, drone videos, and large-scale surface data videos provided by satellites. If the integration system is applied to scenes such as shopping malls, the multi-source heterogeneous video resource data includes shopping mall surveillance videos, security videos, etc. Video resource type information includes dynamic video resource information and static video resource information. Among them, the dynamic video resources in the dynamic video resource information include surveillance camera videos, drone videos, and emergency response videos, which are mainly used for real-time data acquisition and immediate response. Static video resources include satellite videos, historical archive videos, and construction and maintenance videos, which are mainly used for long-term monitoring, historical records, and analysis.
[0075] Furthermore, the multi-source heterogeneous video resource data is segmented to obtain multi-source heterogeneous video resource sub-data, specifically including:
[0076] Based on the acquisition time information and the video resource type information, the multi-source heterogeneous video resource data is classified to obtain dynamic video resource data and static video resource data;
[0077] Extracting dynamic video position sequence information and static video position sequence information from dynamic video resource data and static video resource data respectively;
[0078] Compare the dynamic video position sequence information and the static video position sequence information, obtain common position sequence information, and sort the dynamic video resource data and the static video resource data;
[0079] Setting time segmentation parameters based on the common position sequence information and the sorted dynamic video resource data and static video resource data;
[0080] Segmenting the sorted dynamic video resource data and static video resource data based on the time segmentation parameter to obtain corresponding dynamic video resource segmentation data and static video resource segmentation data;
[0081] Integrate the corresponding dynamic video resource segmentation data and static video resource segmentation data to obtain multi-source heterogeneous video resource sub-data, and establish a multi-source heterogeneous video resource sub-database based on the common position sequence information;
[0082] Among them, the first column of data in the multi-source heterogeneous video resource sub-database is the common position sequence information, the second column of data is the dynamic video resource segmentation data, and the third column of data is the static video resource segmentation data.
[0083] Specifically, based on the acquisition time information and video resource type information, the multi-source heterogeneous video resource data is classified to obtain dynamic video resource data and static video resource data. The acquisition time information is the system time of the integrated system for acquiring multi-source heterogeneous video resource data. The dynamic video position sequence information and the static video position sequence information are the geographic location information in the acquired dynamic video and the geographic location information in the static video, respectively. Compare the dynamic video position sequence information and the static video position sequence information, obtain the common position sequence information, and sort the dynamic video resource data and the static video resource data. Sorting the dynamic video resource data and the static video resource data is to sort them according to their geographic location information. At the same time, when collecting dynamic video resource data and static video resource data, it is necessary to collect dynamic videos and static videos with closer time information, so that the dynamic video resource data and the static video resource data have more analytical significance. A multi-source heterogeneous video resource sub-database is established based on the common position sequence information. The establishment process of the multi-source heterogeneous video resource sub-database also needs to consider the time information in the collected dynamic video resource data and static video resource data and the collection time information of the multi-source heterogeneous video resource data. The common position sequence information is the geographic location information when the geographic location information in the collected dynamic video and the geographic location information in the static video are the same.
[0084] It can be understood that the sorted dynamic video resource data and static video resource data are segmented based on the time segmentation parameter to obtain corresponding dynamic video resource segmentation data and static video resource segmentation data, wherein the time segmentation parameter is the total duration required for similar events occurring at the same location shown in the dynamic video and the static video, and the value of this total duration is used as the time segmentation parameter to perform video segmentation on the dynamic video resource data and the static video resource data. Integrating the corresponding dynamic video resource segmentation data and static video resource segmentation data to obtain multi-source heterogeneous video resource sub-data is to merge the dynamic video resource segmentation data and the static video resource segmentation data.
[0085] Furthermore, the video attribute parameters of each multi-source heterogeneous video resource sub-data are obtained respectively, and the video attribute parameters are evaluated to obtain format conversion parameters, which specifically include:
[0086] Obtaining video metadata of each multi-source heterogeneous video resource sub-data in the multi-source heterogeneous video resource sub-database, and determining video attribute parameters and video attribute parameter type information corresponding to each multi-source heterogeneous video resource sub-data;
[0087] Classifying the video attribute parameters based on the video attribute parameter type information to obtain video attribute basic parameters, video attribute encoding parameters, video attribute encoding parameters, video color attribute parameters, container format parameters and time attribute parameters;
[0088] Set format conversion requirement information, and filter the classified video attribute parameters to obtain video attribute conversion parameters;
[0089] Rounding each video attribute parameter to be converted to an integer to obtain an integer value of the video attribute conversion parameter;
[0090] The mode value of the video attribute conversion parameter integer value corresponding to each video attribute parameter is determined, and the mode value of the video attribute conversion parameter integer value is used as the format conversion parameter corresponding to each video attribute parameter.
[0091] Specifically, video metadata includes basic information of multi-source heterogeneous videos (video ID, etc.), time information (shooting, creation time, etc.), technical specification information (resolution, frame rate, etc.), etc., and determining the video attribute parameters and video attribute parameter type information corresponding to each multi-source heterogeneous video resource sub-data is a process of extracting and quantifying the data belonging to the video technical specification information and time information in the video metadata. For example, if the resolution in the video attribute parameters of the first multi-source heterogeneous video in the multi-source heterogeneous video resource sub-database is (such as 1920*1080, etc.), frame rate (such as 30FPS), then the basic video attribute parameters are + ,in, and They are all weight coefficients, which can be set according to the needs of the integration system, and the value range is 0 to 1. Set the format conversion requirement information, and filter the classified video attribute parameters to obtain the video attribute conversion parameters. Because the integration system does not need to convert all the video attribute parameters when converting the format of multi-source heterogeneous videos, it is necessary to filter them to obtain the video attribute conversion parameters. Therefore, the video attribute conversion parameters are the parameter values that the multi-source heterogeneous videos need to reach when the integration system needs to convert the format of multi-source heterogeneous videos. For example, if the frame rate of the multi-source heterogeneous videos is 30, and the mode value of the video attribute conversion parameter corresponding to each video attribute parameter is 50, the multi-source heterogeneous videos need to be interpolated to make their frame rate reach 50.
[0092] Further, based on the format conversion parameters, the multi-source heterogeneous video resource sub-data is format converted to obtain the selected multi-source heterogeneous video resource sub-data, and key feature factors are extracted from each of the selected multi-source heterogeneous video resource sub-data, specifically including:
[0093] Based on the format conversion parameters, n threads are set, and each thread only processes a single task;
[0094] Evaluate the video attribute parameters corresponding to the sub-data of multi-source heterogeneous video resources, and sequentially put the sub-data of multi-source heterogeneous video resources into the corresponding threads among n threads for format conversion;
[0095] The multi-source heterogeneous video resource sub-data after all format conversions are stored to obtain the multi-source heterogeneous video resource sub-data to be selected;
[0096] Perform frame extraction and audio extraction on the video in the selected multi-source heterogeneous video resource sub-data to obtain multi-source heterogeneous frame image data and multi-source heterogeneous audio track data;
[0097] Perform visual feature extraction and audio feature extraction on multi-source heterogeneous frame image data and multi-source heterogeneous audio track data respectively to obtain image feature factors and audio feature factors;
[0098] The image feature factors and the audio feature factors are screened based on the task category information, and the screened image feature factors and the audio feature factors are reduced in dimension to obtain the image feature key factors and the audio feature key factors;
[0099] The image feature key factors and the audio feature key factors are integrated to obtain the key feature factors.
[0100] Specifically, based on the format conversion parameters, n threads are set, and each thread only processes a single task. If the integrated system needs to convert the format of multi-source heterogeneous videos to m parameters, then n=2m. For example, if the video attribute parameters that need to be format converted for a multi-source heterogeneous video include video attribute basic parameters, video attribute encoding parameters, and video attribute encoding parameters, then 6 threads need to be set. Since the multi-source heterogeneous video may be higher or lower than the video attribute basic parameters in the format conversion parameters, it is necessary to increase or decrease the video attribute basic parameters of the multi-source heterogeneous video.
[0101] It can be understood that if the basic video attribute parameters of a multi-source heterogeneous video are greater than the format conversion parameters, it is necessary to reduce its basic video attribute parameters, and if its video attribute encoding parameters are less than the format conversion parameters, it is necessary to increase its video attribute encoding parameters. For example, if the frame rate of a multi-source heterogeneous video M is 90, its encoding parameters are 264, and the frame rate in the format conversion parameters is 60, and the encoding parameters are 265, then it is necessary to perform frame insertion and transcoding on this multi-source heterogeneous video M. If the frame rate of another multi-source heterogeneous video N is 120 and its encoding parameters are 266, then it is necessary to perform frame reduction (frame rate sampling) and transcoding on N. In this way, it is necessary to set 4 threads to perform format conversion on multi-source heterogeneous videos M and N respectively. At the same time, among these 4 threads, if one of the threads only performs frame insertion, then this thread will only perform frame insertion tasks. If one of the multi-source heterogeneous videos is performing frame insertion, other multi-source heterogeneous videos need to perform other tasks.
[0102] Further, each candidate multi-source heterogeneous video resource sub-data is evaluated based on the key feature factor to obtain a key feature index, and the candidate multi-source heterogeneous video resource sub-data is screened based on the key feature index to obtain the preferred multi-source heterogeneous video resource sub-data, specifically including:
[0103] The key feature factors corresponding to each of the selected multi-source heterogeneous video resource sub-data are input into the key feature evaluation model to obtain the key feature index;
[0104] Determine a key feature index threshold based on the task category information and the key feature evaluation model, and evaluate the key feature index corresponding to each candidate multi-source heterogeneous video resource sub-data based on the key feature index threshold, and record the evaluation result;
[0105] Based on the evaluation results, the multi-source heterogeneous video resource sub-data to be selected are screened to obtain the preferred multi-source heterogeneous video resource sub-data.
[0106] Further, the preferred multi-source heterogeneous video resource sub-data are integrated to obtain comprehensive video resource data, and index information is set based on key characteristic factors of each preferred multi-source heterogeneous video resource sub-data, specifically including:
[0107] Based on the common position sequence information, the videos in the preferred multi-source heterogeneous video resource sub-data are spatially synchronized, and the preferred multi-source heterogeneous videos are merged to obtain a comprehensive video;
[0108] Converting the image feature key factors and the audio feature key factors into text information, and associating the text information with the corresponding preferred multi-source heterogeneous videos, and displaying them in the comprehensive video;
[0109] Combine the comprehensive video and text information to obtain comprehensive video resource data;
[0110] The key characteristic factors of each preferred multi-source heterogeneous video resource sub-data are format-converted to obtain index information.
[0111] Specifically, the index information is information set to facilitate staff to query the video. Visual feature extraction and audio feature extraction are performed on the multi-source heterogeneous frame image data and the multi-source heterogeneous audio track data respectively to obtain image feature factors and audio feature factors. The image feature key factor is the characteristic picture information presented at a fixed time interval or specific conditions (such as scene changes), and the audio feature key factor is the audio track information of the characteristic sound emitted at a fixed time interval or specific conditions.
[0112] The process of extracting visual feature factors is as follows: extracting the color histogram of the frame and converting the color space to obtain color feature information; extracting texture feature information by using Gabor filter, wavelet transform, etc.; obtaining shape information by using edge detection (such as Canny edge detection) and contour extraction; using deep learning models (such as YOLO and Faster R-CNN) to detect and identify objects in the frame and obtain target feature information; merging color feature information, texture feature information, shape information and target feature information to obtain visual feature factors;
[0113] The process of extracting audio feature factors is as follows: using Mel-frequency cepstral coefficients (MFCCs) to capture the short-time power spectrum characteristics of the audio signal, using an audio analyzer to analyze the fundamental frequency, volume and other characteristics of the audio, and combining the short-time power spectrum characteristics with the fundamental frequency, volume and other characteristics to obtain the audio feature factors;
[0114] Image feature factors and audio feature factors are screened based on task category information, and the screened image feature factors and audio feature factors are reduced in dimension to obtain image feature key factors and audio feature key factors. Image feature key factors and audio feature key factors are key information that can be analyzed and warned in multi-source heterogeneous video resource data. For example, in the water conservancy industry, key features in the video include abnormal water flow, structural damage, etc., while key features in the audio include noise levels, voice commands, equipment alarms, etc.
[0115] The steps to establish the key feature evaluation model include:
[0116] S1: Collect a large amount of known multi-source heterogeneous video resource data and annotate the key features to obtain annotated data;
[0117] S2: Through machine learning, we use the labeled data to train a known standard image recognition model to obtain a key feature evaluation model;
[0118] S3: Combine specific algorithm optimization techniques, such as using optimization algorithms (such as genetic algorithms, Bayesian optimization, etc.) to adjust model parameters to improve the evaluation accuracy of the key feature evaluation model.
[0119] It can be understood that the key feature evaluation model is an image detection or recognition model, which can detect or recognize key features in the video and evaluate them, and convert image feature key factors and audio feature key factors into text information, and associate the text information with the corresponding preferred multi-source heterogeneous video and display it in the comprehensive video. If it is in the water conservancy industry, these text information are displayed as abnormal water flow, abnormal noise, etc. The key feature index threshold is determined based on the task category information and the key feature evaluation model, and the key feature index corresponding to each candidate multi-source heterogeneous video resource sub-data is evaluated based on the key feature index threshold, and the evaluation result is recorded, wherein, if the key feature index corresponding to the candidate multi-source heterogeneous video resource sub-data is greater than or equal to the key feature index threshold, the candidate multi-source heterogeneous video resource sub-data is marked, and if the key feature index corresponding to the candidate multi-source heterogeneous video resource sub-data is less than the key feature index threshold, the candidate multi-source heterogeneous video resource sub-data is not marked. Taking abnormal and normal as an example, if the key feature index is greater than or equal to the key feature index threshold, it means that the multi-source heterogeneous video in the candidate multi-source heterogeneous video resource sub-data has an abnormal phenomenon.
[0120] Furthermore, a multi-source heterogeneous video resource integration system based on data features is proposed, which is used to implement any of the above integration methods, including:
[0121] An acquisition module, which is used to acquire task category information, collect multi-source heterogeneous video resource data, dynamic video resources and static video resources, and store them in a video storage unit;
[0122] A main control module, the main control module is used to obtain multi-source heterogeneous video resource data from a video storage unit, and transmit the multi-source heterogeneous video resource data to a video integration unit, segment the multi-source heterogeneous video resource data to obtain multi-source heterogeneous video resource sub-data, perform format conversion on the multi-source heterogeneous video resource sub-data based on a format conversion parameter to obtain selected multi-source heterogeneous video resource sub-data, extract key feature factors from each of the selected multi-source heterogeneous video resource sub-data, and send the key feature factors to a key feature evaluation unit, perform video integration on the preferred multi-source heterogeneous video resource sub-data to obtain comprehensive video resource data, and set index information based on the key feature factors of each preferred multi-source heterogeneous video resource sub-data, classify the comprehensive video resource data, and send the classified comprehensive video resource data to a display module for interface optimization, screen the segmented multi-source heterogeneous video resource data to obtain selected multi-source heterogeneous video resource sub-data, and screen the selected multi-source heterogeneous video resource sub-data to obtain preferred multi-source heterogeneous video resource sub-data;
[0123] A transmission module, which is used for data transmission between the acquisition module, the main control module, the evaluation module and the display module;
[0124] An evaluation module, the evaluation module is used to evaluate the video attribute parameters, obtain the format conversion parameters, and send the format conversion parameters to the judgment unit, and evaluate each to-be-selected multi-source heterogeneous video resource sub-data based on the key feature factor to obtain the key feature index, and send the key feature index to the judgment unit;
[0125] Display module, the display module is used to display the user control interface and the comprehensive video after the interface is optimized.
[0126] Furthermore, the main control module includes:
[0127] A data receiving unit, the data receiving unit is used to generate key information and send it to the video storage unit, obtain multi-source heterogeneous video resource data from the video storage unit, and transmit the multi-source heterogeneous video resource data to the video integration unit;
[0128] A judgment unit, the judgment unit is used to screen the segmented multi-source heterogeneous video resource data to obtain to-be-selected multi-source heterogeneous video resource sub-data, and to screen the to-be-selected multi-source heterogeneous video resource sub-data to obtain preferred multi-source heterogeneous video resource sub-data;
[0129] The video integration unit is used to segment the multi-source heterogeneous video resource data to obtain multi-source heterogeneous video resource sub-data, perform format conversion on the multi-source heterogeneous video resource sub-data based on format conversion parameters to obtain selected multi-source heterogeneous video resource sub-data, extract key feature factors from each of the selected multi-source heterogeneous video resource sub-data, and send the key feature factors to the key feature evaluation unit, perform video integration on the preferred multi-source heterogeneous video resource sub-data to obtain comprehensive video resource data, and set index information based on the key feature factors of each preferred multi-source heterogeneous video resource sub-data, classify the comprehensive video resource data, and send the classified comprehensive video resource data to the display module for interface optimization.
[0130] Furthermore, the acquisition module includes:
[0131] The video acquisition unit is used to collect multi-source heterogeneous video resource data, dynamic video resources and static video resources, and store them in the video storage unit;
[0132] The video storage unit is used to store the video data collected by the video acquisition unit, screen the key information sent by the video receiving unit and transmit the multi-source heterogeneous video resource data to the data receiving unit.
[0133] Furthermore, the assessment modules include:
[0134] A format conversion evaluation unit, the format conversion evaluation unit is used to evaluate the video attribute parameters, obtain format conversion parameters, and send the format conversion parameters to the determination unit;
[0135] The key feature evaluation unit evaluates each candidate multi-source heterogeneous video resource sub-data based on the key feature factor to obtain a key feature index, and sends the key feature index to the judgment unit.
[0136] In summary, the advantages of the present invention are: setting time segmentation parameters based on common position sequence information and sorted dynamic video resource data and static video resource data, by using the common position sequence information, the video can be time segmented more accurately, so as to better capture the time point and position information of the event, combining dynamic and static video data, it can more effectively manage and allocate processing resources, and adjust the time segmentation strategy according to actual needs;
[0137] The sorted dynamic video resource data and static video resource data are divided through the time segmentation parameters to avoid the integration system integrating videos for a long time, which affects the integration efficiency. The format of multi-source heterogeneous videos is unified through the format conversion parameters to avoid the diversity of video data sources and the inconsistency of data formats during integration, which leads to a large integration load and bottlenecks in computing resources and storage resources. At the same time, it also avoids the situation where the video uploaded after integration has a misaligned format or is incompatible. Based on the format conversion parameters, n threads are set, and each thread only processes a single task. The video attribute parameters of each multi-source heterogeneous video are processed separately using n threads, and each thread only processes one task, which improves processing efficiency, simplifies task allocation and management, and improves processing accuracy and stability. At the same time, resources can be allocated according to the needs of each task to avoid resource waste.
[0138] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.
Claims
1. A method for integrating multi-source heterogeneous video resources based on data features, characterized in that: include: Acquire task category information, collect multi-source heterogeneous video resource data, and determine collection time information and video resource type information, wherein the task category information is task feature information that needs to be processed by the integration system, and the video resource type information includes dynamic video resource information and static video resource information; Segment the multi-source heterogeneous video resource data to obtain multi-source heterogeneous video resource sub-data, obtain video attribute parameters of each multi-source heterogeneous video resource sub-data respectively, evaluate the video attribute parameters, and obtain format conversion parameters; Performing format conversion on the multi-source heterogeneous video resource sub-data based on the format conversion parameters to obtain the selected multi-source heterogeneous video resource sub-data, and extracting key feature factors from each of the selected multi-source heterogeneous video resource sub-data; Each candidate multi-source heterogeneous video resource sub-data is evaluated based on the key feature factor to obtain a key feature index, and the candidate multi-source heterogeneous video resource sub-data is screened based on the key feature index to obtain the preferred multi-source heterogeneous video resource sub-data; Perform video integration on the preferred multi-source heterogeneous video resource sub-data to obtain comprehensive video resource data, and set index information based on key feature factors of each preferred multi-source heterogeneous video resource sub-data; Classifying the comprehensive video resource data based on the index information, and optimizing the classified comprehensive video resource data; The multi-source heterogeneous video resource data is segmented to obtain the multi-source heterogeneous video resource sub-data, specifically including: Based on the acquisition time information and the video resource type information, the multi-source heterogeneous video resource data is classified to obtain dynamic video resource data and static video resource data; Extracting dynamic video position sequence information and static video position sequence information from dynamic video resource data and static video resource data respectively; Compare the dynamic video position sequence information and the static video position sequence information, obtain common position sequence information, and sort the dynamic video resource data and the static video resource data; Setting time segmentation parameters based on the common position sequence information and the sorted dynamic video resource data and static video resource data; Segmenting the sorted dynamic video resource data and static video resource data based on the time segmentation parameter to obtain corresponding dynamic video resource segmentation data and static video resource segmentation data; Integrate the corresponding dynamic video resource segmentation data and static video resource segmentation data to obtain multi-source heterogeneous video resource sub-data, and establish a multi-source heterogeneous video resource sub-database based on the common position sequence information; The first column of data in the multi-source heterogeneous video resource sub-database is common position sequence information, the second column of data is dynamic video resource segmentation data, and the third column of data is static video resource segmentation data; The step of respectively obtaining the video attribute parameters of each multi-source heterogeneous video resource sub-data and evaluating the video attribute parameters to obtain the format conversion parameters specifically includes: Obtaining video metadata of each multi-source heterogeneous video resource sub-data in the multi-source heterogeneous video resource sub-database, and determining video attribute parameters and video attribute parameter type information corresponding to each multi-source heterogeneous video resource sub-data; Classifying the video attribute parameters based on the video attribute parameter type information to obtain video attribute basic parameters, video attribute encoding parameters, video attribute encoding parameters, video color attribute parameters, container format parameters and time attribute parameters; Set format conversion requirement information, and filter the classified video attribute parameters to obtain video attribute conversion parameters; Rounding each video attribute parameter to be converted to an integer to obtain an integer value of the video attribute conversion parameter; Determine a mode value of the video attribute conversion parameter integer value corresponding to each video attribute parameter, and use the mode value of the video attribute conversion parameter integer value as the format conversion parameter corresponding to each video attribute parameter; The method of converting the format of the multi-source heterogeneous video resource sub-data based on the format conversion parameters to obtain the selected multi-source heterogeneous video resource sub-data and extracting key feature factors from each of the selected multi-source heterogeneous video resource sub-data specifically includes: Based on the format conversion parameters, n threads are set, and each thread only processes a single task; Evaluate the video attribute parameters corresponding to the sub-data of multi-source heterogeneous video resources, and sequentially put the sub-data of multi-source heterogeneous video resources into the corresponding threads among n threads for format conversion; The multi-source heterogeneous video resource sub-data after all format conversions are stored to obtain the multi-source heterogeneous video resource sub-data to be selected; Perform frame extraction and audio extraction on the video in the selected multi-source heterogeneous video resource sub-data to obtain multi-source heterogeneous frame image data and multi-source heterogeneous audio track data; Perform visual feature extraction and audio feature extraction on multi-source heterogeneous frame image data and multi-source heterogeneous audio track data respectively to obtain image feature factors and audio feature factors; The image feature factors and the audio feature factors are screened based on the task category information, and the screened image feature factors and the audio feature factors are reduced in dimension to obtain the image feature key factors and the audio feature key factors; The image feature key factors and the audio feature key factors are integrated to obtain the key feature factors.
2. According to the method of claim 1, the method is characterized in that: The step of evaluating each candidate multi-source heterogeneous video resource sub-data based on the key feature factor to obtain a key feature index, and screening the candidate multi-source heterogeneous video resource sub-data based on the key feature index to obtain the preferred multi-source heterogeneous video resource sub-data specifically includes: The key feature factors corresponding to each of the selected multi-source heterogeneous video resource sub-data are input into the key feature evaluation model to obtain the key feature index; Determine a key feature index threshold based on the task category information and the key feature evaluation model, and evaluate the key feature index corresponding to each candidate multi-source heterogeneous video resource sub-data based on the key feature index threshold, and record the evaluation result; Based on the evaluation results, the multi-source heterogeneous video resource sub-data to be selected are screened to obtain the preferred multi-source heterogeneous video resource sub-data.
3. The method for integrating multi-source heterogeneous video resources based on data features according to claim 2 is characterized in that: The step of integrating the preferred multi-source heterogeneous video resource sub-data to obtain comprehensive video resource data and setting index information based on key characteristic factors of each preferred multi-source heterogeneous video resource sub-data specifically includes: Based on the common position sequence information, the videos in the preferred multi-source heterogeneous video resource sub-data are spatially synchronized, and the preferred multi-source heterogeneous videos are merged to obtain a comprehensive video; Converting the image feature key factors and the audio feature key factors into text information, and associating the text information with the corresponding preferred multi-source heterogeneous videos, and displaying them in the comprehensive video; Combine the comprehensive video and text information to obtain comprehensive video resource data; The key characteristic factors of each preferred multi-source heterogeneous video resource sub-data are format-converted to obtain index information.
4. A multi-source heterogeneous video resource integration system based on data features, used to implement the integration method according to any one of claims 1 to 3, characterized in that: include: An acquisition module, the acquisition module is used to acquire task category information, collect multi-source heterogeneous video resource data, dynamic video resources and static video resources, and store them in a video storage unit; A main control module, the main control module is used to obtain multi-source heterogeneous video resource data from a video storage unit, and transmit the multi-source heterogeneous video resource data to a video integration unit, segment the multi-source heterogeneous video resource data to obtain multi-source heterogeneous video resource sub-data, perform format conversion on the multi-source heterogeneous video resource sub-data based on a format conversion parameter to obtain selected multi-source heterogeneous video resource sub-data, extract key feature factors from each of the selected multi-source heterogeneous video resource sub-data, and send the key feature factors to a key feature evaluation unit, perform video integration on the preferred multi-source heterogeneous video resource sub-data to obtain comprehensive video resource data, and set index information based on the key feature factors of each preferred multi-source heterogeneous video resource sub-data, classify the comprehensive video resource data, and send the classified comprehensive video resource data to a display module for interface optimization, screen the segmented multi-source heterogeneous video resource data to obtain selected multi-source heterogeneous video resource sub-data, and screen the selected multi-source heterogeneous video resource sub-data to obtain preferred multi-source heterogeneous video resource sub-data; A transmission module, which is used for data transmission between the acquisition module, the main control module, the evaluation module and the display module; An evaluation module, the evaluation module is used to evaluate the video attribute parameters, obtain the format conversion parameters, and send the format conversion parameters to the judgment unit, and evaluate each to-be-selected multi-source heterogeneous video resource sub-data based on the key feature factor to obtain the key feature index, and send the key feature index to the judgment unit; A display module is used to display a user control interface and a comprehensive video after the interface is optimized.
5. The multi-source heterogeneous video resource integration system based on data features according to claim 4 is characterized in that: The main control module comprises: A data receiving unit, the data receiving unit is used to generate key information and send it to the video storage unit, obtain multi-source heterogeneous video resource data from the video storage unit, and transmit the multi-source heterogeneous video resource data to the video integration unit; A judgment unit, the judgment unit is used to screen the segmented multi-source heterogeneous video resource data to obtain to-be-selected multi-source heterogeneous video resource sub-data, and to screen the to-be-selected multi-source heterogeneous video resource sub-data to obtain preferred multi-source heterogeneous video resource sub-data; A video integration unit, wherein the video integration unit is used to segment multi-source heterogeneous video resource data to obtain multi-source heterogeneous video resource sub-data, perform format conversion on the multi-source heterogeneous video resource sub-data based on format conversion parameters to obtain selected multi-source heterogeneous video resource sub-data, extract key feature factors from each of the selected multi-source heterogeneous video resource sub-data, and send the key feature factors to a key feature evaluation unit, perform video integration on the preferred multi-source heterogeneous video resource sub-data to obtain comprehensive video resource data, and set index information based on the key feature factors of each preferred multi-source heterogeneous video resource sub-data, classify the comprehensive video resource data, and send the classified comprehensive video resource data to a display module for interface optimization.
6. The multi-source heterogeneous video resource integration system based on data features according to claim 4 is characterized in that: The acquisition module comprises: A video acquisition unit, the video acquisition unit is used to collect multi-source heterogeneous video resource data, dynamic video resources and static video resources, and store them in a video storage unit; The video storage unit is used to store the video data collected by the video collection unit, screen the key information sent by the video receiving unit and transmit the multi-source heterogeneous video resource data to the data receiving unit.
7. The multi-source heterogeneous video resource integration system based on data features according to claim 4 is characterized in that: The evaluation module includes: A format conversion evaluation unit, the format conversion evaluation unit is used to evaluate the video attribute parameters, obtain format conversion parameters, and send the format conversion parameters to the determination unit; A key feature evaluation unit evaluates each to-be-selected multi-source heterogeneous video resource sub-data based on the key feature factor to obtain a key feature index, and sends the key feature index to the judgment unit.
