A video acquisition method and system based on FPGA parallel processing

Abstract

The present invention relates to the field of video acquisition and analysis, and in particular to a video acquisition method and system based on FPGA parallel processing. The present invention analyzes the video parameters of each video data in each time domain segment, analyzes the regularity representation characteristics of the video data, calculates the regularity representation parameters accordingly, sets regularity category labels for the video data transmitted in different time domain segments, and subsequently adopts different analysis and processing methods for the video data according to the regularity labels, including adopting a segmentation form to determine the abnormal time domain segment and a reference period, and comparing the video frames in the reference period to determine whether the video transmission source has an abnormality based on the difference characteristics, and adopting a frame extraction method to analyze the content of the video frame to determine whether the video transmission source has an abnormality. When analyzing a large amount of video data, the present invention considers the regularity of the video data in different time domain segments, adaptively adjusts the analysis and processing method, ensures the analysis reliability, and improves the analysis efficiency.

Description

Technical Field

[0001] The present invention relates to the field of video acquisition and analysis, and in particular to a video acquisition method and system based on FPGA parallel processing. Background Art

[0002] FPGA (Field Programmable Gate Array) is a programmable hardware device widely used in digital signal processing, communications, video processing and other fields. Due to its parallel processing capabilities, FPGA is suitable for analyzing and processing video data transmitted by multiple video sources. For example, it can verify and identify the transmitted video. Especially when there are many video sources to be processed, it is necessary to analyze the transmitted video data to avoid frame insertion, video frame tampering, or transmission anomalies.

[0003] For example, Chinese patent publication number CN111080637A discloses a remote advertising method, device, system, product, and medium based on cloud services. In the method, each distributed monitoring device first performs a preliminary inspection of the image to be played in its corresponding advertising machine, confirms that there is an abnormal image, and sends the abnormal image to the cloud service end. The cloud service end obtains the image to be detected sent by each distributed monitoring device, performs image analysis on the image to be detected, and detects whether the image to be detected is abnormal. If the image to be detected is abnormal, an alarm message is sent to the user end. After the preliminary inspection by each distributed monitoring device, the cloud service end analyzes and monitors the advertising content, overcoming the inefficiency and lag of traditional manual advertising monitoring methods, realizing remote real-time monitoring and content monitoring, and issuing alarms for faulty images or illegal content.

[0004] However, the prior art still has the following problems:

[0005] When analyzing video data transmitted by a large number of video sources, abnormal frames are often identified by analyzing image content. The data to be processed is massive, and the efficiency is low when the data is aggregated for analysis, which consumes a lot of resources. Summary of the Invention

[0006] To this end, the present invention provides a video acquisition method and system based on FPGA parallel processing, which is used to overcome the problems in the prior art of identifying abnormal frames by analyzing image content when analyzing video data transmitted by a large number of video sources. The data required to be processed is massive, the efficiency is low when the data is collected for analysis, and the resource consumption is large.

[0007] To achieve the above objectives, the present invention provides a video acquisition method based on FPGA parallel processing, comprising:

[0008] The video data transmitted by each video transmission source is collected in parallel by the FPGA device;

[0009] Analyzing the video parameters of each of the video data in each time domain segment, and constructing a corresponding time domain dimension curve based on the video parameters to analyze the regularity characterization features of the video data;

[0010] Calculating regularity characterization parameters based on the regularity characterization features to set regularity category labels for video data transmitted by each of the video transmission sources in different time domain segments;

[0011] Analyze and process the video data transmitted by the video transmission source according to the regularity category label, including:

[0012] The video data is segmented into time-domain dimension curves, and the segmented curve segments are fitted with the remaining curve segments to determine the abnormal time-domain segments. The reference period is located, and the corresponding video frames in the abnormal time-domain segments are extracted and compared with the corresponding video frames in the reference period. Based on the comparison results, it is determined whether the video transmission source is abnormal.

[0013] Alternatively, a frame extraction interval is determined based on the regularity characterization parameter, video frames in the video segment data are extracted at the corresponding frame extraction interval, and whether the video transmission source has an abnormality is determined based on each of the extracted video frames;

[0014] The regularity characterization features include pixel regularity characterization features and smoothness regularity characterization features.

[0015] Furthermore, the process of analyzing the regularity characterization features of video data includes:

[0016] Get video parameters, including pixel mean and image gradient;

[0017] Construct the time domain variation curve of pixel mean and image gradient;

[0018] Determine the pixel mean and the autocorrelation coefficient of the image gradient at different time lags;

[0019] Determine the significant peak of the autocorrelation coefficient corresponding to the pixel mean outside the confidence interval, solve the mean of the significant peak, and obtain the pixel regularity characterization feature;

[0020] The significant peak of the autocorrelation coefficient corresponding to the image gradient outside the confidence interval is determined, the mean value of the significant peak is solved, and the smoothing regularity characterization feature is obtained.

[0021] Furthermore, the calculation of regularity characterization parameters includes,

[0022] The pixel regularity characterization feature and the smooth regularity characterization feature are weighted and summed to obtain the regularity characterization parameter.

[0023] Furthermore, the process of setting regularity category labels of the video data transmitted by each of the video transmission sources in different time domain segments includes:

[0024] If the regularity characterization parameter is greater than or equal to a preset regularity standard threshold, determining to set a regularity label for the video data;

[0025] If the regularity characterization parameter is less than a preset regularity standard threshold, it is determined to set an irregularity label for the video data.

[0026] Furthermore, if the video data transmitted by the video transmission source is set with a regularity label, the corresponding video frames in the abnormal time domain segment are extracted and compared with the corresponding video frames in the reference period, and whether the video transmission source is abnormal is determined based on the comparison results;

[0027] If the video data transmitted by the video transmission source is set with a regularity label, it is determined whether the video transmission source is abnormal based on each of the extracted video frames.

[0028] Furthermore, the process of fitting the segmented curve segments with the remaining curve segments includes:

[0029] Analyze the fitting degree of the curve segment corresponding to the pixel mean and the remaining curve segments;

[0030] Analyze the fitting degree of the curve segment corresponding to the image gradient and the remaining curve segments;

[0031] Cluster each curve segment according to its fitting degree;

[0032] Obtaining several clustering sets for pixel means and several clustering sets for image gradients;

[0033] Identify the time domain segment corresponding to each cluster set as the abnormal time domain segment;

[0034] The cluster set must satisfy the following conditions: each curve segment is adjacent and the maximum fitting degree of each curve segment with the remaining curve segments is less than the fitting degree threshold.

[0035] Furthermore, the process of locating the reference period includes,

[0036] Determining a time domain segment adjacent to the abnormal time domain segment and video data within the adjacent time domain segment;

[0037] Compare the time domain curve segments of the video parameters corresponding to the video data in the adjacent time domain segments with the video data in each remaining time domain segment, determine the maximum fitting mean, and determine whether the positioning conditions are met;

[0038] Determine the minimum time endpoint and the maximum time endpoint corresponding to the remaining time domain segment that meets the positioning conditions;

[0039] Determine the time domain segments corresponding to the minimum time endpoint and the maximum time endpoint as the reference period;

[0040] Among them, the positioning condition is that the maximum fitting degree average of the remaining time domain segment and the corresponding adjacent time domain segment is greater than the corresponding fitting degree threshold, and the time interval between the remaining time domain segments is equal to the duration corresponding to the abnormal time domain segment.

[0041] Furthermore, the process of determining whether the video transmission source is abnormal based on the comparison result includes:

[0042] Extract the video frames within the abnormal time domain segment and determine the pixel mean and image gradient mean corresponding to each video frame;

[0043] Determining a reference pixel mean range and an image gradient range based on a reference period;

[0044] If the pixel mean is not within the pixel mean range and / or the image gradient mean is not within the image gradient range, it is determined that an abnormality exists in the video transmission source.

[0045] Furthermore, extracting video frames in the video segment data at a corresponding frame extraction interval to determine whether there is an abnormality in the video transmission source includes:

[0046] Analyze each extracted video frame according to a predetermined image processing model to determine whether abnormal features appear;

[0047] If abnormal features appear, it is determined that the video transmission source is abnormal;

[0048] The determined frame extraction interval is positively correlated with the regularity characterization parameter.

[0049] On the other hand, a system for applying a video acquisition method based on FPGA parallel processing is provided, comprising:

[0050] FPGA collector, which is used to collect video data transmitted by each video transmission source in parallel;

[0051] A feature extractor for analyzing video parameters of the video data in each time domain segment, and constructing a corresponding time domain dimension curve based on the video parameters to analyze regularity characterization features of the video data;

[0052] a label setter, configured to calculate a regularity characterization parameter based on the regularity characterization feature, so as to set a regularity category label for the video data transmitted by each of the video transmission sources in different time domain segments;

[0053] A data analyzer is used to analyze and process the video data transmitted by the video transmission source according to regularity category labels, including:

[0054] It is used to segment the time domain dimension curve corresponding to the video data, fit the segmented curve segments with the remaining curve segments to determine the abnormal time domain segment, and locate the reference period to extract the video frame corresponding to the abnormal time domain segment and compare it with the corresponding video frame in the reference period. Based on the comparison results, it is determined whether there is an abnormality in the video transmission source;

[0055] Alternatively, the method may be used to determine a frame extraction interval based on the regularity characterization parameter, extract video frames from the video segment data corresponding to the frame extraction interval, and determine whether the video transmission source has an abnormality based on each extracted video frame.

[0056] Compared with the prior art, the present invention analyzes the video parameters of each of the video data in each time domain segment, analyzes the regularity characterization features of the video data, calculates the regularity characterization parameters accordingly, sets regularity category labels for the video data transmitted in different time domain segments, and subsequently adopts different analysis and processing methods for the video data according to the regularity labels, including adopting a segmentation form to determine the abnormal time domain segment and the reference period, and comparing the video frames in the reference period, judging whether the video transmission source has an abnormality based on the difference characteristics, and analyzing the content of the video frame by frame extraction to judge whether the video transmission source has an abnormality. When analyzing a large amount of video data, the present invention considers the regularity of the video data in different time domain segments, adaptively adjusts the analysis and processing methods, ensures the reliability of the analysis, and improves the efficiency of the analysis.

[0057] In particular, the present invention specifically extracts regular characterization features of video data in different time domain segments. In actual situations, video transmission sources are mostly used to display advertisements, pictures or real-time videos, and the transmitted video data is diverse. In some cases, the video data transmitted by the video transmission source has strong regularity, such as playing advertisements, maintaining a single screen, displaying surveillance videos in a certain area, etc. Therefore, the present invention considers collecting regular characterization features, including pixel regular characterization features and smoothing regular characterization features, and the regular characterization features are all determined based on video parameters. Video parameters are easy to collect quickly for video frames, and constructing time domain dimension curves also does not consume computing power. Based on this, the pixel regular characterization features are determined to consider the regularity from the overall perspective of the image, and the smoothing regular characterization features are determined to represent the regularity of the video data content from the side. The regular characterization features are further calculated to facilitate the accurate setting of regularity labels. Subsequently, different analysis and processing methods for video data are adopted with the help of the characteristics of regularity, thereby ensuring the reliability of the analysis and improving the efficiency of the analysis when analyzing a large amount of video data.

[0058] In particular, the present invention adopts different analysis and processing methods for different regularity labels. When targeting video data with regularity labels, the corresponding time domain dimension curve is segmented, the abnormal time domain segment is determined, and the reference period is synchronously located. In actual situations, if frame insertion or tampering is performed on video data with regularity, it will be reflected in the video parameter dimension. Compared with using image processing models to analyze video frames, video parameters are easy to collect quickly. Combined with the regularity of video data, the abnormal time domain segment and the reference period can be quickly identified. Since there is a corresponding relationship between the reference period and the abnormal time domain segment, it is possible to quickly analyze whether there is an abnormality in the video transmission source based on this, and then when analyzing a large amount of video data, the analysis reliability is guaranteed and the analysis efficiency is improved.

[0059] In particular, the present invention adopts different analysis and processing methods for different regularity labels. When it comes to video data with irregular labels, the frame extraction interval is adaptively adjusted based on the regularity characterization parameters to reduce feature omissions. Moreover, since this type of data does not have regularity, the frame extraction form is used to traverse and analyze the video frame content through the image processing model to determine whether there is an abnormality in the video transmission source, thereby ensuring the reliability of the analysis and improving the analysis efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 A schematic diagram of the steps of a video acquisition method based on FPGA parallel processing according to an embodiment of the invention;

[0061] Figure 2 A logic block diagram for setting regularity labels for video data according to an embodiment of the invention;

[0062] Figure 3 This is a logic block diagram of analyzing and processing video data transmitted by a video transmission source based on regularity category labels according to an embodiment of the invention;

[0063] Figure 4 This is a logic block diagram of an embodiment of the invention for determining whether a video transmission source is abnormal based on the difference characteristics;

[0064] Figure 5 This is a logic block diagram of an embodiment of the present invention for determining whether the video transmission source is abnormal based on each video frame. DETAILED DESCRIPTION

[0065] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are merely used to explain the present invention and are not intended to limit the present invention.

[0066] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0067] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside", and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0068] Furthermore, it should be noted that, in the description of the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0069] See also Figure 1 As shown in FIG, which is a schematic diagram of the steps of a video acquisition method based on FPGA parallel processing according to an embodiment of the present invention, the video acquisition method based on FPGA parallel processing according to the present invention includes:

[0070] Step S1, collecting video data transmitted by each video transmission source in parallel through an FPGA device;

[0071] Step S2, analyzing the video parameters of each of the video data in each time domain segment, and constructing a corresponding time domain dimension curve based on the video parameters to analyze the regularity characterization features of the video data;

[0072] Step S3, calculating regularity characterization parameters based on the regularity characterization features to set regularity category labels for the video data transmitted by each of the video transmission sources in different time domain segments;

[0073] Step S4, analyzing and processing the video data transmitted by the video transmission source according to the regularity category label, including:

[0074] The video data is segmented into time-domain dimension curves, and the segmented curve segments are fitted with the remaining curve segments to determine the abnormal time-domain segments. The reference period is located, and the corresponding video frames in the abnormal time-domain segments are extracted and compared with the corresponding video frames in the reference period. Based on the comparison results, it is determined whether the video transmission source is abnormal.

[0075] Alternatively, a frame extraction interval is determined based on the regularity characterization parameter, video frames in the video segment data are extracted at the corresponding frame extraction interval, and whether the video transmission source has an abnormality is determined based on each of the extracted video frames;

[0076] The regularity characterization features include pixel regularity characterization features and smoothness regularity characterization features.

[0077] Specifically, in implementation, there is no limitation on the method of collecting video data transmitted by the video transmission source. For example, the FPGA device can be deployed locally, the video data of the video transmission source can be collected through the cloud platform, and analyzed based on the FPGA device.

[0078] Specifically, there is no limitation on the specific form of the FPGA device. The FPGA device is a video processor that can receive multiple video streams. Those skilled in the art can select the model of the FPGA device based on the adaptability of the video data to be processed, which will not be elaborated here.

[0079] Specifically, in the implementation, the video transmission source is a display device that can display video data after being deployed, and the video data to be displayed is synchronously transmitted to the FPGA device for analysis, which will not be repeated here.

[0080] Specifically, the process of analyzing the regularity representation features of video data includes:

[0081] Get video parameters, including pixel mean and image gradient;

[0082] Construct the time domain variation curve of pixel mean and image gradient;

[0083] Determine the pixel mean and the autocorrelation coefficient of the image gradient at different time lags;

[0084] Determine the significant peak of the autocorrelation coefficient corresponding to the pixel mean outside the confidence interval, solve the mean of the significant peak, and obtain the pixel regularity characterization feature;

[0085] The significant peak of the autocorrelation coefficient corresponding to the image gradient outside the confidence interval is determined, the mean value of the significant peak is solved, and the smoothing regularity characterization feature is obtained.

[0086] It can be understood that the pixel mean is the mean of the pixel values ​​in the video frame, and the image gradient is the gradient of the pixel values ​​in the horizontal direction in the video frame, which can be calculated using the Sobel operator or the Prewitt operator.

[0087] The autocorrelation coefficients at different time lags are calculated using the autocorrelation function, which is used to measure the self-similarity, such as periodicity, of a time series at different time lags.

[0088] The formula for the autocorrelation function is as follows,

[0089] ,

[0090] In the formula, R(K) represents the autocorrelation coefficient at time lag k, represents the mean of the observations, N represents the total number of moments, X t represents the observation value at time t, X t+k Represents the observation value at time t+k.

[0091] It is understandable that, in implementation, the observation values ​​may be selected as the pixel mean and the image gradient to calculate the autocorrelation coefficient, which will not be described in detail.

[0092] It is understandable that it is necessary to construct an ACF diagram of the autocorrelation coefficient relative to the lag time, with a confidence interval of 95%. In the ACF diagram, a significant peak will appear at the position where the lag time is equal to the cycle length. The larger the significant peak, the stronger the regularity of the cycle. In practice, a peak value exceeding 0.6 is determined to be a significant peak.

[0093] The present invention specifically extracts regular characterization features of video data in different time domain segments. In actual situations, video transmission sources are mostly used to display advertisements, pictures or real-time videos, and the transmitted video data is diverse. In some cases, the video data transmitted by the video transmission source has strong regularity, such as playing advertisements, maintaining a single screen, displaying surveillance videos in a certain area, etc. Therefore, the present invention considers collecting regularity characterization features, including pixel regularity characterization features and smoothing regularity characterization features, and the regularity characterization features are all determined based on video parameters. Video parameters are convenient for fast acquisition for video frames, and constructing time domain dimension curves also does not consume computing power. Based on this, the pixel regularity characterization features are determined to consider the regularity from the overall perspective of the image, and the smoothing regularity characterization features are determined to characterize the regularity of the video data content from the side. The regularity characterization features are further calculated to facilitate the accurate setting of regularity labels. Subsequently, different analysis and processing methods for video data are adopted with the help of the regularity characteristics, thereby ensuring the reliability of the analysis and improving the efficiency of the analysis when analyzing a large amount of video data.

[0094] Specifically, the calculation regularity characterization parameters include,

[0095] The pixel regularity characterization feature and the smooth regularity characterization feature are weighted and summed to obtain the regularity characterization parameter.

[0096] In the implementation, the weight of the pixel regularity characterization feature is 0.35, and the weight of the smoothness regularity characterization feature is 0.65.

[0097] Specifically, see Figure 2 As shown, Figure 2 This is a logic block diagram of setting regularity labels for video data according to an embodiment of the invention. The process of setting regularity category labels for video data transmitted by each video transmission source in different time domain segments includes:

[0098] If the regularity characterization parameter is greater than or equal to a preset regularity standard threshold, determining to set a regularity label for the video data;

[0099] If the regularity characterization parameter is less than a preset regularity standard threshold, it is determined to set an irregularity label for the video data.

[0100] Generally, when the peak value of a significant peak is greater than 0.6, it has a certain periodic regularity. In practice, in order to indicate a strong regularity, the regularity standard threshold is set to 0.8.

[0101] Specifically, see Figure 3 As shown, Figure 3 This is a logic block diagram of an embodiment of the invention for analyzing and processing video data transmitted by a video transmission source based on regularity category labels. If the video data transmitted by the video transmission source is set with a regularity label, the corresponding video frames in the abnormal time domain segment are extracted and compared with the corresponding video frames in the reference period. Based on the comparison results, it is determined whether the video transmission source has an abnormality.

[0102] If the video data transmitted by the video transmission source is set with a regularity label, it is determined whether the video transmission source is abnormal based on each of the extracted video frames.

[0103] Specifically, the process of fitting the segmented curve segments with the remaining curve segments includes:

[0104] Analyze the fitting degree of the curve segment corresponding to the pixel mean and the remaining curve segments;

[0105] Analyze the fitting degree of the curve segment corresponding to the image gradient and the remaining curve segments;

[0106] Clustering is performed on each curve segment based on its fitting degree. It is understood that when clustering, the curve segments corresponding to the pixel mean and the curve segments corresponding to the image gradient need to be clustered separately.

[0107] Obtaining several clustering sets for pixel means and several clustering sets for image gradients;

[0108] Identify the time domain segment corresponding to each cluster set as the abnormal time domain segment;

[0109] The cluster set must satisfy the following conditions: each curve segment is adjacent and the maximum fitting degree of each curve segment with the remaining curve segments is less than the fitting degree threshold.

[0110] Specifically, when segmenting, each curve segment is ensured to have the same dimension in the time domain as much as possible, which will not be elaborated here.

[0111] Specifically, the degree of fit between the curve segments is calculated using the Euclidean distance method, and the reciprocal of the Euclidean distance corresponding to the curve segments is determined as the degree of fit. The greater the degree of fit, the more fitted the curve segment is, which will not be elaborated here.

[0112] It can be understood that the clustering set includes several clustering sets for pixel means and several clustering sets for image gradients;

[0113] Therefore, for several clustering sets of pixel means, the curve segments must be adjacent and the maximum fitting degree of each curve segment with the remaining curve segments is less than the fitting degree threshold corresponding to the pixel mean;

[0114] Therefore, several clustering sets for image gradients must satisfy the following conditions: each curve segment is adjacent and the maximum fitting degree of each curve segment with the remaining curve segments is less than the fitting degree threshold corresponding to the image gradient;

[0115] In the implementation, the fitting threshold includes the fitting threshold corresponding to the pixel mean and the fitting threshold corresponding to the image gradient, both of which are pre-set.

[0116] Those skilled in the art determine time-domain dimension curves corresponding to several periodic pixel means, segment the time-domain dimension curves corresponding to the pixel means based on the periodicity, obtain several curve segments corresponding to the pixel means, and analyze the mean values ​​of the fit between the curve segments. In implementation, the threshold value of the pixel mean corresponding to the fit is between 0.65 and 0.85 times the mean value of the fit.

[0117] Similarly, the mean value of the degree of fit between the curve segments corresponding to the image gradient is determined, and the threshold value of the degree of fit corresponding to the image gradient is set to between 0.65 and 0.85 times the mean value of the degree of fit.

[0118] It is understandable that, since the curve segments in the cluster set are adjacent, several continuous curve segments are represented, and the fit between these curve segments and the pixel mean corresponding curve segments of any other time domain interval is poor.

[0119] Specifically, the process of locating the reference cycle includes,

[0120] Determining a time domain segment adjacent to the abnormal time domain segment and video data within the adjacent time domain segment;

[0121] Compare the time domain curve segments of the video parameters corresponding to the video data in the adjacent time domain segments with the video data in each remaining time domain segment, determine the maximum fitting mean, and determine whether the positioning conditions are met;

[0122] It can be understood that since there are two video parameters, the maximum degree of fit corresponding to the pixel mean and the maximum degree of fit corresponding to the image gradient can be determined respectively, and then the mean can be solved to obtain the maximum degree of fit mean;

[0123] Determine the minimum time endpoint and the maximum time endpoint corresponding to the remaining time domain segments that meet the positioning conditions. It can be understood that there are two remaining time domain segments. If adjacent time domain segments are marked on the time axis, the corresponding minimum time endpoint and the maximum time endpoint can be determined;

[0124] The time domain segment corresponding to the minimum time endpoint and the maximum time endpoint is determined as a reference period. It can be understood that the two time endpoints can constitute a time domain segment;

[0125] Among them, the positioning condition is that the maximum fitting degree average of the remaining time domain segment and the corresponding adjacent time domain segment is greater than the corresponding fitting degree threshold, and the time interval between the remaining time domain segments is equal to the duration corresponding to the abnormal time domain segment.

[0126] It can be understood that when the positioning conditions are met, two remaining time domain segments with a high degree of fit with the adjacent time domain segments can be found, and the time interval between the remaining time domain segments is exactly opposite to the corresponding duration of the abnormal time domain segment, and thus a reference period with a periodic corresponding relationship can be found.

[0127] Specifically, see Figure 4 As shown, Figure 4 This is a logic block diagram of an embodiment of the invention for determining whether a video transmission source is abnormal based on the difference characteristics. The process of determining whether a video transmission source is abnormal based on the comparison result includes:

[0128] Extract the video frames within the abnormal time domain segment and determine the pixel mean and image gradient mean corresponding to each video frame;

[0129] Determining a reference pixel mean range and an image gradient range based on a reference period;

[0130] If the pixel mean is not within the pixel mean range and / or the image gradient mean is not within the image gradient range, it is determined that an abnormality exists in the video transmission source.

[0131] The upper limit of the pixel mean range is 1.3 times the pixel mean corresponding to the video frame in the reference period, and the lower limit of the pixel mean range is 0.7 times the pixel mean corresponding to the video data in the reference period.

[0132] The upper limit of the image gradient range is 1.3 times the image gradient corresponding to the video frame in the reference period, and the lower limit of the image gradient range is 0.7 times the mean image gradient corresponding to the video data in the reference period.

[0133] The present invention adopts different analysis and processing methods for different regularity labels. When targeting video data with regularity labels, the corresponding time domain dimension curve is segmented, the abnormal time domain segment is determined, and the reference period is synchronously located. In actual situations, if frame insertion or tampering is performed on video data with regularity, it will be reflected in the video parameter dimension. Compared with the use of image processing models to analyze video frames, video parameters are easy to collect quickly. Combined with the regularity of video data, the abnormal time domain segment and the reference period can be quickly identified. Since there is a corresponding relationship between the reference period and the abnormal time domain segment, it can be used to quickly analyze whether there is an abnormality in the video transmission source, thereby ensuring the reliability of the analysis and improving the efficiency of the analysis when analyzing a large amount of video data.

[0134] Specifically, see Figure 5 As shown, Figure 5 A logic block diagram of an embodiment of the invention for determining whether the video transmission source has an abnormality based on each video frame, extracting video frames in the video segment data at corresponding frame extraction intervals, and determining whether the video transmission source has an abnormality includes:

[0135] Analyze each extracted video frame according to a predetermined image processing model to determine whether abnormal features appear;

[0136] If abnormal features appear, it is determined that the video transmission source is abnormal;

[0137] The determined frame extraction interval is positively correlated with the regularity characterization parameter;

[0138] In implementation, optionally,

[0139] If the regularity characterization parameter belongs to the interval (0.75, 8], the frame extraction interval is set to 1.3 times the initial frame extraction interval;

[0140] If the regularity characterization parameter belongs to the interval (0.65, 0.75], the frame extraction interval is set to the initial frame extraction interval

[0141] If the regularity characterization parameter belongs to the interval (0, 0.65], the frame extraction interval is set to 0.7 times the initial frame extraction interval.

[0142] The initial frame interval is 30.

[0143] Specifically, an image processing model capable of identifying abnormal features can be pre-trained, and those skilled in the art can set abnormal features according to needs. The abnormal features can be in image form, such as prohibited icons, prohibited image targets, which will not be repeated here.

[0144] There is no limitation on the form of the image processing model. An image processing model with a neural network architecture can be used. Through training, the image processing model can recognize predetermined abnormal features, which will not be elaborated here.

[0145] The present invention adopts different analysis and processing methods for different regularity labels. When targeting video data with irregular labels, the frame extraction interval is adaptively adjusted based on the regularity characterization parameters to reduce feature omissions. Moreover, since this type of data does not have regularity, the frame extraction form is used to traverse and analyze the video frame content through the image processing model to determine whether there is an abnormality in the video transmission source, thereby ensuring the reliability of the analysis and improving the analysis efficiency.

[0146] Also provided is a system for applying a video acquisition method based on FPGA parallel processing, including:

[0147] FPGA collector, which is used to collect video data transmitted by each video transmission source in parallel;

[0148] A feature extractor for analyzing video parameters of the video data in each time domain segment, and constructing a corresponding time domain dimension curve based on the video parameters to analyze regularity characterization features of the video data;

[0149] a label setter, configured to calculate a regularity characterization parameter based on the regularity characterization feature, so as to set a regularity category label for the video data transmitted by each of the video transmission sources in different time domain segments;

[0150] A data analyzer is used to analyze and process the video data transmitted by the video transmission source according to regularity category labels, including:

[0151] It is used to segment the time domain dimension curve corresponding to the video data, fit the segmented curve segments with the remaining curve segments to determine the abnormal time domain segment, and locate the reference period to extract the video frame corresponding to the abnormal time domain segment and compare it with the corresponding video frame in the reference period. Based on the comparison results, it is determined whether there is an abnormality in the video transmission source;

[0152] Alternatively, the method may be used to determine a frame extraction interval based on the regularity characterization parameter, extract video frames from the video segment data corresponding to the frame extraction interval, and determine whether the video transmission source has an abnormality based on each extracted video frame.

[0153] The FPGA collector can be an FPGA device or a video processor that can receive multiple video streams. Those skilled in the art can choose the model by themselves, which will not be described in detail here.

[0154] The feature extractor, label setter and data analyzer may be composed of logic components or a combination of logic components, including a field programmable processor, a computer or a microprocessor in a computer.

[0155] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

Claims

1. A video acquisition method based on FPGA parallel processing, characterized in that: include: The video data transmitted by each video transmission source is collected in parallel by the FPGA device; Analyzing the video parameters of each of the video data in each time domain segment, and constructing a corresponding time domain dimension curve based on the video parameters to analyze the regularity characterization features of the video data; Calculating regularity characterization parameters based on the regularity characterization features to set regularity category labels for video data transmitted by each of the video transmission sources in different time domain segments; Analyze and process the video data transmitted by the video transmission source according to the regularity category label, including: The video data is segmented into time-domain dimension curves, and the segmented curve segments are fitted with the remaining curve segments to determine the abnormal time-domain segments. The reference period is located, and the corresponding video frames in the abnormal time-domain segments are extracted and compared with the corresponding video frames in the reference period. Based on the comparison results, it is determined whether the video transmission source is abnormal. Alternatively, a frame extraction interval is determined based on the regularity characterization parameter, video frames in the video segment data are extracted at the corresponding frame extraction interval, and whether the video transmission source has an abnormality is determined based on each of the extracted video frames; The regularity representation features include pixel regularity representation features and smoothness regularity representation features; The process of analyzing the regularity representation features of video data includes: Get video parameters, including pixel mean and image gradient; Construct the time domain variation curve of pixel mean and image gradient; Determine the pixel mean and the autocorrelation coefficient of the image gradient at different time lags; Determine the significant peak of the autocorrelation coefficient corresponding to the pixel mean outside the confidence interval, solve the mean of the significant peak, and obtain the pixel regularity characterization feature; The significant peak of the autocorrelation coefficient corresponding to the image gradient outside the confidence interval is determined, the mean value of the significant peak is solved, and the smoothing regularity characterization feature is obtained.

2. The video acquisition method based on FPGA parallel processing according to claim 1, characterized in that: The parameters characterizing the regularity of calculation include: The pixel regularity characterization feature and the smooth regularity characterization feature are weighted and summed to obtain the regularity characterization parameter.

3. The video acquisition method based on FPGA parallel processing according to claim 1, characterized in that: The process of setting regularity category labels of video data transmitted by each of the video transmission sources in different time domain segments includes: If the regularity characterization parameter is greater than or equal to a preset regularity standard threshold, determining to set a regularity label for the video data; If the regularity characterization parameter is less than a preset regularity standard threshold, it is determined to set an irregularity label for the video data.

4. The video acquisition method based on FPGA parallel processing according to claim 3, characterized in that: The video data transmitted by the video transmission source is analyzed and processed according to the regularity category label, including: If the video data transmitted by the video transmission source is set with a regularity label, the corresponding video frame in the abnormal time domain segment is extracted and compared with the corresponding video frame in the reference period, and the video transmission source is judged to have an abnormality based on the comparison result; If the video data transmitted by the video transmission source is set with a regularity label, it is determined whether the video transmission source is abnormal based on each of the extracted video frames.

5. The video acquisition method based on FPGA parallel processing according to claim 4, characterized in that: The process of fitting the segmented curve segments with the remaining curve segments includes: Analyze the fitting degree of the curve segment corresponding to the pixel mean and the remaining curve segments; Analyze the fitting degree of the curve segment corresponding to the image gradient and the remaining curve segments; Cluster each curve segment according to its fitting degree; Obtaining several clustering sets for pixel means and several clustering sets for image gradients; Identify the time domain segment corresponding to each cluster set as the abnormal time domain segment; The cluster set must satisfy the following conditions: each curve segment is adjacent and the maximum fitting degree of each curve segment with the remaining curve segments is less than the fitting degree threshold.

6. The video acquisition method based on FPGA parallel processing according to claim 5, characterized in that: The process of locating the reference cycle includes, Determining a time domain segment adjacent to the abnormal time domain segment and video data within the adjacent time domain segment; Compare the time domain curve segments of the video parameters corresponding to the video data in the adjacent time domain segments with the video data in each remaining time domain segment, determine the maximum fitting mean, and determine whether the positioning conditions are met; Determine the minimum time endpoint and the maximum time endpoint corresponding to the remaining time domain segment that meets the positioning conditions; Determine the time domain segments corresponding to the minimum time endpoint and the maximum time endpoint as the reference period; Among them, the positioning condition is that the maximum fitting degree average of the remaining time domain segment and the corresponding adjacent time domain segment is greater than the corresponding fitting degree threshold, and the time interval between the remaining time domain segments is equal to the duration corresponding to the abnormal time domain segment.

7. The video acquisition method based on FPGA parallel processing according to claim 6, characterized in that: The process of determining whether there is an abnormality in the video transmission source based on the comparison results includes: Extract the video frames within the abnormal time domain segment and determine the pixel mean and image gradient mean corresponding to each video frame; Determining a reference pixel mean range and an image gradient range based on a reference period; If the pixel mean is not within the pixel mean range and / or the image gradient mean is not within the image gradient range, it is determined that an abnormality exists in the video transmission source.

8. The video acquisition method based on FPGA parallel processing according to claim 1, characterized in that: Extracting video frames from the video segment data at a corresponding frame extraction interval to determine whether a video transmission source is abnormal includes: Analyze each extracted video frame according to a predetermined image processing model to determine whether abnormal features appear; If abnormal features appear, it is determined that the video transmission source is abnormal; The determined frame extraction interval is positively correlated with the regularity characterization parameter.

9. A system using the video acquisition method based on FPGA parallel processing according to any one of claims 1 to 8, characterized in that: include, FPGA collector, which is used to collect video data transmitted by each video transmission source in parallel; A feature extractor for analyzing video parameters of the video data in each time domain segment, and constructing a corresponding time domain dimension curve based on the video parameters to analyze regularity characterization features of the video data; a label setter, configured to calculate a regularity characterization parameter based on the regularity characterization feature, so as to set a regularity category label for the video data transmitted by each of the video transmission sources in different time domain segments; A data analyzer is used to analyze and process the video data transmitted by the video transmission source according to regularity category labels, including: It is used to segment the time domain dimension curve corresponding to the video data, fit the segmented curve segments with the remaining curve segments to determine the abnormal time domain segment, and locate the reference period to extract the video frame corresponding to the abnormal time domain segment and compare it with the corresponding video frame in the reference period. Based on the comparison results, it is determined whether there is an abnormality in the video transmission source; Alternatively, the method may be used to determine a frame extraction interval based on the regularity characterization parameter, extract video frames from the video segment data corresponding to the frame extraction interval, and determine whether the video transmission source has an abnormality based on each extracted video frame.

Images