A method and apparatus for sorting a video sequence, an electronic device, and a storage medium

Abstract

The application provides a video sequence sorting method and device, electronic equipment and storage medium. The sorting method comprises the following steps: acquiring a plurality of video frames of each video sequence in a plurality of video sequences shot by an analysis target; for each video sequence, determining a foreground frame with the analysis target and a background frame without the analysis target in the plurality of video frames of the video sequence; determining the quality of the video sequence based on the foreground frame and the background frame; and sorting the plurality of video sequences according to the quality of each video sequence. The technical solution provided by the application can analyze the analysis target in the video sequence with better quality in priority according to the sorted video sequence, improve the efficiency of analyzing the video sequence, and reduce the burden of doctors.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, electronic device, and storage medium for sorting video sequences. Background Technology

[0002] A video sequence is a series of video images arranged in chronological order of capture time, which are spatially and temporally related. For example, a coronary angiography sequence is a common and effective method for diagnosing coronary atherosclerotic heart disease (CAD). It involves ingesting a substance containing elements with high atomic numbers and then taking multiple radiographic images of the body at the site of the diagnosis for medical diagnosis. Specifically, a contrast agent is injected into the coronary arteries through a catheter, and 2D coronary angiography images are obtained by X-ray imaging.

[0003] Currently, among multiple video sequences targeting the same analytical objective, such as multiple coronary angiography sequences for the same vascular site, not every video sequence is suitable for analysis and diagnosis. Therefore, experienced physicians are required to subjectively evaluate the quality of multiple angiographic images within each coronary angiography sequence to determine the video sequence with the best quality for analysis. However, this manual evaluation method is extremely time-consuming and labor-intensive, increasing the surgical burden on both doctors and patients. Therefore, improving the efficiency of video sequence analysis to reduce the burden on physicians has become an urgent problem to be solved. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method, apparatus, electronic device and storage medium for sorting video sequences, which can determine the quality of each video sequence by using the foreground and background frames in each video sequence, sort multiple video sequences according to the quality of each video sequence, and prioritize the analysis of the analysis target in the video sequence with better quality according to the sorted video sequence, thereby improving the efficiency of video sequence analysis and reducing the burden on doctors.

[0005] This application mainly includes the following aspects:

[0006] In a first aspect, embodiments of this application provide a method for sorting video sequences, the sorting method comprising:

[0007] Obtain multiple video frames from each of the multiple video sequences captured for the target of analysis;

[0008] For each video sequence, among multiple video frames in the video sequence, foreground frames that have the analysis target and background frames that do not have the analysis target are identified;

[0009] The quality of the video sequence is determined based on the foreground frame and the background frame;

[0010] The plurality of video sequences are sorted according to the quality of each video sequence, so that the analysis target is analyzed according to the sorted video sequences.

[0011] Furthermore, the step of determining the quality of the video sequence based on the foreground frame and the background frame includes:

[0012] Based on the foreground frame, the foreground frame is transformed in the frequency domain to obtain a foreground frequency domain map, and based on the background frame, the background frame is transformed in the frequency domain to obtain a background frequency domain map.

[0013] Based on the foreground frequency domain map and the background frequency domain map, a target frequency domain map containing only the analysis target is obtained;

[0014] The target frequency domain map is divided into multiple target frequency domain sub-regions according to a preset removal area, and the background frequency domain map is divided into multiple background frequency domain sub-regions according to a preset removal area;

[0015] The image quality of the target frequency domain map is determined based on the plurality of target frequency domain sub-regions, and the image quality of the background frequency domain map is determined based on the plurality of background frequency domain sub-regions;

[0016] The quality of the video sequence is determined based on the image quality of the target frequency domain map and the image quality of the background frequency domain map.

[0017] Furthermore, the step of determining the foreground frame containing the analysis target and the background frame not containing the analysis target among multiple video frames in the video sequence includes:

[0018] In the multiple video frames of the video sequence, for each video frame, obtain the grayscale value corresponding to each pixel in the video frame and the number of pixels included in the video frame;

[0019] Add the grayscale values ​​corresponding to each pixel of the video frame to get the total grayscale value of the video frame;

[0020] The quotient of the sum of the grayscale values ​​corresponding to the video frame and the number of pixels included in the video frame is determined as the average grayscale value corresponding to the video frame.

[0021] Based on the average grayscale value of each video frame in the video sequence, the video frame with the largest average grayscale value is determined as the background frame without the analysis target, and the video frame with the smallest average grayscale value is determined as the foreground frame with the analysis target.

[0022] Furthermore, the step of obtaining a target frequency domain map containing only the analysis target based on the foreground frequency domain map and the background frequency domain map includes:

[0023] Based on the foreground frequency domain map, obtain the pixel value of each pixel in the foreground frequency domain map;

[0024] Based on the background frequency domain map, obtain the pixel value of each pixel in the background frequency domain map;

[0025] Subtract the pixel value of each pixel in the foreground frequency domain map from the pixel value of the corresponding pixel in the background frequency domain map to obtain a target frequency domain map containing only the analysis target.

[0026] Furthermore, the step of determining the image quality of the target frequency domain map based on the plurality of target frequency domain sub-regions includes:

[0027] Based on the plurality of target frequency domain sub-regions, obtain the maximum pixel value of each target frequency domain sub-region and the average variance of each target frequency domain sub-region in advance;

[0028] For each target frequency domain sub-region, the quality of the target frequency domain sub-region is determined based on the maximum pixel value and average variance in that target frequency domain sub-region.

[0029] Based on the quality of each target frequency domain sub-region, the sum of the qualities of each target frequency domain sub-region and the quotient of the number of target frequency domain sub-regions are determined as the image quality of the target frequency domain map.

[0030] Furthermore, the maximum pixel value of each target frequency domain sub-region is determined through the following steps:

[0031] Obtain the pixel value of each pixel in each of the multiple target frequency domain sub-regions;

[0032] For each target frequency domain sub-region, among the pixel values ​​of all pixels in the target frequency domain sub-region, the pixel value with the largest value is determined as the maximum pixel value in the target frequency domain sub-region.

[0033] Furthermore, the step of determining the quality of the video sequence based on the image quality of the target frequency domain map and the image quality of the background frequency domain map includes:

[0034] Based on the image quality of the target frequency domain map, the product of the image quality of the target frequency domain map and the first preset weight is determined as the first quality component of the video sequence;

[0035] Based on the image quality of the background frequency domain map, the product of the image quality of the background frequency domain map and the second preset weight is determined as the second quality component of the video sequence; wherein, the second preset weight is the difference between the number 1 and the first preset weight;

[0036] The quality of the video sequence is determined by summing the first quality component and the second quality component of the video sequence.

[0037] Secondly, embodiments of this application also provide a video sequence sorting apparatus, the sorting apparatus comprising:

[0038] The acquisition module is used to acquire multiple video frames from each of the multiple video sequences captured for the analysis target.

[0039] The first determining module is used to determine, for each video sequence, a foreground frame having the analysis target and a background frame not having the analysis target among multiple video frames in the video sequence;

[0040] The second determining module is used to determine the quality of the video sequence based on the foreground frame and the background frame;

[0041] The sorting module is used to sort the plurality of video sequences according to the quality of each video sequence, so as to analyze the analysis target according to the sorted video sequences.

[0042] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the video sequence sorting method described above are performed.

[0043] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the video sequence sorting method described above.

[0044] This application provides a method, apparatus, electronic device, and storage medium for sorting video sequences. The sorting method includes: acquiring multiple video frames of each video sequence from multiple video sequences captured for an analysis target; for each video sequence, identifying a foreground frame containing the analysis target and a background frame not containing the analysis target from the multiple video frames; determining the quality of the video sequence based on the foreground frame and the background frame; and sorting the multiple video sequences according to the quality of each video sequence, so as to analyze the analysis target according to the sorted video sequences.

[0045] Thus, by using the technical solution provided in this application, the quality of each video sequence can be determined by the foreground and background frames in each video sequence. Multiple video sequences can be sorted according to the quality of each video sequence. The analysis targets in the video sequences with better quality can be analyzed first according to the sorted video sequences, which improves the efficiency of video sequence analysis and reduces the burden on doctors.

[0046] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 A flowchart illustrating a video sequence sorting method provided in an embodiment of this application is shown;

[0049] Figure 2 A flowchart illustrating another video sequence sorting method provided in an embodiment of this application is shown;

[0050] Figure 3 This illustration shows a schematic diagram of background frequency domain map segmentation provided in an embodiment of this application;

[0051] Figure 4 This illustration shows a schematic diagram of a target frequency domain map segmentation provided in an embodiment of this application;

[0052] Figure 5 This diagram illustrates a structural diagram of a video sequence sorting apparatus provided in an embodiment of this application;

[0053] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0055] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0056] To enable those skilled in the art to use the content of this application, and in conjunction with the specific application scenario of "sorting video sequences", the following implementation methods are provided. For those skilled in the art, the general principles defined herein can be applied to other embodiments and application scenarios without departing from the spirit and scope of this application.

[0057] The methods, apparatus, electronic devices, or computer-readable storage media described in this application can be applied to any scenario that requires sorting video sequences. This application does not limit the specific application scenario. Any scheme that uses a video sequence sorting method, apparatus, electronic device, and storage medium provided in this application is within the protection scope of this application.

[0058] It is worth noting that a video sequence is an arrangement of multiple video images with spatiotemporal connections in chronological order of capture time. For example, coronary angiography sequences are a common and effective method for diagnosing coronary atherosclerotic heart disease (CHD). This involves ingesting a substance containing elements with high atomic numbers and then taking multiple radiographic images of the body at the site of the diagnosis for medical diagnosis. Specifically, a contrast agent is injected into the coronary arteries through a catheter, and 2D coronary angiography images are obtained by X-ray imaging.

[0059] Currently, among multiple video sequences targeting the same analytical objective, such as multiple coronary angiography sequences for the same vascular site, not every video sequence is suitable for analysis and diagnosis. Therefore, experienced physicians are required to subjectively evaluate the quality of multiple angiographic images within each coronary angiography sequence to determine the video sequence with the best quality for analysis. However, this manual evaluation method is extremely time-consuming and labor-intensive, increasing the surgical burden on both doctors and patients. Therefore, improving the efficiency of video sequence analysis to reduce the burden on physicians has become an urgent problem to be solved.

[0060] Based on this, this application proposes a method, apparatus, electronic device, and storage medium for sorting video sequences. The sorting method includes: acquiring multiple video frames of each video sequence from multiple video sequences captured for an analysis target; for each video sequence, identifying a foreground frame containing the analysis target and a background frame not containing the analysis target from the multiple video frames of that video sequence; determining the quality of the video sequence based on the foreground frame and the background frame; and sorting the multiple video sequences according to the quality of each video sequence, so as to analyze the analysis target according to the sorted video sequences.

[0061] Thus, by using the technical solution provided in this application, the quality of each video sequence can be determined by the foreground and background frames in each video sequence. Multiple video sequences can be sorted according to the quality of each video sequence. The analysis targets in the video sequences with better quality can be analyzed first according to the sorted video sequences, which improves the efficiency of video sequence analysis and reduces the burden on doctors.

[0062] To facilitate understanding of this application, the technical solutions provided in this application will be described in detail below with reference to specific embodiments.

[0063] Please see Figure 1 , Figure 1 A flowchart illustrating a video sequence sorting method provided in an embodiment of this application is shown below. Figure 1 As shown, the sorting method includes:

[0064] S101. Obtain multiple video frames from each of the multiple video sequences captured for the analysis target;

[0065] In this step, the analysis target can be blood vessels or fallopian tubes, and the video sequence can be either a coronary angiography sequence or a fallopian tube angiography sequence. Currently, the quality of angiography images (video frames) in coronary angiography sequences is generally subjectively assessed by experienced physicians, or roughly calculated using basic statistical indicators such as mean, variance, and signal-to-noise ratio. However, manual assessment requires a significant amount of manpower and time, creating a clear contradiction between limited medical resources (doctors) and the ever-increasing number of angiography procedures. For angiography analysis, automatically filtering out high-quality images can greatly accelerate the analysis speed, especially during surgery when time is extremely precious. Manual analysis becomes a major bottleneck, undoubtedly increasing the surgical burden on both doctors and patients. Moreover, using basic statistical indicators for quality scoring is too one-sided and too global. Generally speaking, data with low variance (one-dimensional or two-dimensional signals, etc.) indicates good stability, but in angiography images, it is essential to ensure that blood vessels are visible. For example, angiography images without visible blood vessels have very low variance, but the contrast agent has not yet flowed into the blood vessels, making them unanalyzable. For example, the signal-to-noise ratio (SNR) is calculated based on the overall image. For instance, if the background noise is very low while the foreground noise is high, the overall SNR may decrease. Therefore, these hard metrics are based on global calculations and can easily overlook the importance of the foreground (blood vessels) in the angiographic image.

[0066] For example, coronary angiography is required to diagnose coronary atherosclerotic heart disease (CAD). Each patient typically has multiple sequences of angiographic images (two-dimensional dynamic sequences). For instance, videos of a specific blood vessel are taken from different angles, resulting in multiple angiographic sequences (video sequences). Each sequence contains multiple angiographic images (video frames). However, some angiographic images are not visible because the contrast agent has not yet flowed into the blood vessel, making them unusable for disease analysis. Therefore, not every angiographic sequence is suitable for analysis. How to quickly and correctly sort these images according to quality is a current problem. Solving this problem can save doctors unnecessary time searching for and interpreting images, allowing them to focus on crucial analyses such as the patient's actual condition and vascular morphology and function. This would reduce the burden on doctors, shorten surgical or postoperative analysis time, and benefit both doctors and patients.

[0067] S102. For each video sequence, among multiple video frames in the video sequence, determine the foreground frame that has the analysis target and the background frame that does not have the analysis target;

[0068] It should be noted that the steps of identifying the foreground frame containing the analysis target and the background frame not containing the analysis target among multiple video frames in this video sequence include:

[0069] S1021. In the multiple video frames of the video sequence, for each video frame, obtain the grayscale value corresponding to each pixel of the video frame and the number of pixels included in the video frame.

[0070] S1022. Add up the gray values ​​corresponding to each pixel of the video frame to obtain the total gray values ​​of the video frame.

[0071] S1023. The quotient of the sum of the gray values ​​corresponding to the video frame and the number of pixels included in the video frame is determined as the average gray value corresponding to the video frame.

[0072] S1024. Based on the average grayscale value of each video frame in the video sequence, the video frame with the largest average grayscale value is determined as the background frame without the analysis target, and the video frame with the smallest average grayscale value is determined as the foreground frame with the analysis target.

[0073] For example, the purpose of steps S1021-S1024 above is to automatically and accurately filter out the foreground frames (foreground frames with analysis targets) when the angiography is in progress and the background frames (background frames without analysis targets) when the contrast agent is not injected. First, the grayscale mean of each video frame in the angiography sequence is calculated. Here, for each video frame, the grayscale mean of the video frame is calculated as follows: the sum of the grayscale values ​​of each pixel in the video frame is divided by the quotient of the number of pixels included in the video frame, and the grayscale mean of the video frame is determined. Based on the grayscale mean of all video frames included in the angiography sequence, a grayscale mean change curve is plotted along the shooting time, and the video frame with the largest grayscale mean in the change curve is taken as the background frame, and the video frame with the smallest grayscale mean is taken as the angiography frame (foreground frame). Here, because the contrast agent appears dark in the image, the one with the smaller grayscale mean is the angiography frame.

[0074] S103. Determine the quality of the video sequence based on the foreground frame and the background frame;

[0075] It should be noted that the steps for determining the quality of this video sequence based on foreground and background frames are detailed in the following section. Figure 2 , Figure 2 A flowchart illustrating another video sequence sorting method provided in this application embodiment is shown below. Figure 2 As shown, the steps for determining the quality of the video sequence based on foreground and background frames include:

[0076] S201. Based on the foreground frame, the foreground frame is transformed in the frequency domain to obtain a foreground frequency domain map, and based on the background frame, the background frame is transformed in the frequency domain to obtain a background frequency domain map.

[0077] In this step, the frequency domain transformation can be performed using the Discrete Fast Fourier Transform (FFT). The foreground and background frames are transformed from the time domain to the frequency domain using the FFT, as shown in the following formula:

[0078]

[0079] Where m and n represent the horizontal and vertical coordinates of the two-dimensional image (i.e., the foreground frame and the background frame), p and q represent the spatial coordinates in the transformed frequency domain image (i.e., the foreground frequency domain image and the background frequency domain image), and M and N represent the length and width of the two-dimensional image, respectively.

[0080] Here, the characteristic of the frequency domain map is that the surrounding areas contain high-frequency information, including image noise, while the center contains low-frequency information, including a large number of important image features. For example, the center of the blood vessel frequency domain map (foreground frequency domain map) is rich, while the surrounding areas of the background frame have relatively poor information.

[0081] S202. Based on the foreground frequency domain map and the background frequency domain map, a target frequency domain map containing only the analysis target is obtained;

[0082] It should be noted that the steps for obtaining a target frequency domain map containing only the target being analyzed, based on the foreground and background frequency domain maps, include:

[0083] S2021. Based on the foreground frequency domain map, obtain the pixel value of each pixel in the foreground frequency domain map;

[0084] S2022. Based on the background frequency domain map, obtain the pixel value of each pixel in the background frequency domain map;

[0085] S2023. Subtract the pixel value of each pixel in the foreground frequency domain map from the pixel value of the corresponding pixel in the background frequency domain map to obtain a target frequency domain map containing only the analysis target.

[0086] The purpose of this step is to independently evaluate the background and foreground quality, which requires separating the foreground and background. For example, image segmentation techniques can generally be used to separate the analysis target from the background in the foreground frame. However, in a foreground frame where the analysis target is blood vessels, considering that this embodiment does not require highly precise foreground and background regions (not accurate to every pixel), precise and complex image segmentation is unnecessary. Due to the heartbeat, blood vessels and catheters are in motion within the image sequence; therefore, subtracting in the time domain would result in unusable subtraction results due to misalignment. Therefore, this embodiment uses frequency domain subtraction to obtain the foreground frequency domain map, subtracting the foreground frequency domain map from the background frequency domain map. This subtracts the background region from the foreground frequency domain map, yielding the foreground region (analysis target, such as blood vessels). This method avoids spatial misalignment issues. After subtraction in the frequency domain, the noisy regions are weakened, and the important foreground information of blood vessels is preserved after inverse transformation back to the time domain. This demonstrates the effectiveness of obtaining the foreground through frequency domain subtraction.

[0087] S203. Divide the target frequency domain map into multiple target frequency domain sub-regions according to a preset removal area, and divide the background frequency domain map into multiple background frequency domain sub-regions according to a preset removal area;

[0088] For example, after obtaining the target frequency domain map and the background frequency domain map, quality assessment can be performed on them separately. First, the target frequency domain map and the background frequency domain map need to be segmented into regions respectively. Please refer to [link to relevant documentation]. Figure 3 and Figure 4 , Figure 3 This is a schematic diagram of background frequency domain map segmentation provided in an embodiment of this application. Figure 4 This is a schematic diagram illustrating a target frequency domain map segmentation method provided in an embodiment of this application. Figure 3 As shown, the preset removal area is the cross-shaped region in the background frequency domain image. This cross-shaped region divides the background frequency domain image into four sub-regions, which are respectively... Figure 3 x in 11 x 12 x 21 and x 22 ;like Figure 4 As shown, the preset removal region is the cross-shaped region in the target frequency domain map. This cross-shaped region divides the target frequency domain map into four target frequency domain sub-regions, namely... Figure 4 y in 11 y 12 y 21 and y 22 The preset removal area here can be a region determined in advance based on historical experience or experimental data.

[0089] S204. Determine the image quality of the target frequency domain map based on the plurality of target frequency domain sub-regions, and determine the image quality of the background frequency domain map based on the plurality of background frequency domain sub-regions;

[0090] It should be noted that the steps for determining the image quality of the target frequency domain map based on multiple target frequency domain sub-regions include:

[0091] S2041. Based on the plurality of target frequency domain sub-regions, obtain the maximum pixel value of each target frequency domain sub-region and the average variance of each target frequency domain sub-region;

[0092] It should be noted that the maximum pixel value of each target frequency domain sub-region is determined through the following steps:

[0093] 1) Obtain the pixel value of each pixel in each of the multiple target frequency domain sub-regions;

[0094] 2) For each target frequency domain sub-region, among the pixel values ​​of all pixels in the target frequency domain sub-region, the pixel value with the largest value is determined as the maximum pixel value in the target frequency domain sub-region.

[0095] This step requires determining the maximum pixel value and average variance of each target frequency domain sub-region in the target frequency domain map. Here, the step of determining the average variance of each target frequency domain sub-region includes:

[0096] 1. The average pixel value of the target frequency domain sub-region is determined by the sum of the pixel values ​​of each pixel in the target frequency domain sub-region and the quotient of the number of pixels included in the target frequency domain sub-region.

[0097] 2. The sum of the squares of the differences between the pixel value of each pixel in the target frequency domain sub-region and the average pixel value of the target frequency domain sub-region is determined as the variance of the target frequency domain sub-region.

[0098] Third, the average variance of the target frequency domain sub-region is determined by the quotient of the variance of the target frequency domain sub-region and the number of pixels included in the target frequency domain sub-region.

[0099] Here, it is also necessary to obtain the maximum pixel value of each background frequency domain sub-region and the average variance of each target frequency domain sub-region based on the multiple background frequency domain sub-regions; the maximum pixel value of each target frequency domain sub-region is determined through the following steps:

[0100] 1) Obtain the pixel value of each pixel in each of the multiple background frequency domain sub-regions;

[0101] 2) For each background frequency domain sub-region, among the pixel values ​​of all pixels in the background frequency domain sub-region, the pixel value with the largest value is determined as the maximum pixel value in the background frequency domain sub-region.

[0102] Here, the step of determining the average variance of each background frequency domain sub-region includes:

[0103] 1. The average pixel value of the background frequency domain sub-region is determined by the sum of the pixel values ​​of each pixel in the obtained background frequency domain sub-region and the quotient of the number of pixels included in the background frequency domain sub-region.

[0104] 2. The variance of the background frequency domain sub-region is determined by the sum of the squares of the differences between the pixel value of each pixel in the background frequency domain sub-region and the average pixel value of the background frequency domain sub-region.

[0105] Third, the quotient of the variance of the background frequency domain sub-region and the number of pixels included in the background frequency domain sub-region is determined as the average variance of the background frequency domain sub-region.

[0106] S2042. For each target frequency domain sub-region, determine the quality of the target frequency domain sub-region based on the maximum pixel value and average variance in the target frequency domain sub-region;

[0107] S2043. Based on the quality of each target frequency domain sub-region, the sum of the quality of each target frequency domain sub-region and the quotient of the number of target frequency domain sub-regions are determined as the image quality of the target frequency domain map.

[0108] In this step, the formula for determining the image quality of the target frequency domain map is as follows:

[0109]

[0110] Among them, y_MSE ij Let be the average variance of the target frequency domain sub-region in the i-th row and j-th column of the target frequency domain plot y. PSNR is the square of the maximum pixel value of the target frequency domain sub-region in the i-th row and j-th column of the target frequency domain image y. y The peak signal-to-noise ratio (PSNR) of the target frequency domain image (i.e., the image quality of the target frequency domain image).

[0111] Here, the method for determining the image quality of the background frequency domain image is the same as the method for determining the image quality of the target frequency domain image. The method for determining the image quality of the background frequency domain image includes:

[0112] (1) For each background frequency domain sub-region, determine the quality of the background frequency domain sub-region based on the maximum pixel value and average variance in the background frequency domain sub-region;

[0113] (2) Based on the quality of each background frequency domain sub-region, the sum of the quality of each background frequency domain sub-region and the quotient of the number of background frequency domain sub-regions are determined as the image quality of the background frequency domain map.

[0114] In this step, the formula for determining the image quality of the background frequency domain map is as follows:

[0115]

[0116] Among them, x_MSE ij The average variance of the background frequency domain sub-region in the i-th row and j-th column of the background frequency domain plot x. PSNR is the square of the maximum pixel value of the background frequency domain sub-region in the i-th row and j-th column of the background frequency domain image x. x This represents the peak signal-to-noise ratio (PSNR) of the background frequency domain image (i.e., the image quality of the background frequency domain image).

[0117] S205. Determine the quality of the video sequence based on the image quality of the target frequency domain map and the image quality of the background frequency domain map.

[0118] It should be noted that the steps for determining the quality of the video sequence based on the image quality of the target frequency domain map and the image quality of the background frequency domain map include:

[0119] S2051. Based on the image quality of the target frequency domain map, the product of the image quality of the target frequency domain map and the first preset weight is determined as the first quality component of the video sequence.

[0120] S2052. Based on the image quality of the background frequency domain map, the product of the image quality of the background frequency domain map and the second preset weight is determined as the second quality component of the video sequence.

[0121] In this step, the second preset weight is the difference between the number one and the first preset weight.

[0122] S2053. The sum of the first quality component and the second quality component of the video sequence is used to determine the quality of the video sequence.

[0123] In this step, the first preset weight is set in advance based on historical experience or experimental data. For each video sequence, the formula for determining the quality of that video sequence is as follows:

[0124] PSNR = αPSNR x +(1-α)PSNR y ;

[0125] Where α is the weighting coefficient (second preset weight), PSNR xαPSNR is the peak signal-to-noise ratio (PSNR) of the background frequency domain image (image quality of the background frequency domain image). x The second quality component of the video sequence is (1-α), where (1-α) is the first preset weight, and PSNR is... y For the peak signal-to-noise ratio (PSNR) of the target frequency domain map (image quality of the target frequency domain map), (1-α)PSNR y The first quality component of the video sequence is denoted by , and PSNR represents the quality of the video sequence.

[0126] For example, α is used to control the weight of the target frequency domain map and the background frequency domain map in the final quality score, and α can be taken as 0.3; thus, the automated imaging image quality score is completed, and the larger the PSNR, the higher the image quality.

[0127] S104. Sort the plurality of video sequences according to the quality of each video sequence, so as to analyze the analysis target according to the sorted video sequences.

[0128] In this step, each video sequence is sorted according to quality, from highest to lowest. This embodiment does not require highly accurate and complex image segmentation techniques to distinguish the foreground and background of video frames. Leveraging the advantage that the frequency domain better reflects noise, video frames are mapped to the frequency domain for noise analysis. Using the image frequency domain, which is suitable for expressing high and low frequency information, high and low frequencies are separated to distinguish the foreground and background, resulting in a target frequency domain map and a background frequency domain map. Considering both the global and local aspects of image quality, the noise levels of the background and foreground are calculated separately and weighted to automatically obtain the final image quality score. This embodiment no longer analyzes the temporal / spatial domain of the image, but rather starts from the frequency domain, combining different frames of the angiography to determine the noise level, thereby judging the image quality. This allows for automated scoring of the image quality of a large number of angiography images, quickly prioritizing the presentation of the highest quality images to doctors.

[0129] This application provides a method for sorting video sequences. The sorting method includes: acquiring multiple video frames of each video sequence from multiple video sequences captured for an analysis target; for each video sequence, identifying a foreground frame containing the analysis target and a background frame not containing the analysis target from the multiple video frames of the video sequence; determining the quality of the video sequence based on the foreground frame and the background frame; and sorting the multiple video sequences according to the quality of each video sequence, so as to analyze the analysis target according to the sorted video sequences.

[0130] Thus, by using the technical solution provided in this application, the quality of each video sequence can be determined by the foreground and background frames in each video sequence. Multiple video sequences can be sorted according to the quality of each video sequence. The analysis targets in the video sequences with better quality can be analyzed first according to the sorted video sequences, which improves the efficiency of video sequence analysis and reduces the burden on doctors.

[0131] Based on the same application concept, this application also provides a video sequence sorting device corresponding to the video sequence sorting method provided in the above embodiment. Since the principle of the device in this application is similar to the video sequence sorting method in the above embodiment, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0132] Please see Figure 5 , Figure 5 This is a structural diagram of a video sequence sorting device provided in an embodiment of this application. Figure 5 As shown, the sorting device 510 includes:

[0133] The acquisition module 511 is used to acquire multiple video frames of each video sequence in multiple video sequences captured for the analysis target;

[0134] The first determining module 512 is used to determine, for each video sequence, a foreground frame having the analysis target and a background frame not having the analysis target among multiple video frames of the video sequence;

[0135] The second determining module 513 is used to determine the quality of the video sequence based on the foreground frame and the background frame;

[0136] The sorting module 514 is used to sort the plurality of video sequences according to the quality of each video sequence, so as to analyze the analysis target according to the sorted video sequences.

[0137] Optionally, when the second determining module 513 determines the quality of the video sequence based on the foreground frame and the background frame, the second determining module 513 is specifically used for:

[0138] Based on the foreground frame, the foreground frame is transformed in the frequency domain to obtain a foreground frequency domain map, and based on the background frame, the background frame is transformed in the frequency domain to obtain a background frequency domain map.

[0139] Based on the foreground frequency domain map and the background frequency domain map, a target frequency domain map containing only the analysis target is obtained;

[0140] The target frequency domain map is divided into multiple target frequency domain sub-regions according to a preset removal area, and the background frequency domain map is divided into multiple background frequency domain sub-regions according to a preset removal area;

[0141] The image quality of the target frequency domain map is determined based on the plurality of target frequency domain sub-regions, and the image quality of the background frequency domain map is determined based on the plurality of background frequency domain sub-regions;

[0142] The quality of the video sequence is determined based on the image quality of the target frequency domain map and the image quality of the background frequency domain map.

[0143] Optionally, when the first determining module 512 determines, in the context of multiple video frames in the video sequence, a foreground frame having the analysis target and a background frame not having the analysis target, the first determining module 512 is specifically used for:

[0144] In the multiple video frames of the video sequence, for each video frame, obtain the grayscale value corresponding to each pixel in the video frame and the number of pixels included in the video frame;

[0145] Add the grayscale values ​​corresponding to each pixel of the video frame to get the total grayscale value of the video frame;

[0146] The quotient of the sum of the grayscale values ​​corresponding to the video frame and the number of pixels included in the video frame is determined as the average grayscale value corresponding to the video frame.

[0147] Based on the average grayscale value of each video frame in the video sequence, the video frame with the largest average grayscale value is determined as the background frame without the analysis target, and the video frame with the smallest average grayscale value is determined as the foreground frame with the analysis target.

[0148] Optionally, when the second determining module 513 is used to obtain a target frequency domain map containing only the analysis target based on the foreground frequency domain map and the background frequency domain map, the second determining module 513 is specifically used for:

[0149] Based on the foreground frequency domain map, obtain the pixel value of each pixel in the foreground frequency domain map;

[0150] Based on the background frequency domain map, obtain the pixel value of each pixel in the background frequency domain map;

[0151] Subtract the pixel value of each pixel in the foreground frequency domain map from the pixel value of the corresponding pixel in the background frequency domain map to obtain a target frequency domain map containing only the analysis target.

[0152] Optionally, when the second determining module 513 determines the image quality of the target frequency domain map based on the plurality of target frequency domain sub-regions, the second determining module 513 is specifically used for:

[0153] Based on the plurality of target frequency domain sub-regions, obtain the maximum pixel value of each target frequency domain sub-region and the average variance of each target frequency domain sub-region in advance;

[0154] For each target frequency domain sub-region, the quality of the target frequency domain sub-region is determined based on the maximum pixel value and average variance in that target frequency domain sub-region.

[0155] Based on the quality of each target frequency domain sub-region, the sum of the qualities of each target frequency domain sub-region and the quotient of the number of target frequency domain sub-regions are determined as the image quality of the target frequency domain map.

[0156] Optionally, the second determining module 513 is further configured to:

[0157] Obtain the pixel value of each pixel in each of the multiple target frequency domain sub-regions;

[0158] For each target frequency domain sub-region, among the pixel values ​​of all pixels in the target frequency domain sub-region, the pixel value with the largest value is determined as the maximum pixel value in the target frequency domain sub-region.

[0159] Optionally, when determining the quality of the video sequence based on the image quality of the target frequency domain map and the image quality of the background frequency domain map, the second determining module 513 is specifically used for:

[0160] Based on the image quality of the target frequency domain map, the product of the image quality of the target frequency domain map and the first preset weight is determined as the first quality component of the video sequence;

[0161] Based on the image quality of the background frequency domain map, the product of the image quality of the background frequency domain map and the second preset weight is determined as the second quality component of the video sequence; wherein, the second preset weight is the difference between the number 1 and the first preset weight;

[0162] The quality of the video sequence is determined by summing the first quality component and the second quality component of the video sequence.

[0163] This application provides a video sequence sorting device, comprising: an acquisition module for acquiring multiple video frames of each video sequence captured for an analysis target; a first determination module for determining, for each video sequence, a foreground frame containing the analysis target and a background frame not containing the analysis target from among the multiple video frames of the video sequence; a second determination module for determining the quality of the video sequence based on the foreground frame and the background frame; and a sorting module for sorting the multiple video sequences according to the quality of each video sequence, so as to analyze the analysis target according to the sorted video sequences.

[0164] Thus, by using the technical solution provided in this application, the quality of each video sequence can be determined by the foreground and background frames in each video sequence. Multiple video sequences can be sorted according to the quality of each video sequence. The analysis targets in the video sequences with better quality can be analyzed first according to the sorted video sequences, which improves the efficiency of video sequence analysis and reduces the burden on doctors.

[0165] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 6 As shown, the electronic device 600 includes a processor 610, a memory 620, and a bus 630.

[0166] The memory 620 stores machine-readable instructions executable by the processor 610. When the electronic device 600 is running, the processor 610 and the memory 620 communicate via the bus 630. When the machine-readable instructions are executed by the processor 610, they can perform the operations described above. Figure 1 as well as Figure 2 The specific implementation of the video sequence sorting method in the method embodiment shown can be found in the method embodiment, and will not be repeated here.

[0167] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 as well as Figure 2 The specific implementation of the video sequence sorting method in the method embodiment shown can be found in the method embodiment, and will not be repeated here.

[0168] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0169] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0170] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0171] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0172] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0173] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for sorting video sequences, characterized in that, The sorting method includes: Obtain multiple video frames from each of the multiple video sequences captured for the target of analysis; For each video sequence, among multiple video frames in the video sequence, foreground frames that have the analysis target and background frames that do not have the analysis target are identified; The quality of the video sequence is determined based on the foreground frame and the background frame; The step of determining the quality of the video sequence based on the foreground frame and the background frame includes: Based on the foreground frame, the foreground frame is transformed in the frequency domain to obtain a foreground frequency domain map, and based on the background frame, the background frame is transformed in the frequency domain to obtain a background frequency domain map. Based on the foreground frequency domain map and the background frequency domain map, a target frequency domain map containing only the analysis target is obtained; The target frequency domain map is divided into multiple target frequency domain sub-regions according to a preset removal area, and the background frequency domain map is divided into multiple background frequency domain sub-regions according to a preset removal area; The image quality of the target frequency domain map is determined based on the plurality of target frequency domain sub-regions, and the image quality of the background frequency domain map is determined based on the plurality of background frequency domain sub-regions; The quality of the video sequence is determined based on the image quality of the target frequency domain map and the image quality of the background frequency domain map; The plurality of video sequences are sorted according to the quality of each video sequence, so that the analysis target is analyzed according to the sorted video sequences.

2. The ranking method of claim 1, wherein, The step of determining, among multiple video frames in the video sequence, the foreground frame containing the analysis target and the background frame not containing the analysis target includes: In the multiple video frames of the video sequence, for each video frame, obtain the grayscale value corresponding to each pixel in the video frame and the number of pixels included in the video frame; Add the grayscale values ​​corresponding to each pixel of the video frame to get the total grayscale value of the video frame; The quotient of the sum of the grayscale values ​​corresponding to the video frame and the number of pixels included in the video frame is determined as the average grayscale value corresponding to the video frame. Based on the average grayscale value of each video frame in the video sequence, the video frame with the largest average grayscale value is determined as the background frame without the analysis target, and the video frame with the smallest average grayscale value is determined as the foreground frame with the analysis target.

3. The ranking method of claim 1, wherein, The step of obtaining a target frequency domain map containing only the analysis target based on the foreground frequency domain map and the background frequency domain map includes: Based on the foreground frequency domain map, obtain the pixel value of each pixel in the foreground frequency domain map; Based on the background frequency domain map, obtain the pixel value of each pixel in the background frequency domain map; Subtract the pixel value of each pixel in the foreground frequency domain map from the pixel value of the corresponding pixel in the background frequency domain map to obtain a target frequency domain map containing only the analysis target.

4. The ranking method of claim 1, wherein, The step of determining the image quality of the target frequency domain map based on the plurality of target frequency domain sub-regions includes: Based on the plurality of target frequency domain sub-regions, obtain the maximum pixel value of each target frequency domain sub-region and the average variance of each target frequency domain sub-region in advance; For each target frequency domain sub-region, the quality of the target frequency domain sub-region is determined based on the maximum pixel value and average variance in that target frequency domain sub-region. Based on the quality of each target frequency domain sub-region, the sum of the qualities of each target frequency domain sub-region and the quotient of the number of target frequency domain sub-regions are determined as the image quality of the target frequency domain map.

5. The ranking method of claim 4, wherein, The maximum pixel value for each target frequency domain sub-region is determined using the following steps: Obtain the pixel value of each pixel in each of the multiple target frequency domain sub-regions; For each target frequency domain sub-region, among the pixel values ​​of all pixels in the target frequency domain sub-region, the pixel value with the largest value is determined as the maximum pixel value in the target frequency domain sub-region.

6. The ranking method of claim 4, wherein, The step of determining the quality of the video sequence based on the image quality of the target frequency domain map and the image quality of the background frequency domain map includes: Based on the image quality of the target frequency domain map, the product of the image quality of the target frequency domain map and the first preset weight is determined as the first quality component of the video sequence; Based on the image quality of the background frequency domain map, the product of the image quality of the background frequency domain map and the second preset weight is determined as the second quality component of the video sequence; wherein, the second preset weight is the difference between the number 1 and the first preset weight; The quality of the video sequence is determined by summing the first quality component and the second quality component of the video sequence.

7. An apparatus for ordering of a video sequence, characterized in that The sorting device includes: The acquisition module is used to acquire multiple video frames from each of the multiple video sequences captured for the analysis target. The first determining module is used to determine, for each video sequence, a foreground frame having the analysis target and a background frame not having the analysis target among multiple video frames in the video sequence; The second determining module is used to determine the quality of the video sequence based on the foreground frame and the background frame; When the second determining module is used to determine the quality of the video sequence based on the foreground frame and the background frame, the second determining module is specifically used for: Based on the foreground frame, the foreground frame is transformed in the frequency domain to obtain a foreground frequency domain map, and based on the background frame, the background frame is transformed in the frequency domain to obtain a background frequency domain map. Based on the foreground frequency domain map and the background frequency domain map, a target frequency domain map containing only the analysis target is obtained; The target frequency domain map is divided into multiple target frequency domain sub-regions according to a preset removal area, and the background frequency domain map is divided into multiple background frequency domain sub-regions according to a preset removal area; The image quality of the target frequency domain map is determined based on the plurality of target frequency domain sub-regions, and the image quality of the background frequency domain map is determined based on the plurality of background frequency domain sub-regions; The quality of the video sequence is determined based on the image quality of the target frequency domain map and the image quality of the background frequency domain map; The sorting module is used to sort the plurality of video sequences according to the quality of each video sequence, so as to analyze the analysis target according to the sorted video sequences.

8. An electronic device, comprising: include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. The machine-readable instructions are executed by the processor to perform the steps of the video sequence sorting method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the video sequence sorting method as described in any one of claims 1 to 6.

Images