A computer vision-based image picture display method and system, and a storage medium

Abstract

The application discloses a kind of based on computer vision's image picture display method, system and storage medium, it is related to image processing technical field, including: the motion video stream of target object is collected, target object area is positioned and effective analysis area is extracted, dominant frequency is detected;Extract fundamental component, based on the spectral coherence of angular velocity and dominant frequency determines synchronous rotation state, generates adaptive blur kernel and short-range blur kernel, and is convolved and superimposed as composite blur kernel;Wavelet decomposition is executed to the effective analysis area of original image frame, low-frequency sub-band is convolved and handled using composite blur kernel, based on direction selection corresponding short-range blur kernel, and local convolution is executed to each high-frequency detail sub-band, and reconstruction image is generated by inverse wavelet transform step-by-step fusion sub-band;The effective analysis area of corrected blur image and original image background area are weighted and fused, effectively reduce frequency domain aliasing phenomenon, to improve the quality and visual perception of image display.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an image picture display method and system based on computer vision and a storage medium. BACKGROUND

[0002] In the field of image technology, the image picture display method based on computer vision is a key technology, which widely relies on image processing techniques to achieve effective display of images. These traditional image processing techniques usually cover multiple core steps such as image segmentation, spectral analysis, phase correction, etc., and the purpose is to improve the quality of images or extract valuable information from them through a series of complex algorithms and operations.

[0003] However, in the existing image picture display method based on computer vision technology, there is a prominent defect, specifically, when adding motion blur effect to the image in an artificial way, usually only the filter kernel corresponding to the motion direction is simply superimposed on the image. This operation process does not fully consider the inherent frequency characteristics of the object itself. For example, for rotating machinery and other objects with periodic motion characteristics, their motion state contains specific frequency information. Due to the lack of consideration of the inherent frequency of the object, frequency domain aliasing phenomenon is easily caused in the image processing process. This frequency domain aliasing problem will directly reflect on the final visual presentation, resulting in an unpleasant "ghosting" effect in the image, which seriously affects the quality and visual perception of image display. SUMMARY

[0004] (I) Technical problems solved

[0005] In view of the deficiencies of the prior art, the present application provides an image picture display method and system based on computer vision and a storage medium. By introducing spectral analysis and dominant frequency detection technology, the motion characteristics of the target object are more accurately captured. By generating and superimposing adaptive blur kernels and short-range blur kernels, the motion blur effect is more accurately simulated. By phase alignment and weighted fusion, frequency domain aliasing phenomenon is effectively reduced, solving the image "ghosting" problem caused by ignoring the inherent frequency characteristics of the object in the prior art, thereby improving the quality and visual perception of image display.

[0006] (II) Technical solutions

[0007] To achieve the above purpose, the present application is implemented by the following technical solutions: an image picture display method based on computer vision, comprising:

[0008] Collecting a moving video stream of a target object, generating a sequence of original image frames, synchronously recording motion parameters, performing semantic segmentation on the sequence of image frames, locating a target object region and extracting an effective analysis region, performing spectral analysis on the effective analysis region, detecting a dominant frequency and harmonic energy thereof;

[0009] Locating the dominant frequency according to a sequence of linear velocities in the motion parameters, constructing a narrow-band filter to extract a fundamental component, determining a synchronous rotation state based on spectral coherence of the angular velocity and the dominant frequency, generating an adaptive blur kernel and a short-range blur kernel, and convolving and superimposing the adaptive blur kernel and the short-range blur kernel into a composite blur kernel;

[0010] Performing wavelet decomposition on the effective analysis region of the original image frame, decomposing into a low-frequency subband and a plurality of high-frequency detail subbands, convolving the low-frequency subband using the composite blur kernel, selecting a corresponding short-range blur kernel based on a direction, and performing local convolution on each high-frequency detail subband, generating a reconstructed image by gradually fusing the subbands through inverse wavelet transform, extracting phase information of the reconstructed image, and aligning the phase information with that of the original image to generate a corrected blurred image;

[0011] Weightedly fusing the effective analysis region of the corrected blurred image with a background region of the original image, verifying whether a high-frequency residual energy ratio meets a preset residual suppression threshold, and rendering and outputting an image sequence that passes the verification to a display device.

[0012] Further, the target object region is located by a semantic segmentation network, a mask is generated, a region of a fixed percentage area of the target object center is intercepted as an effective analysis region based on the mask, a two-dimensional fast Fourier transform is performed on the effective analysis region, an energy threshold is set in the frequency spectrum, local peak values are detected by a non-maximum suppression algorithm, frequency points with amplitudes greater than the energy threshold are retained as candidate frequency points, the first three harmonic energies of the candidate frequency points are calculated, and the frequency with the maximum total energy is selected as the dominant frequency .

[0013] Further, a discrete Fourier transform is performed on the sequence of linear velocities to generate an original frequency spectrum, the position of the dominant frequency is located in the original frequency spectrum, a narrow-band filter with a bandwidth of is constructed with the dominant frequency as the center frequency, the fundamental frequency spectrum and the fundamental component are extracted, and the high-frequency residual component is calculated by the difference between the original frequency spectrum and the fundamental frequency spectrum;

[0014] The angular velocity is converted into a circular frequency, the spectral coherence coefficient of the circular frequency and the dominant frequency is calculated, if the spectral coherence coefficient is greater than a coherence threshold, it is determined as synchronous rotation, otherwise, it is marked as asynchronous disturbance.

[0015] Further, an elliptical band-pass filter is constructed with the dominant frequency as a constraint, including a center frequency of and a bandwidth of 0.1 And set the filter order, if there is asynchronous disturbance, the bandwidth is extended to 0.2 ;

[0016] Initialize the rectangular blur kernel, length is set to: Wherein, is the time interval between adjacent frames, and a is a safety factor, , is the fundamental component, using a two-dimensional Gaussian function to generate the kernel value: Wherein, x , y are the horizontal and vertical coordinates of the pixel, is the attenuation coefficient, , perform fast Fourier transform on the rectangular blur kernel, and multiply it point by point with the elliptical band-pass filter, and generate an adaptive blur kernel through inverse fast Fourier transform .

[0017] Further, the pixel displacement amount is calculated for the high-frequency residual component: Wherein, is the high-frequency residual component, and the pixel displacement amount is mapped to the length, generating a short-range blur kernel , the generation method is the same as that of the rectangular blur kernel, and the adaptive blur kernel and the short-range blur kernel are convolved and superimposed to generate a composite blur kernel:

[0018] .

[0019] Further, wavelet decomposition is performed on the effective analysis area of the original image frame, which is decomposed into a low-frequency subband and several high-frequency detail subbands. The composite blur kernel is used for convolution operation on the low-frequency subband; for each high-frequency detail subband, the corresponding short-range blur kernel is selected according to the direction, and local convolution is performed: Wherein, is the value of the blurred high-frequency detail subband at the pixel position x, y , is the pixel value of the high-frequency detail subband at the offset position x + m, y + n , is the value of the short-range blur kernel at the position m, n , r is the radius of the short-range blur kernel.

[0020] Further, the inverse discrete wavelet transform is adopted to gradually fuse the low-frequency sub-band and several high-frequency detail sub-bands to generate a reconstructed image, a fast Fourier transform is performed on the reconstructed image to extract a complex spectrum, a phase is calculated through a real part and an imaginary part of the complex spectrum, a difference between a spectrum phase of the original image frame and a phase of the blurred image is calculated, the phase alignment is realized through a complex spectrum rotation, a corrected spectrum is obtained, an inverse fast Fourier transform is performed on the phase-corrected spectrum to generate a phase-aligned blurred image.

[0021] Further, an effective analysis region of the phase-aligned blurred image and a background region of the original image frame are weightedly fused, wherein, is the phase-aligned blurred image, is the original image frame, β is a fusion weight matrix, in the effective analysis region in the background region in the boundary transition region wherein, d represents the shortest distance, D represents the maximum width of the boundary transition region;

[0022] A two-dimensional fast Fourier transform is performed on the weightedly fused image, a high-frequency cutoff frequency is set, when the frequency is greater than the high-frequency cutoff frequency, it is marked as a high-frequency band, a high-frequency residual energy ratio is calculated: high-frequency residual energy ratio = high-frequency band energy / full-frequency band energy, if the high-frequency residual energy ratio is less than a residual suppression threshold, the high-frequency aliasing suppression is qualified, otherwise, the dominant frequency is recalculated.

[0023] An image picture display system based on computer vision, comprising:

[0024] A data acquisition module acquires a motion video stream of a target object, generates an original image frame sequence, synchronously records motion parameters, performs semantic segmentation on the image frame sequence, locates a target object region and extracts an effective analysis region, and performs spectrum analysis on the effective analysis region to detect a dominant frequency and harmonic energy thereof;

[0025] An adaptive filtering module locates the dominant frequency according to a linear velocity sequence in the motion parameters, constructs a narrow-band filter to extract a fundamental wave component, judges a synchronous rotation state based on spectral coherence of the angular velocity and the dominant frequency, generates an adaptive blur kernel and a short-range blur kernel, and convolves and superimposes them into a composite blur kernel;

[0026] The image processing module performs wavelet decomposition on the effective analysis area of the original image frame, decomposes into a low-frequency subband and a plurality of high-frequency detail subbands, convolves the low-frequency subband using a composite blur kernel, selects a corresponding short-range blur kernel based on the direction, and performs local convolution on each high-frequency detail subband, generates a reconstructed image by gradually fusing the subbands through inverse wavelet transform, extracts the phase information of the reconstructed image, and aligns the phase with the original image to generate a corrected blurred image;

[0027] The image output module performs weighted fusion on the effective analysis area of the corrected blurred image and the background area of the original image, verifies whether the high-frequency residual energy ratio meets the preset residual suppression threshold, renders and outputs the image sequence that passes the verification to a display device.

[0028] A storage medium stores one or more computer instructions for implementing the computer vision-based image display method described above.

[0029] (Three) beneficial effects

[0030] The application provides a computer vision-based image display method, system and storage medium, which has the following beneficial effects:

[0031] (1) By performing frequency spectrum analysis on the effective analysis area to detect the dominant frequency and its harmonic energy, the motion characteristics of the target object can be accurately obtained, which provides key data support for the generation of adaptive blur kernel and phase correction, effectively reduces the image "ghosting" problem caused by ignoring the inherent frequency characteristics of the object, and significantly improves the quality and visual perception of image display.

[0032] (2) By accurately extracting the fundamental component and determining the synchronous rotation state, the adaptive blur kernel and short-range blur kernel highly matched with the motion characteristics of the target object are generated, the real motion blur effect is effectively simulated, the frequency domain aliasing phenomenon is significantly reduced, and the clarity and realism of image display are greatly improved, and the visual perception is optimized.

[0033] (3) By wavelet decomposition and composite blur kernel processing of the image, combined with the phase alignment technology, the motion blurred image is accurately reconstructed, the image distortion caused by frequency domain aliasing is effectively reduced, the image detail retention and overall visual effect are significantly improved, the displayed image is more natural and clear, and the viewing experience is enhanced.

[0034] (4) By weighted fusion of the corrected blurred image and the original background, and verification of the high-frequency residual energy ratio, the frequency domain aliasing phenomenon is effectively suppressed, the high-frequency details of the image are reasonably retained, the overall quality and visual coherence of the image display are significantly improved, and the final output image is more realistic and natural. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 The step schematic diagram of the image picture display method based on computer vision of the present application;

[0036] Figure 2 The structure schematic diagram of the image picture display system based on computer vision of the present application. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the present application.

[0038] Please refer to Figure 1 The present application provides an image picture display method based on computer vision, comprising the following steps:

[0039] Step one: collect the motion video stream of the target object, generate the original image frame sequence, synchronously record the motion parameters, perform semantic segmentation on the image frame sequence, locate the target object region and extract the effective analysis region, perform spectrum analysis on the effective analysis region, and detect the dominant frequency and its harmonic energy;

[0040] The step one comprises the following contents:

[0041] Step 101: collect the motion video stream of the target object by an industrial area camera, and generate the original image frame sequence Wherein, record the motion parameters of the target object, including the linear velocity, the angular velocity and the motion direction, the linear velocity is measured by an encoder in real time (unit: pixel / frame), the angular velocity is obtained by a rotation speed sensor (unit: radian / frame), and the motion direction is calibrated by the initial position;

[0042] When the motion parameters are unknown, the optical flow method (such as Lucas-Kanade optical flow algorithm) is used, the edge feature points of the target object are taken as the tracking targets, the displacement of the feature points between adjacent frames is calculated, and the linear velocity is obtained: Wherein, And are the displacements of the feature points on the x axis and the y axis respectively, is the time interval between adjacent frames; the angular velocity is calculated by the displacement direction change rate :

[0043] ​It should be noted that, in order to improve the tracking accuracy of feature points, a Shi-Tomasi corner detector is used to extract a plurality of high-confidence feature points of the edge of the target object, and a pyramid LK optical flow is used to realize sub-pixel level accuracy tracking, and the median error is less than or equal to 0.15 pixels;

[0044] Step 102: For the collected image frame sequence, the target object region is located by a U-Net semantic segmentation network, and a mask is output , and a fixed percentage (such as 75%) area region of the target object center is intercepted as an effective analysis region, a sliding step (such as 256 pixels) is set, and a two-dimensional fast Fourier transform (FFT) is performed on the effective analysis region to cover the entire effective analysis region;

[0045] In order to reduce spectral leakage, a Hanning window is used for windowing processing on the effective analysis region, and in the spectral preprocessing stage, a logarithmic transformation is used to enhance low-energy components and use Gaussian filtering to smooth spectral noise, wherein, represents the amplitude (modulus) of the complex spectrum obtained after the Fourier transform of the image frame I, and the absolute value is taken in order to convert the complex spectrum into a non-negative real amplitude, represents a small positive constant, such as , to prevent the input of the logarithmic operation from being zero or negative;

[0046] It should be noted that the mask is a binary matrix (or grayscale image) with the same size as the input image, which is used to mark the region of interest or shield in the image, and its mathematical expression is:

[0047] The mask can extract the accurate contour of the target object and avoid the interference of the background on the frequency domain analysis;

[0048] Step 103: Set the energy threshold: , wherein, μ is the spectral mean, σ is the standard deviation, and when detecting the energy peak value in the spectrum, a non-maximum suppression algorithm is used to detect the local peak value (amplitude) in the sliding detection window (such as 8-neighborhood), and the frequency point with an amplitude greater than the energy threshold is reserved as a candidate frequency point;

[0049] Step 104: For the detected candidate frequency point , the first three harmonics are calculated: , wherein, i is the serial number of the candidate frequency point, k is the order, k =1, 2, 3, and a dynamic frequency band is intercepted around each in the spectrum: , wherein, The Nyquist frequency; calculate the harmonic energy: wherein, f represents the frequency, and the total energy The largest frequency is selected as the dominant frequency;

[0050] In use, the contents of steps 101 to 104 are combined:

[0051] By performing a spectral analysis on the effective analysis area to detect the dominant frequency and its harmonic energy, the motion characteristics of the target object can be accurately obtained, providing key data support for the subsequent generation of adaptive blur kernels and phase correction, effectively reducing the image "ghost" problem caused by ignoring the inherent frequency characteristics of the object, thereby significantly improving the quality and visual perception of the image display.

[0052] Step two: According to the linear velocity sequence in the motion parameter, the dominant frequency is located, a narrow band filter is constructed to extract the fundamental component, the spectral coherence of the angular velocity and the dominant frequency is judged to determine the synchronous rotation state, the adaptive blur kernel and the short-range blur kernel are generated, and the composite blur kernel is obtained by convolution and superposition;

[0053] The second step includes the following contents:

[0054] Step 201: Perform discrete Fourier transform (DFT) on the obtained linear velocity sequence to generate an original spectrum, locate the position of the dominant frequency in the original spectrum, and construct a narrow band filter with a bandwidth of around the center frequency to extract the fundamental component: wherein, IFFT () represents the inverse fast Fourier transform, represents the original spectrum obtained by performing discrete Fourier transform on the linear velocity sequence, rect () is a rectangular window function used to select a specific frequency range in the frequency domain, represents the dominant frequency, represents the bandwidth, f represents the frequency; calculate the high frequency residual component by the difference between the original spectrum and the fundamental spectrum: wherein, is the fundamental spectrum, ;

[0055] Step 202: Convert the angular velocity to the circular frequency: wherein, ω represents the angular velocity, and the spectral coherence coefficient of the circular frequency and the dominant frequency is calculated: wherein, represents the mutual power spectral density of the dominant frequency and the circular frequency, represents the dominant frequency self-power spectral density, The circular frequency self-power spectrum density is represented; if the spectral coherence coefficient is greater than a coherence threshold (such as 0.8), it is determined that the rotation is synchronous, otherwise, it is marked as an asynchronous disturbance;

[0056] Step 203: Construct an elliptical band-pass filter with the dominant frequency as a constraint , including taking as the center frequency, the bandwidth is 0.1 , and setting the filter order (such as 8 orders), if there is an asynchronous disturbance, the bandwidth is expanded to 0.2 ; initialize the rectangular blur kernel, the length is set to: , wherein is the time interval between adjacent frames, and alpha represents a safety factor, , a two-dimensional Gaussian function is used to generate the kernel value:

[0057] , wherein x , y are the horizontal and vertical coordinates of the pixel, is the attenuation coefficient, ; after performing fast Fourier transform on the rectangular blur kernel, multiply it point by point with the elliptical band-pass filter: , denotes point-by-point multiplication, and after inverse fast Fourier transform, an adaptive blur kernel is generated;

[0058] Step 204: Calculate the pixel displacement of the high-frequency residual component: , map the pixel displacement to the length to generate a short-range blur kernel , the short-range blur kernel generation method is the same as the generation of the rectangular blur kernel in step 203, and the adaptive blur kernel and the short-range blur kernel are convolved and superimposed to generate a composite blur kernel:

[0059] , wherein * represents convolution operation;

[0060] When used, the contents of steps 201 to 204 are combined:

[0061] By accurately extracting the fundamental component and determining the synchronous rotation state, an adaptive blur kernel and a short-range blur kernel highly matched with the motion characteristics of the target object are generated, effectively simulating the real motion blur effect, significantly reducing the frequency domain aliasing phenomenon, and further greatly improving the clarity and realism of image display, and optimizing the visual perception.

[0062] Step 3: Perform wavelet decomposition on the effective analysis region of the original image frame, decomposing it into low-frequency sub-bands and multiple high-frequency detail sub-bands; use a composite blur kernel to perform convolution processing on the low-frequency sub-bands, select the corresponding short-range blur kernel based on the direction, and perform local convolution on each high-frequency detail sub-band. Use inverse wavelet transform to fuse the sub-bands step by step to generate a reconstructed image, extract the phase information of the reconstructed image, and align it with the phase of the original image to generate a corrected blurred image.

[0063] Step three includes the following:

[0064] Step 301: Perform wavelet decomposition (such as Daubechies-4 wavelet basis) on the effective analysis region of the original image frame, decomposing it into low-frequency sub-bands and several high-frequency detail sub-bands. Use a composite blur kernel to perform convolution operation on the low-frequency sub-bands, and use fast Fourier transform to accelerate the calculation and generation of blurred values.

[0065] ,in, Indicates the low-frequency sub-band;

[0066] Step 302: For each high-frequency detail subband, select the corresponding short-range blur kernel according to the direction and perform local convolution: ,in, This indicates the high-frequency detail subband at the pixel location after blurring. x, y The value at ) Indicates the high-frequency detail subband at the offset position ( x + m, y + n The pixel value at position () is the short-range blur kernel at position () m, n The value at ) r Represents the radius of the short-range fuzzy kernel (corresponding to kernel size ). );

[0067] Step 303: Use inverse discrete wavelet transform to fuse low-frequency subbands and several high-frequency detail subbands step by step. For example, if a 3-level discrete wavelet transform is used in wavelet decomposition, then a 3-level inverse discrete wavelet transform is used to fuse the subbands step by step.

[0068] Level 1 Reconstruction: Merge the Level 3 low-frequency subband with the Level 3 high-frequency subband to generate the Level 2 low-frequency approximation;

[0069] Level 2 Reconstruction: The Level 2 low-frequency approximation is fused with the Level 2 high-frequency subband to generate the Level 1 low-frequency approximation;

[0070] Level 3 Reconstruction: The Level 3 low-frequency approximation is fused with the Level 1 high-frequency subband to generate a full-resolution reconstructed image. ;

[0071] Step 304: Reconstruct the image Performing fast Fourier transform, extracting complex spectrum, calculating phase through real part and imaginary part of complex spectrum: , wherein, represents complex spectrum, u, v represents spectrum coordinate, Im represents imaginary part, Re represents imaginary part;

[0072] Step 305: Calculate the difference between the original image frame spectrum phase and the blur image phase, and generate a phase correction matrix: , wherein, represents the original image frame spectrum phase, and the phase alignment is realized by complex spectrum rotation to obtain the corrected spectrum: , wherein j represents imaginary unit, and inverse fast Fourier transform is performed on the phase-corrected spectrum to generate the phase-aligned blur image;

[0073] In use, the contents of steps 301 to 305 are combined:

[0074] By wavelet decomposition and composite blur kernel processing image, combined with phase alignment technology, the motion blur image is accurately reconstructed, the image distortion caused by frequency domain aliasing is effectively reduced, the image detail retention and overall visual effect are significantly improved, the displayed image is more natural and clear, and the viewing experience is enhanced.

[0075] Step four: Weighted fusion of the effective analysis area of the corrected blur image and the background area of the original image, verify whether the high-frequency residual energy ratio meets the preset residual suppression threshold, render and output the image sequence that passes the verification to the display device.

[0076] The step four includes the following contents:

[0077] Step 401: Weighted fusion of the effective analysis area (i.e. 75% area of the target object center cut in step 102) of the phase-aligned blur image and the background area of the original image frame:

[0078] , wherein, is the phase-aligned blur image, is the original image frame, β is the fusion weight matrix, in the effective analysis area, in the background area (non-target object area), and in the boundary transition area (part of the target object in the non-effective analysis area, i.e. the remaining 25% area) adopts cosine function gradual change dThe shortest distance represents the shortest distance of each pixel in the boundary transition region to the effective analysis region, and D represents the maximum width of the boundary transition region;

[0079] Step 402: performing two-dimensional fast Fourier transform on the weighted fused image, setting a high-frequency cutoff frequency When the frequency is greater than the high-frequency cutoff frequency, it is marked as a high-frequency band, and the high-frequency residual energy ratio is calculated: high-frequency residual energy ratio = high-frequency band energy / full-frequency band energy, if the high-frequency residual energy ratio is less than the residual suppression threshold (such as 5%), the high-frequency aliasing suppression is qualified, otherwise, return to step one to recalculate the dominant frequency;

[0080] Step 403: sorting the verified images according to the video stream timestamp, outputting to the display device through the graphics rendering interface (such as OpenGL / Vulkan), and storing as a video file (encoding format H.264, frame rate consistent with the input video stream).

[0081] In use, the contents of steps 401 to 403 are combined:

[0082] By weighting and fusing the corrected blurred image and the original background, and verifying the high-frequency residual energy ratio, the frequency domain aliasing phenomenon is effectively suppressed, ensuring that the high-frequency details of the image are reasonably preserved, thereby significantly improving the overall quality and visual coherence of the image display, making the final output image more realistic and natural.

[0083] Please refer to Figure 2 The application also provides an image picture display system based on computer vision, comprising: a data acquisition module, an adaptive filtering module, an image processing module, and an image output module, wherein:

[0084] The data acquisition module acquires the motion video stream of the target object, generates the original image frame sequence, synchronously records the motion parameters, performs semantic segmentation on the image frame sequence, locates the target object region and extracts the effective analysis region, performs frequency spectrum analysis on the effective analysis region, and detects the dominant frequency and its harmonic energy;

[0085] The adaptive filtering module locates the dominant frequency according to the linear velocity sequence in the motion parameters, constructs a narrowband filter to extract the fundamental wave component, judges the synchronous rotation state based on the spectral coherence of the angular velocity and the dominant frequency, generates an adaptive blur kernel and a short-range blur kernel, and convolves and superimposes them into a composite blur kernel;

[0086] The image processing module performs wavelet decomposition on an effective analysis area of an original image frame, and decomposes into a low-frequency subband and a plurality of high-frequency detail subbands; a composite blur kernel is used to perform convolution processing on the low-frequency subband, a corresponding short-range blur kernel is selected based on a direction, and local convolution is performed on each high-frequency detail subband, a reconstructed image is generated by gradually fusing the subbands through inverse wavelet transformation, phase information of the reconstructed image is extracted, and a corrected blurred image is generated by aligning the phase with the original image;

[0087] The image output module performs weighted fusion on an effective analysis area of the corrected blurred image and a background area of the original image, verifies whether a high-frequency residual energy ratio meets a preset residual suppression threshold, renders and outputs an image sequence that passes the verification to a display device.

[0088] The application further provides a storage medium, which stores one or more computer instructions for implementing the computer vision-based image picture display method.

[0089] In the application, the several formulas involved are all dimensionless numerical calculations, and the formula is obtained by software simulation of a large amount of collected data to obtain a formula of the nearest real situation, and the coefficients in the formula are set by a person skilled in the art according to the actual situation.

[0090] The above embodiments can be realized wholly or partially by software, hardware, firmware or any combination thereof. When realized by software, the above embodiments can be realized in the form of a computer program product wholly or partially. Those skilled in the art can realize that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be realized by electronic hardware, computer software and a combination of electronic hardware and computer software. Whether the functions are realized by hardware or software depends on the specific application and design constraints of the technical solutions.

[0091] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, and can be located in one place or distributed on multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiment according to actual needs.

[0092] The above is only a specific implementation of the application, but the protection scope of the application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the application, which should be covered within the protection scope of the application.

Claims

1. A computer vision-based image frame presentation method, characterized by: Comprise: Collecting a moving video stream of a target object, generating a sequence of original image frames, synchronously recording motion parameters, performing semantic segmentation on the sequence of image frames, locating a target object region and extracting an effective analysis region, performing spectral analysis on the effective analysis region, detecting a dominant frequency and its harmonic energy; Locating the dominant frequency according to the linear velocity sequence in the motion parameters, constructing a narrowband filter to extract the fundamental component, determining the synchronous rotation state based on the spectral coherence of the angular velocity and the dominant frequency, generating an adaptive blur kernel and a short-range blur kernel, and convolving and superimposing them into a composite blur kernel; Performing wavelet decomposition on the effective analysis region of the original image frame, decomposing it into a low-frequency subband and a plurality of high-frequency detail subbands, convolving the low-frequency subband with the composite blur kernel, selecting the corresponding short-range blur kernel based on the direction, and performing local convolution on each high-frequency detail subband, generating a reconstructed image by gradually fusing the subbands through inverse wavelet transform, extracting the phase information of the reconstructed image, and aligning the phase with the original image to generate a corrected blurred image; Weighted fusion of the effective analysis region of the corrected blurred image and the background region of the original image, verification of whether the high-frequency residual energy ratio meets the preset residual suppression threshold, rendering of the verified image sequence, and output to a display device.

2. The image display method based on computer vision according to claim 1, wherein: The target object region is located by a semantic segmentation network, a mask is generated, a fixed percentage area region of the center of the target object is intercepted as an effective analysis region based on the mask, a two-dimensional fast Fourier transform is performed on the effective analysis region, an energy threshold is set in the frequency spectrum, local peak values are detected by a non-maximum suppression algorithm, frequency points with an amplitude greater than the energy threshold are reserved as candidate frequency points, the harmonic energy of the first three orders of the candidate frequency points is calculated, and the frequency with the maximum total energy is selected as the dominant frequency .

3. The image display method based on computer vision according to claim 1, wherein: Performing a discrete Fourier transform on the sequence of linear velocities generates a raw spectrum, the position of the dominant frequency is located in the raw spectrum, a narrow-band filter with a bandwidth of is constructed with the dominant frequency as the center frequency, the fundamental spectrum and the fundamental component are extracted, and the high-frequency residual component is calculated by the difference between the raw spectrum and the fundamental spectrum; Convert the angular velocity to the circular frequency, calculate the spectral coherence coefficient of the circular frequency and the dominant frequency, and if the spectral coherence coefficient is greater than the coherence threshold, determine that it is synchronous rotation; otherwise, mark it as asynchronous disturbance.

4. The image display method based on computer vision according to claim 3, wherein: Constructing an elliptic bandpass filter with a dominant frequency as a constraint, including a center frequency of 0.1 and setting the filter order, if there is an asynchronous disturbance, the bandwidth is expanded to 0.2 ; Initialize the rectangular blur kernel with length set to: where, is the time interval between adjacent frames, and α is a safety factor, , is the fundamental component, and the kernel value is generated using a two-dimensional Gaussian function: wherein, x , y are horizontal and vertical coordinates of the pixel, is an attenuation coefficient, After performing a fast Fourier transform on the rectangular blur kernel, multiply it point by point with the elliptical band-pass filter, and generate an adaptive blur kernel after inverse fast Fourier transform .

5. The image display method based on computer vision according to claim 4, wherein: calculating pixel displacement amount for high frequency residual component: wherein, mapping pixel displacement amount to length for high frequency residual component, generating short-range blur kernel which is generated in the same way as the rectangular blur kernel, and the adaptive blur kernel and the short-range blur kernel are convolved and superimposed to generate a composite blur kernel: 。 6. The image display method based on computer vision according to claim 1, wherein: Wavelet decomposition is performed on the effective analysis region of the original image frame into a low-frequency subband and several high-frequency detail subbands, and a composite blur kernel is used to perform convolution operation on the low-frequency subband; for each high-frequency detail subband, a corresponding short-range blur kernel is selected according to the direction, and local convolution is performed: ; wherein is the value of the blurred high frequency detail subband at pixel position (x, y), x, y is the pixel value of the high frequency detail subband at offset position (x, y), x + m, y + n is the value of the short range blur kernel at position (x, y), m, n r is the radius of the short range blur kernel.​​​ 7. The image display method based on computer vision according to claim 6, wherein: Use inverse discrete wavelet transform to gradually fuse the low-frequency subband and a plurality of high-frequency detail subbands to generate a reconstructed image, perform fast Fourier transform on the reconstructed image to extract a complex spectrum, calculate the phase through the real part and the imaginary part of the complex spectrum, calculate the difference between the spectrum phase of the original image frame and the phase of the blurred image, achieve phase alignment through complex spectrum rotation, obtain the corrected spectrum, and perform inverse fast Fourier transform on the phase-corrected spectrum to generate a phase-aligned blurred image.

8. The image display method based on computer vision according to claim 1, wherein: weighting and fusing the valid analysis region of the phase-aligned blurred image with the background region of the original image frame, wherein, is the phase-aligned blurred image, is the original image frame, β is the fusion weight matrix, within the valid analysis region within the background region within the boundary transition region wherein, d denotes the shortest distance, D denotes the maximum width of the boundary transition region; Performing two-dimensional fast Fourier transform on the image after weighted fusion, setting a high-frequency cutoff frequency, marking as a high-frequency band when the frequency is greater than the high-frequency cutoff frequency, calculating a high-frequency residual energy ratio: high-frequency residual energy ratio = high-frequency band energy / full-frequency band energy, if the high-frequency residual energy ratio is less than a residual suppression threshold, the high-frequency aliasing suppression is qualified, otherwise, recalculate the dominant frequency.

9. A computer vision based image rendering system for implementing the method of any one of claims 1 to 8, characterized in that: The method comprises the following steps: A data acquisition module acquires a motion video stream of a target object, generates a sequence of original image frames, synchronously records motion parameters, performs semantic segmentation on the sequence of image frames, locates a target object region and extracts an effective analysis region, performs spectral analysis on the effective analysis region, and detects a dominant frequency and harmonic energy thereof; An adaptive filtering module locates the dominant frequency according to a linear velocity sequence in the motion parameters, constructs a narrowband filter to extract a fundamental component, determines a synchronous rotation state based on spectral coherence of an angular velocity and the dominant frequency, generates an adaptive blur kernel and a short-range blur kernel, and convolves and superimposes the adaptive blur kernel and the short-range blur kernel into a composite blur kernel; An image processing module performs wavelet decomposition on the effective analysis region of the original image frame, decomposes the effective analysis region into a low-frequency subband and a plurality of high-frequency detail subbands, convolves and processes the low-frequency subband using the composite blur kernel, selects a corresponding short-range blur kernel based on a direction, and performs local convolution on each high-frequency detail subband, gradually fuses the subbands through inverse wavelet transform to generate a reconstructed image, extracts phase information of the reconstructed image, and aligns the phase information with a phase of an original image to generate a corrected blurred image; An image output module performs weighted fusion on an effective analysis region of the corrected blurred image and a background region of an original image, verifies whether a high-frequency residual energy ratio meets a preset residual suppression threshold, and renders and outputs a sequence of images that pass the verification to a display device.

10. A storage medium characterized by: The storage medium stores one or more computer instructions for implementing the method of any one of claims 1 to 8. The storage medium stores one or more computer instructions for implementing the method of any one of claims 1 to 8.

Images