Image picture display method and system based on computer vision and storage medium

Abstract

The invention discloses an image picture display method and system based on computer vision and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: collecting a motion video stream of a target object, positioning a target object region, extracting an effective analysis region, and detecting a dominant frequency; extracting a fundamental component, judging a synchronous rotation state based on spectrum coherence of an angular velocity and a dominant frequency, generating an adaptive fuzzy kernel and a short-range fuzzy kernel, and performing convolution superposition to form a composite fuzzy kernel; performing wavelet decomposition on an effective analysis area of an original image frame, performing convolution processing on a low-frequency sub-band by using a composite blurring kernel, selecting a corresponding short-range blurring kernel based on a direction, performing local convolution on each high-frequency detail sub-band, and fusing the sub-bands step by step through inverse wavelet transform to generate a reconstructed image; the effective analysis area of the corrected blurred image and the background area of the original image are subjected to weighted fusion, and the frequency domain aliasing phenomenon is effectively reduced, so that the image display quality and the visual impression are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a computer vision-based image display method, system and storage medium. Background Art

[0002] In the field of image technology, the image display method based on computer vision is a key technology. It widely relies on image processing technology to achieve effective image display. These traditional image processing technologies usually cover multiple core steps such as image segmentation, spectrum analysis, and phase correction. Their purpose is to improve the quality of the image or extract valuable information from it through a series of complex algorithms and operations.

[0003] However, there is a prominent defect in the existing image display methods based on computer vision technology. Specifically, when manually adding motion blur effects to images, the filter kernel corresponding to the motion direction is usually simply superimposed directly on the image. This operation process does not fully consider the inherent frequency characteristics of the object itself. For example, for objects with periodic motion characteristics such as rotating machinery, their motion state contains specific frequency information. Due to the lack of consideration of the object's inherent frequency, frequency domain aliasing is easily caused during the image processing process. This frequency domain aliasing problem will be directly reflected in the final visual presentation, resulting in an unpleasant "ghosting" effect in the image, seriously affecting the quality of image display and visual perception. Summary of the Invention

[0004] (1) Technical problems solved In response to the shortcomings of the existing technology, the present invention provides an image display method, system and storage medium based on computer vision. By introducing spectrum analysis and dominant frequency detection technology, the motion characteristics of the target object can be more accurately captured; the motion blur effect can be more accurately simulated through the generation and superposition of adaptive blur kernels and short-range blur kernels; through phase alignment and weighted fusion, the frequency domain aliasing phenomenon is effectively reduced, solving the image "ghosting" problem caused by ignoring the inherent frequency characteristics of the object in the existing technology, thereby improving the quality of image display and visual perception.

[0005] (2) Technical solution To achieve the above objectives, the present invention is implemented through the following technical solutions: a computer vision-based image display method, comprising: Capture the target object's motion video stream, generate an original image frame sequence, synchronously record motion parameters, perform semantic segmentation on the image frame sequence, locate the target object area and extract the effective analysis area, perform spectrum analysis on the effective analysis area, and detect the dominant frequency and its harmonic energy; The dominant frequency is located according to the linear velocity sequence in the motion parameters, and a narrowband filter is constructed to extract the fundamental component. The synchronous rotation state is determined based on the spectral coherence between the angular velocity and the dominant frequency. An adaptive blur kernel and a short-range blur kernel are generated and convoluted to form a composite blur kernel. Perform wavelet decomposition on the effective analysis area of ​​the original image frame, decomposing it into a low-frequency sub-band and multiple high-frequency detail sub-bands. Use a composite blur kernel to convolve the low-frequency sub-band. Select the corresponding short-range blur kernel based on the direction, and perform local convolution on each high-frequency detail sub-band. Use inverse wavelet transform to fuse the sub-bands step by step to generate a reconstructed image. Extract the phase information of the reconstructed image and align it with the phase of the original image to generate a corrected blurred image. The effective analysis area of ​​the corrected blurred image is weightedly fused with the background area of ​​the original image to verify whether the high-frequency residual energy ratio meets the preset residual suppression threshold. The verified image sequence is rendered and output to the display device.

[0006] Furthermore, the target object area is located through the semantic segmentation network, a mask is generated, and a fixed percentage area of ​​the target object center is intercepted based on the mask as the effective analysis area. A two-dimensional fast Fourier transform is performed on the effective analysis area, and an energy threshold is set in the spectrum. The local peak is detected by the non-maximum suppression algorithm, and the frequency points with amplitudes greater than the energy threshold are retained as candidate frequency points. The energy of the first three harmonics of the candidate frequency points is calculated, and the frequency with the largest total energy is selected as the dominant frequency. .

[0007] Furthermore, discrete Fourier transform is performed on the linear velocity sequence to generate the original spectrum, the position of the dominant frequency is located in the original spectrum, and the bandwidth is constructed with it as the center frequency. The fundamental spectrum and fundamental component are extracted by using a narrowband filter, and the high-frequency residual component is calculated by the difference between the original spectrum and the fundamental spectrum; The angular velocity is converted into circumferential frequency, and the spectral coherence coefficient between the circumferential frequency and the dominant frequency is calculated. If the spectral coherence coefficient is greater than the coherence threshold, it is determined to be synchronous rotation; otherwise, it is marked as asynchronous disturbance.

[0008] Furthermore, an elliptical bandpass filter with dominant frequency as constraint is constructed, including is the center frequency and the bandwidth is 0.1 , and set the filter order. If there is asynchronous disturbance, the bandwidth is extended to 0.2 ; Initialize the rectangular blur kernel with the length set to: ,in, is the time interval between adjacent frames, α is the safety factor, , is the fundamental component, and a two-dimensional Gaussian function is used to generate the kernel value: ,in, x 、 y are the horizontal and vertical coordinates of the pixel, is the attenuation coefficient, After performing fast Fourier transform on the rectangular blur kernel, it is multiplied point by point with the elliptical bandpass filter, and the adaptive blur kernel is generated by inverse fast Fourier transform. .

[0009] Furthermore, the pixel displacement is calculated for the high-frequency residual component: ,in, For the high-frequency residual component, the pixel displacement is mapped to length to generate a short-range blur kernel The generation method is the same as that of the rectangular blur kernel. The adaptive blur kernel and the short-range blur kernel are convolved and superimposed to generate a composite blur kernel: .

[0010] Furthermore, wavelet decomposition is performed on the effective analysis area of ​​the original image frame, decomposing it into low-frequency sub-bands and several high-frequency detail sub-bands. A composite blur kernel is used to perform a convolution operation on the low-frequency sub-bands. For each high-frequency detail sub-band, the corresponding short-range blur kernel is selected according to the direction, and local convolution is performed: ,in, The high-frequency detail subband after blurring is at the pixel position ( x, y ), The high frequency detail subband is at the offset position ( x + m, y + n ), is the short-range blur kernel at position ( m, n ), r is the radius of the short-range blur kernel.

[0011] Furthermore, an inverse discrete wavelet transform is used to gradually fuse low-frequency subbands and several high-frequency detail subbands to generate a reconstructed image. A fast Fourier transform is performed on the reconstructed image to extract the complex spectrum. The phase is calculated by the real and imaginary parts of the complex spectrum. The difference between the phase of the original image frame spectrum and the phase of the blurred image is calculated. The phase is aligned by rotating the complex spectrum to obtain a corrected spectrum. An inverse fast Fourier transform is performed on the phase-corrected spectrum to generate a phase-aligned blurred image.

[0012] Furthermore, the effective analysis area of ​​the phase-aligned blurred image is weightedly fused with the background area of ​​the original image frame. ,in, is the phase-aligned blurred image, is the original image frame, β is the fusion weight matrix, within the effective analysis area , in the background area , in the boundary transition area ,in, d represents the shortest distance, and D represents the maximum width of the boundary transition area; Perform two-dimensional fast Fourier transform on the weighted fused image, set the high-frequency cutoff frequency, and when the frequency is greater than the high-frequency cutoff frequency, mark it as the high-frequency band, and calculate the high-frequency residual energy ratio: high-frequency residual energy ratio = high-frequency band energy / full-band energy. If the high-frequency residual energy ratio is less than the residual suppression threshold, the high-frequency aliasing suppression is qualified, otherwise, recalculate the dominant frequency.

[0013] A computer vision-based image display system, comprising: The data acquisition module collects the motion video stream of the target object, generates the original image frame sequence, synchronously records the motion parameters, performs semantic segmentation on the image frame sequence, locates the target object area and extracts the effective analysis area, performs spectrum analysis on the effective analysis area, and detects the dominant frequency and its harmonic energy; The adaptive filtering module locates the dominant frequency based on the linear velocity sequence in the motion parameters, constructs a narrowband filter to extract the fundamental component, determines the synchronous rotation state based on the spectral coherence between the angular velocity and the dominant frequency, generates an adaptive blur kernel and a short-range blur kernel, and convolves and superimposes them to form a composite blur kernel. The image processing module performs wavelet decomposition on the effective analysis area of ​​the original image frame, decomposing it into a low-frequency sub-band and multiple high-frequency detail sub-bands. The low-frequency sub-bands are convolved with a composite blur kernel, and the corresponding short-range blur kernel is selected based on the direction. Local convolution is performed on each high-frequency detail sub-band. The sub-bands are gradually fused through inverse wavelet transform to generate a reconstructed image. The phase information of the reconstructed image is extracted and aligned with the phase of the original image to generate a corrected blurred image. The image output module performs weighted fusion of the effective analysis area of ​​the corrected blurred image and the background area of ​​the original image, verifies whether the high-frequency residual energy ratio meets the preset residual suppression threshold, and renders and outputs the verified image sequence to the display device.

[0014] A storage medium stores one or more computer instructions, wherein the one or more computer instructions are used to implement the above-mentioned computer vision-based image display method.

[0015] (3) Beneficial effects The present invention provides a computer vision-based image display method, system, and storage medium, which have the following beneficial effects: (1) By performing spectrum analysis on the effective analysis area to detect the dominant frequency and its harmonic energy, the motion characteristics of the target object can be accurately obtained, providing key data support for the subsequent generation of adaptive blur kernel and phase correction, effectively reducing the image "ghosting" problem caused by ignoring the inherent frequency characteristics of the object, thereby significantly improving the quality of image display and visual perception.

[0016] (2) By accurately extracting the fundamental component and determining the synchronous rotation state, an adaptive blur kernel and a short-range blur kernel are generated that are highly matched with the motion characteristics of the target object, effectively simulating the real motion blur effect and significantly reducing the frequency domain aliasing phenomenon, thereby greatly improving the clarity and realism of the image display and optimizing the visual perception.

[0017] (3) By processing images using wavelet decomposition and composite blur kernels, combined with phase alignment technology, the motion blurred image is accurately reconstructed, effectively reducing image distortion caused by frequency domain aliasing, significantly improving image detail retention and overall visual effects, making the displayed image more natural and clear, and enhancing the viewing experience.

[0018] (4) By weighted fusion of the corrected blurred image and the original background and verifying the high-frequency residual energy ratio, the frequency domain aliasing phenomenon is effectively suppressed, ensuring that the high-frequency details of the image are reasonably preserved, thereby significantly improving the overall quality and visual coherence of the image display, making the final output image more realistic and natural. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a schematic diagram of the steps of the image display method based on computer vision of the present invention; Figure 2 This is a schematic diagram of the structure of the image display system based on computer vision of the present invention. DETAILED DESCRIPTION

[0020] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0021] See also Figure 1 The present invention provides an image display method based on computer vision, comprising the following steps: Step 1: Capture the motion video stream of the target object, generate the original image frame sequence, synchronously record the motion parameters, perform semantic segmentation on the image frame sequence, locate the target object area and extract the effective analysis area, perform spectrum analysis on the effective analysis area, and detect the dominant frequency and its harmonic energy; The step 1 includes the following contents: Step 101: Use an industrial area array camera to capture the target object's motion video stream and generate a sequence of original image frames. ,in, Frame, synchronously record the target object's motion parameters, including linear velocity, angular velocity and motion direction. The linear velocity is measured in real time by the encoder (unit: pixel / frame), the angular velocity is obtained by the speed sensor (unit: radian / frame), and the motion direction is calibrated by the initial position; When the motion parameters are unknown, the optical flow method (such as the Lucas-Kanade optical flow algorithm) is used to track the edge feature points of the target object, calculate the displacement of the feature points between adjacent frames, and obtain the linear velocity: ,in, and The feature points are x Axis and y Displacement on the axis, is the time interval between adjacent frames; the rate of change of displacement direction Calculate angular velocity: ; It should be noted that in order to improve the accuracy of feature point tracking, the Shi-Tomasi corner detector is used to extract several high-confidence feature points on the edge of the target object, and the pyramid LK optical flow is used to achieve sub-pixel accuracy tracking, with a median error of ≤0.15 pixels; Step 102: For the acquired image frame sequence, locate the target object area through the U-Net semantic segmentation network and output the mask , and intercept a fixed percentage (e.g. 75%) of the area in the center of the target object as the effective analysis area, set the sliding step size (e.g. 256 pixels), and perform a two-dimensional fast Fourier transform (FFT) on the effective analysis area to cover the entire effective analysis area; In order to reduce spectrum leakage, Hanning window is used to perform windowing on the effective analysis area. In the spectrum preprocessing stage, logarithmic transformation is used to The low energy components are enhanced and the spectral noise is smoothed using Gaussian filtering, where It represents the amplitude (modulus) of the complex spectrum obtained after Fourier transform of the image frame I. The absolute value is taken to convert the complex spectrum into a non-negative real amplitude. Represents a small positive constant, such as , to prevent the input of the logarithm operation from being zero or negative; It should be noted that the mask is a binary matrix (or grayscale image) of the same size as the input image, which is used to mark the areas in the image that need to be paid attention to or blocked. Its mathematical expression is: ,The precise outline of the target object can be extracted through the mask, avoiding the ,interference of the background on the frequency domain analysis; Step 103: Set the energy threshold: ,in, μ is the spectrum mean, σ is the standard deviation. When detecting energy peaks in the spectrum, a sliding detection window (such as an 8-neighborhood) uses a non-maximum suppression algorithm to detect local peaks (amplitudes), and retains frequency points with amplitudes greater than the energy threshold as candidate frequency points. Step 104: Detect the candidate frequency points , calculate its first three harmonics: ,in, i is the candidate frequency point number, k is the order, k =1,2,3, in the spectrum As the center, intercept the dynamic frequency band: ,in, is the Nyquist frequency; calculate the energy of each harmonic: ,in, f Indicates frequency, select total energy The largest frequency is taken as the dominant frequency; When using, combine the contents of steps 101 to 104: By performing spectral analysis on the effective analysis area to detect the dominant frequency and its harmonic energy, the motion characteristics of the target object can be accurately obtained, providing key data support for the subsequent generation of the adaptive blur kernel and phase correction, effectively reducing the image "ghosting" problem caused by ignoring the inherent frequency characteristics of the object, thereby significantly improving the quality of image display and visual perception.

[0022] Step 2: Locate the dominant frequency based on the linear velocity sequence in the motion parameters, construct a narrowband filter to extract the fundamental component, determine the synchronous rotation state based on the spectral coherence between the angular velocity and the dominant frequency, generate an adaptive blur kernel and a short-range blur kernel, and convolute and superimpose them to form a composite blur kernel; The second step includes the following contents: Step 201: Perform discrete Fourier transform (DFT) on the acquired linear velocity sequence to generate the original spectrum, locate the position of the dominant frequency in the original spectrum, and use it as the center frequency to construct a bandwidth of A narrowband filter is used to extract the fundamental component: ,in, IFFT () represents inverse fast Fourier transform, Represents the original spectrum obtained after discrete Fourier transform of the linear velocity sequence, rect () is a rectangular window function, which is used to select a specific frequency range in the frequency domain. represents the dominant frequency, Indicates bandwidth, f Represents frequency; the high-frequency residual component is calculated by the difference between the original spectrum and the fundamental spectrum: ,in, is the fundamental spectrum, ; Step 202: Convert the angular velocity to the circumferential frequency: ,in, ω Denote the angular velocity and calculate the spectral coherence coefficient between the circular frequency and the dominant frequency: ,in, represents the cross-power spectral density between the dominant frequency and the circular frequency, represents the dominant frequency autopower spectral density, Represents the circumferential frequency autopower spectrum density; if the spectral coherence coefficient is greater than the coherence threshold (such as 0.8), it is judged as synchronous rotation, otherwise it is marked as asynchronous disturbance; Step 203: Construct an elliptical bandpass filter with the dominant frequency as the constraint , including is the center frequency and the bandwidth is 0.1 , and set the filter order (such as 8th order). If there is asynchronous disturbance, the bandwidth is extended to 0.2 ; Initialize the rectangular blur kernel, the length is set to: ,in, is the time interval between adjacent frames, α represents the safety factor, , use a two-dimensional Gaussian function to generate kernel values: ,in, x 、 y are the horizontal and vertical coordinates of the pixel, is the attenuation coefficient, ; After performing a fast Fourier transform on the rectangular blur kernel, it is point-wise multiplied with the elliptical bandpass filter: , Indicates point-by-point multiplication, generating an adaptive blur kernel through inverse fast Fourier transform ; Step 204: Calculate the pixel displacement of the high-frequency residual component: , mapping the pixel displacement to length to generate a short-range blur kernel The short-range blur kernel generation method is the same as the rectangular blur kernel generation in step 203. The adaptive blur kernel and the short-range blur kernel are convolved and superimposed to generate a composite blur kernel: , where * represents the convolution operation; When using, combine the contents of steps 201 to 204: By accurately extracting the fundamental component and determining the synchronous rotation state, we generate adaptive blur kernels and short-range blur kernels that highly match the motion characteristics of the target object, effectively simulating the real motion blur effect and significantly reducing the frequency domain aliasing phenomenon, thereby greatly improving the clarity and realism of the image display and optimizing the visual experience.

[0023] Step 3: Perform wavelet decomposition on the effective analysis area of ​​the original image frame, decomposing it into a low-frequency sub-band and multiple high-frequency detail sub-bands. Convolve the low-frequency sub-bands with a composite blur kernel, select the corresponding short-range blur kernel based on the direction, and perform local convolution on each high-frequency detail sub-band. Then, use the inverse wavelet transform to fuse the sub-bands step by step to generate a reconstructed image. The phase information of the reconstructed image is extracted and aligned with the phase of the original image to generate a corrected blurred image. The step three includes the following contents: Step 301: Perform wavelet decomposition (such as Daubechies-4 wavelet basis) on the effective analysis area of ​​the original image frame, decomposing it into a low-frequency sub-band and several high-frequency detail sub-bands. Perform convolution operation on the low-frequency sub-band using a composite blur kernel, and use fast Fourier transform to accelerate the calculation and generate the blurred value: ,in, represents the low-frequency subband; Step 302: For each high-frequency detail subband, select the corresponding short-range blur kernel by direction and perform local convolution: ,in, Indicates the high-frequency detail subband after blurring at the pixel position ( x, y ), Indicates that the high-frequency detail subband is at the offset position ( x + m, y + n ), the pixel value at position ( m, n ), r Represents the radius of the short-range blur kernel (corresponding to the kernel size of ); Step 303: Use inverse discrete wavelet transform to fuse low-frequency sub-bands and several high-frequency detail sub-bands step by step. For example, if three-level discrete wavelet transform is used in wavelet decomposition, three-level inverse discrete wavelet transform is used to fuse sub-bands step by step: Level 1 reconstruction: The level 3 low-frequency subband is fused with the level 3 high-frequency subband to generate a level 2 low-frequency approximation; Level 2 reconstruction: The level 2 low-frequency approximation is fused with the level 2 high-frequency subband to generate the level 1 low-frequency approximation; Level 3 reconstruction: Fusion of the level 3 low-frequency approximation with the level 1 high-frequency subband to generate a full-resolution reconstructed image ; Step 304: Reconstruct the image Perform a fast Fourier transform, extract the complex spectrum, and calculate the phase from the real and imaginary parts of the complex spectrum: ,in, represents the complex spectrum, ( u, v ) represents the spectrum coordinates, Im () represents the imaginary part, Re () represents the imaginary part; Step 305: Calculate the difference between the original image frame spectrum phase and the blurred image phase to generate a phase correction matrix: ,in, Represents the spectral phase of the original image frame. Phase alignment is achieved through complex spectrum rotation to obtain the corrected spectrum: , where j represents the imaginary unit, and the inverse fast Fourier transform is performed on the phase-corrected spectrum to generate a phase-aligned blurred image; When using, combine the contents of steps 301 to 305: By processing images through wavelet decomposition and composite blur kernels, combined with phase alignment technology, motion-blurred images are accurately reconstructed, effectively reducing image distortion caused by frequency domain aliasing, significantly improving image detail retention and overall visual effects, making the displayed images more natural and clear, and enhancing the viewing experience.

[0024] Step 4: Perform weighted fusion of the effective analysis area of ​​the corrected blurred image and the background area of ​​the original image to verify whether the high-frequency residual energy ratio meets the preset residual suppression threshold, and render the verified image sequence and output it to the display device.

[0025] The fourth step includes the following contents: Step 401: Perform weighted fusion on the effective analysis area of ​​the phase-aligned blurred image (i.e., the central 75% area of ​​the target object intercepted in step 102) and the background area of ​​the original image frame: ,in, is the phase-aligned blurred image, is the original image frame, β is the fusion weight matrix, within the effective analysis area , in the background area (non-target object area) , in the boundary transition area (the part of the target object that is not the effective analysis area, that is, the remaining 25% area), a cosine function gradient is used ,in, d represents the shortest distance, that is, the shortest distance between each pixel in the boundary transition area and the effective analysis area, and D represents the maximum width of the boundary transition area; Step 402: Perform a two-dimensional fast Fourier transform on the weighted fused image and set the high frequency cutoff frequency. When the frequency is greater than the high-frequency cutoff frequency, it is marked as a high-frequency band, and the high-frequency residual energy ratio is calculated: high-frequency residual energy ratio = high-frequency band energy / full-band energy. If the high-frequency residual energy ratio is less than the residual suppression threshold (such as 5%), the high-frequency aliasing suppression is qualified. Otherwise, return to step 1 to recalculate the dominant frequency. Step 403: sort the verified images by video stream timestamp, output them to the display device through a graphics rendering interface (such as OpenGL / Vulkan), and store them as video files (encoding format H.264, frame rate consistent with the input video stream).

[0026] When using, combine the contents of step 401 to step 403: By weighted fusion of the corrected blurred image and the original background and verifying the high-frequency residual energy ratio, the frequency domain aliasing phenomenon is effectively suppressed, ensuring that the high-frequency details of the image are reasonably preserved, thereby significantly improving the overall quality and visual coherence of the image display, making the final output image more realistic and natural.

[0027] See also Figure 2 The present invention also provides an image display system based on computer vision, comprising: a data acquisition module, an adaptive filtering module, an image processing module and an image output module, wherein: The data acquisition module collects the motion video stream of the target object, generates the original image frame sequence, synchronously records the motion parameters, performs semantic segmentation on the image frame sequence, locates the target object area and extracts the effective analysis area, performs spectrum analysis on the effective analysis area, and detects the dominant frequency and its harmonic energy; The adaptive filtering module locates the dominant frequency based on the linear velocity sequence in the motion parameters, constructs a narrowband filter to extract the fundamental component, determines the synchronous rotation state based on the spectral coherence between the angular velocity and the dominant frequency, generates an adaptive blur kernel and a short-range blur kernel, and convolves and superimposes them to form a composite blur kernel. The image processing module performs wavelet decomposition on the effective analysis area of ​​the original image frame, decomposing it into a low-frequency sub-band and multiple high-frequency detail sub-bands. The low-frequency sub-bands are convolved with a composite blur kernel, and the corresponding short-range blur kernel is selected based on the direction. Local convolution is performed on each high-frequency detail sub-band. The sub-bands are gradually fused through inverse wavelet transform to generate a reconstructed image. The phase information of the reconstructed image is extracted and aligned with the phase of the original image to generate a corrected blurred image. The image output module performs weighted fusion of the effective analysis area of ​​the corrected blurred image and the background area of ​​the original image, verifies whether the high-frequency residual energy ratio meets the preset residual suppression threshold, and renders and outputs the verified image sequence to the display device.

[0028] The present invention also provides a storage medium, which stores one or more computer instructions, and the one or more computer instructions are used to implement the computer vision-based image display method provided by the present invention.

[0029] In the application, the several formulas involved are all calculated by taking their numerical values ​​after removing the dimensions, and the formula is a formula obtained by collecting a large amount of data and performing software simulation to obtain the latest real situation. The coefficients in the formula are set by technical personnel in this field according to actual conditions.

[0030] The above embodiments can be implemented in whole or in part through software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. Those skilled in the art will appreciate that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented using a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution.

[0031] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, and may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment as needed.

[0032] The above is only a specific implementation method of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the scope of protection of the present application.

Claims

1. A computer vision-based image display method, characterized by: include: Capture the target object's motion video stream, generate an original image frame sequence, synchronously record motion parameters, perform semantic segmentation on the image frame sequence, locate the target object area and extract the effective analysis area, perform spectrum analysis on the effective analysis area, and detect the dominant frequency and its harmonic energy; The dominant frequency is located according to the linear velocity sequence in the motion parameters, and a narrowband filter is constructed to extract the fundamental component. The synchronous rotation state is determined based on the spectral coherence between the angular velocity and the dominant frequency. An adaptive blur kernel and a short-range blur kernel are generated and convoluted to form a composite blur kernel. Perform wavelet decomposition on the effective analysis area of ​​the original image frame, decomposing it into a low-frequency sub-band and multiple high-frequency detail sub-bands. Use a composite blur kernel to convolve the low-frequency sub-band. Select the corresponding short-range blur kernel based on the direction, and perform local convolution on each high-frequency detail sub-band. Use inverse wavelet transform to fuse the sub-bands step by step to generate a reconstructed image. Extract the phase information of the reconstructed image and align it with the phase of the original image to generate a corrected blurred image. The effective analysis area of ​​the corrected blurred image is weightedly fused with the background area of ​​the original image to verify whether the high-frequency residual energy ratio meets the preset residual suppression threshold. The verified image sequence is rendered and output to the display device.

2. The computer vision-based image display method according to claim 1, characterized in that: The target object area is located through the semantic segmentation network, a mask is generated, and a fixed percentage of the area in the center of the target object is intercepted based on the mask as the effective analysis area. A two-dimensional fast Fourier transform is performed on the effective analysis area, and an energy threshold is set in the spectrum. The local peak is detected by the non-maximum suppression algorithm, and the frequency points with amplitudes greater than the energy threshold are retained as candidate frequency points. The energy of the first three harmonics of the candidate frequency points is calculated, and the frequency with the largest total energy is selected as the dominant frequency. .

3. The computer vision-based image display method according to claim 1, characterized in that: Perform discrete Fourier transform on the linear velocity sequence to generate the original spectrum, locate the position of the dominant frequency in the original spectrum, and use it as the center frequency to construct a bandwidth of The fundamental spectrum and fundamental component are extracted by using a narrowband filter, and the high-frequency residual component is calculated by the difference between the original spectrum and the fundamental spectrum; The angular velocity is converted into circumferential frequency, and the spectral coherence coefficient between the circumferential frequency and the dominant frequency is calculated. If the spectral coherence coefficient is greater than the coherence threshold, it is determined to be synchronous rotation; otherwise, it is marked as asynchronous disturbance.

4. The computer vision-based image display method according to claim 3, characterized in that: Construct an elliptical bandpass filter constrained by the dominant frequency, including is the center frequency and the bandwidth is 0.1 , and set the filter order. If there is asynchronous disturbance, the bandwidth is extended to 0.2 ; Initialize the rectangular blur kernel with the length set to: ,in, is the time interval between adjacent frames, α is the safety factor, , is the fundamental component, and a two-dimensional Gaussian function is used to generate the kernel value: ,in, x 、 y are the horizontal and vertical coordinates of the pixel, is the attenuation coefficient, After performing fast Fourier transform on the rectangular blur kernel, it is multiplied point by point with the elliptical bandpass filter, and the adaptive blur kernel is generated by inverse fast Fourier transform. .

5. The computer vision-based image display method according to claim 4, characterized in that: Calculate the pixel displacement for the high-frequency residual component: ,in, For the high-frequency residual component, the pixel displacement is mapped to length to generate a short-range blur kernel The generation method is the same as that of the rectangular blur kernel. The adaptive blur kernel and the short-range blur kernel are convolved and superimposed to generate a composite blur kernel: 。 6. The computer vision-based image display method according to claim 1, characterized in that: Perform wavelet decomposition on the effective analysis area of ​​the original image frame and decompose it into low-frequency sub-bands and several high-frequency detail sub-bands. Use a composite blur kernel to perform convolution operation on the low-frequency sub-band; for each high-frequency detail sub-band, select the corresponding short-range blur kernel according to the direction and perform local convolution: ; in, The high-frequency detail subband after blurring is at the pixel position ( x,y ), The high frequency detail subband is at the offset position ( x+m,y+n ), is the short-range blur kernel at position ( m,n ), r is the radius of the short-range blur kernel.

7. The computer vision-based image display method according to claim 6, characterized in that: The inverse discrete wavelet transform is used to step-by-step fuse low-frequency sub-bands and several high-frequency detail sub-bands to generate a reconstructed image. The reconstructed image is subjected to a fast Fourier transform, and the complex spectrum is extracted. The phase is calculated by the real and imaginary parts of the complex spectrum. The difference between the phase of the original image frame spectrum and the phase of the blurred image is calculated. The phase is aligned by rotating the complex spectrum to obtain the corrected spectrum. The inverse fast Fourier transform is performed on the phase-corrected spectrum to generate a phase-aligned blurred image.

8. The computer vision-based image display method according to claim 1, characterized in that: The effective analysis area of ​​the phase-aligned blurred image is weightedly fused with the background area of ​​the original image frame. ,in, is the phase-aligned blurred image, is the original image frame, β is the fusion weight matrix, within the effective analysis area , in the background area , in the boundary transition area ,in, d represents the shortest distance, and D represents the maximum width of the boundary transition area; Perform two-dimensional fast Fourier transform on the weighted fused image, set the high-frequency cutoff frequency, and when the frequency is greater than the high-frequency cutoff frequency, mark it as the high-frequency band, and calculate the high-frequency residual energy ratio: high-frequency residual energy ratio = high-frequency band energy / full-band energy. If the high-frequency residual energy ratio is less than the residual suppression threshold, the high-frequency aliasing suppression is qualified, otherwise, recalculate the dominant frequency.

9. A computer vision-based image display system, used to implement the method according to any one of claims 1 to 8, characterized in that: include: The data acquisition module collects the motion video stream of the target object, generates the original image frame sequence, synchronously records the motion parameters, performs semantic segmentation on the image frame sequence, locates the target object area and extracts the effective analysis area, performs spectrum analysis on the effective analysis area, and detects the dominant frequency and its harmonic energy; The adaptive filtering module locates the dominant frequency based on the linear velocity sequence in the motion parameters, constructs a narrowband filter to extract the fundamental component, determines the synchronous rotation state based on the spectral coherence between the angular velocity and the dominant frequency, generates an adaptive blur kernel and a short-range blur kernel, and convolves and superimposes them to form a composite blur kernel. The image processing module performs wavelet decomposition on the effective analysis area of ​​the original image frame, decomposing it into a low-frequency sub-band and multiple high-frequency detail sub-bands. The low-frequency sub-bands are convolved with a composite blur kernel, and the corresponding short-range blur kernel is selected based on the direction. Local convolution is performed on each high-frequency detail sub-band. The sub-bands are gradually fused through inverse wavelet transform to generate a reconstructed image. The phase information of the reconstructed image is extracted and aligned with the phase of the original image to generate a corrected blurred image. The image output module performs weighted fusion of the effective analysis area of ​​the corrected blurred image and the background area of ​​the original image, verifies whether the high-frequency residual energy ratio meets the preset residual suppression threshold, and renders and outputs the verified image sequence to the display device.

10. A storage medium, characterized in that: The storage medium stores one or more computer instructions, and the one or more computer instructions are used to implement the method according to any one of claims 1 to 8.

Images