A target detection method and system based on frequency domain enhancement

Abstract

The present invention provides a target detection method and system based on frequency domain enhancement, which belongs to the field of computer vision technology. The method includes: calculating the similarity of the spatial features of foreground images of adjacent frames in the video to be detected as spatial similarity; converting the spatial features of the foreground images of adjacent frames into foreground image frequency domain features, and calculating the similarity of the foreground image frequency domain features of adjacent frames as frequency similarity; using a pre-set medium-high frequency filter to filter the second foreground image frequency domain features, and converting the filtered foreground image frequency domain features into a third foreground image spatial features; determining the final foreground image spatial features of the current frame based on the spatial similarity, frequency similarity, the first foreground image spatial features, and the third foreground image spatial features to enhance the detection of foreground targets. The present invention better establishes the semantic connection between image pixel features and targets, realizes feature enhancement, and improves detection accuracy.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a target detection method and system based on frequency domain enhancement. Background Art

[0002] Colonoscopy is the gold standard for diagnosing colorectal cancer and precancerous lesions and is the cornerstone of CRC prevention. Diagnosing and treating precancerous polyps through colonoscopy and removing them before they transform into cancer can significantly reduce the incidence and mortality of CRC and improve patient survival. Although colonoscopy provides information on the appearance and location of polyps, it requires expensive labor resources and has a high misdiagnosis rate of up to 27% for the following reasons: First, colonoscopy screening is highly professional, and the accuracy of diagnosis depends heavily on the doctor's experience. Manual screening consumes a lot of time and costs. Second, polyps show significant differences in color, size, and shape. At the same time, polyps look similar to surrounding tissues, have unclear boundaries, and are highly hidden.

[0003] Therefore, it is necessary to develop an automated and accurate colorectal polyp detection system to reduce the burden of manual screening and the occurrence of misdiagnosis. Currently, traditional methods and deep learning-based colonoscopy target detection methods are designed only at the image level. Few studies have focused on the advantages of frequency domain information in enhancing image features for better target detection. Summary of the Invention

[0004] The present invention provides a target detection method and system based on frequency domain enhancement, which is used to solve the defect that most of the existing technologies ignore the potential of frequency domain analysis in feature enhancement. It can divide the spectral information obtained from image features into different frequency ranges, and specifically utilize the frequency domain features of different frequency bands to enhance the foreground and background of the image respectively.

[0005] In a first aspect, the present invention provides a method for object detection based on frequency domain enhancement, comprising: calculating the similarity of foreground image spatial features of adjacent frames in a video to be detected as spatial similarity; wherein the adjacent frames include a previous frame and a current frame, the foreground image spatial features of the previous frame are first foreground image spatial features, and the foreground image spatial features of the current frame are second foreground image spatial features;

[0006] Converting the spatial features of the foreground images of adjacent frames into frequency domain features of the foreground images, and calculating the similarity of the frequency domain features of the foreground images of adjacent frames as the frequency similarity; wherein the frequency domain features of the foreground image of the previous frame are the first frequency domain features of the foreground image, and the frequency domain features of the foreground image of the current frame are the second frequency domain features of the foreground image;

[0007] Using a preset medium-high frequency filter to filter the frequency domain features of the second foreground image, and converting the filtered frequency domain features of the foreground image into spatial features of the third foreground image;

[0008] The final foreground image spatial feature of the current frame is determined according to the spatial similarity, the frequency similarity, the first foreground image spatial feature, and the third foreground image spatial feature to enhance the detection of the foreground object.

[0009] According to a target detection method based on frequency domain enhancement provided by the present invention, it also includes: using a pre-set low-frequency filter to filter the second background image frequency domain features, and converting the filtered background image frequency domain features into a third background image spatial features; wherein the second background image frequency domain features are the background image frequency domain features of the current frame; based on spatial similarity, frequency similarity, the first background image spatial features and the third background image spatial features, the final background image spatial features of the current frame are determined to weaken background interference; wherein the first background image spatial features are the background image spatial features of the previous frame.

[0010] According to a target detection method based on frequency domain enhancement provided by the present invention, the final foreground image spatial features of the current frame are determined according to spatial similarity, frequency similarity, first foreground image spatial features and third foreground image spatial features, including: taking a weighted average of the spatial similarity and the frequency similarity to determine the comprehensive similarity; after weighting the first foreground image spatial features using the comprehensive similarity as a weighted score, enhancing the third foreground image spatial features to determine the final foreground image spatial features of the current frame.

[0011] According to a target detection method based on frequency domain enhancement provided by the present invention, the final background image spatial features of the current frame are determined according to spatial similarity, frequency similarity, the first background image spatial features and the third background image spatial features, including: taking a weighted average of the spatial similarity and the frequency similarity to determine the comprehensive similarity; after weighting the first background image spatial features with the comprehensive similarity as a weighted score, enhancing the third background image spatial features to determine the final background image spatial features of the current frame.

[0012] According to the target detection method based on frequency domain enhancement provided by the present invention, the spatial similarity and the frequency similarity are characterized by using pre-similarity.

[0013] According to a target detection method based on frequency domain enhancement provided by the present invention, it also includes: determining the enhanced image features of the current frame based on the third foreground image spatial features and the third background image spatial features; and obtaining the enhanced image of the current frame based on the enhanced image features of the current frame.

[0014] According to a target detection method based on frequency domain enhancement provided by the present invention, the video to be detected is a colonoscopy video, and the target to be detected is an intestinal polyp.

[0015] In a second aspect, the present invention further provides a target detection system based on frequency domain enhancement, comprising:

[0016] a first processing unit, configured to calculate a similarity of spatial features of foreground images of adjacent frames in the video to be detected as spatial similarity; wherein the adjacent frames include a previous frame and a current frame, the spatial features of the foreground image of the previous frame are first spatial features of the foreground image, and the spatial features of the foreground image of the current frame are second spatial features of the foreground image;

[0017] a second processing unit, configured to convert spatial features of foreground images of adjacent frames into frequency domain features of foreground images, and calculate similarities of the frequency domain features of the foreground images of adjacent frames as frequency similarities; wherein the frequency domain features of the foreground image of the previous frame are the first frequency domain features of the foreground image, and the frequency domain features of the foreground image of the current frame are the second frequency domain features of the foreground image;

[0018] a third processing unit, configured to filter the frequency domain features of the second foreground image using a preset medium-high frequency filter, and convert the filtered frequency domain features of the foreground image into spatial features of a third foreground image;

[0019] The fourth processing unit is configured to determine a final foreground image spatial feature of the current frame according to the spatial similarity, the frequency similarity, the first foreground image spatial feature, and the third foreground image spatial feature, so as to enhance the detection of the foreground object.

[0020] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the target detection method based on frequency domain enhancement as described above are implemented.

[0021] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the frequency domain enhancement-based target detection methods described above.

[0022] The target detection method and system based on frequency domain enhancement provided by the present invention have the following beneficial effects compared with the prior art:

[0023] (1) The present invention makes full use of the frequency domain information within the frame. By performing targeted analysis on the extracted frequency domain information and constraining the image features based on the frequency information, the semantic connection between the image pixel features and the target is better established, feature enhancement is achieved, and detection accuracy is improved.

[0024] (2) In the target detection scenario, the present invention starts from the perspective of segmented analysis of different frequency segments of the image feature spectrum, considers the differences in the areas of interest using information from different frequency segments, and proposes a complete set of frequency extraction, analysis and enhancement methods.

[0025] (3) The foreground similarity analysis designed by the present invention integrates frequency similarity analysis. Compared with the method that only considers spatial feature similarity, the foreground similarity analysis that integrates frequency information similarity and spatial similarity for weighted analysis can better improve the effectiveness of adjacent frame references in dynamic scenes and complex backgrounds.

[0026] (4) The present invention uses high-frequency, medium-frequency and low-frequency information to enhance image features based on the foreground and background information of the image, highlighting foreground details, weakening background interference, and improving the detection effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0028] Figure 1 Schematic diagram of a framework for enhancing colonoscopy target detection using spectral information designed in an embodiment of the present invention;

[0029] Figure 2 1 is a schematic diagram of a process for enhancing colonoscopy target detection using spectral information designed in an embodiment of the present invention;

[0030] Figure 3 Schematic diagram of a frequency enhancement module verifying the focus of image information on high-frequency, mid-frequency, and low-frequency components in an embodiment of the present invention;

[0031] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0032] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0033] It should be noted that, in the description of the embodiments of the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include a ..." do not exclude the presence of other identical elements in the process, method, article or device comprising the elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0034] The terms "first," "second," and the like in this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that such terms are interchangeable where appropriate, so that embodiments of this application can be implemented in an order other than that illustrated or described herein. Furthermore, the terms "first," "second," and the like generally distinguish objects of a class and do not limit the number of objects; for example, the first object can be one or more. Furthermore, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates an "or" relationship between the connected objects.

[0035] First, the overall process of the present invention is described:

[0036] Step 1: Calculate the similarity of the spatial features of the foreground images of adjacent frames in the video to be detected as the spatial similarity.

[0037] The video to be detected may be a colonoscopy video, the adjacent frames include a previous frame and a current frame, the foreground image spatial feature of the previous frame is the first foreground image spatial feature, and the foreground image spatial feature of the current frame is the second foreground image spatial feature.

[0038] The foreground image spatial features are extracted from the foreground image of the image frame using a convolutional neural network; among them, the spatial similarity is used to indicate the degree of similarity between the foreground objects in the spatial domain between the two frames.

[0039] Step 2: The spatial features of the foreground images of adjacent frames are converted into frequency domain features of the foreground images, and the similarity of the frequency domain features of the foreground images of adjacent frames is calculated as the frequency similarity.

[0040] The image frequency domain features can be obtained by converting the image space features based on Fourier transform, wherein the foreground image frequency domain features of the previous frame are the first foreground image frequency domain features, and the foreground image frequency domain features of the current frame are the second foreground image frequency domain features.

[0041] Frequency similarity is used to reflect the similarity of foreground objects between two frames in the frequency domain.

[0042] Step 3: Filter the frequency domain features of the second foreground image using a preset medium-high frequency filter, and convert the filtered frequency domain features of the foreground image into spatial features of the third foreground image.

[0043] The medium-high frequency filter refers to a medium frequency filter and a high frequency filter. The present invention can realize the conversion of spatial features and frequency domain features through Fourier transform and inverse Fourier transform.

[0044] Step 4: Determine the final foreground image spatial features of the current frame based on the spatial similarity, the frequency similarity, the first foreground image spatial features, and the third foreground image spatial features to enhance the detection of foreground objects.

[0045] Optionally, step 4 includes:

[0046] Step 401: Perform weighted averaging on the spatial similarity and the frequency similarity to determine the comprehensive similarity.

[0047] Among them, the weights can be adjusted according to the specific application scenarios or experimental results to ensure that the comprehensive similarity can accurately reflect the changes of the foreground targets in the spatial domain and frequency domain.

[0048] Step 402: After weighting the first foreground image spatial feature using the comprehensive similarity as a weighted score, the third foreground image spatial feature is enhanced to determine the final foreground image spatial feature of the current frame.

[0049] Based on the content of the above embodiment, as an optional embodiment, the target detection method based on frequency domain enhancement provided by the present invention further includes:

[0050] Step 5: Filter the frequency domain features of the second background image using a preset low-frequency filter, and convert the filtered frequency domain features of the background image into third background image spatial features.

[0051] The second background image frequency domain feature is the background image frequency domain feature of the current frame. The background image frequency domain feature is obtained by Fourier transforming the background image spatial feature, which will not be described in detail here. The background image spatial feature is extracted from the background image of the image frame using a convolutional neural network.

[0052] Step 6: Determine the final background image spatial feature of the current frame based on the spatial similarity, the frequency similarity, the first background image spatial feature, and the third background image spatial feature to weaken background interference.

[0053] The first background image spatial feature is the background image spatial feature of the previous frame.

[0054] Optionally, step 6 includes:

[0055] Step 601: performing weighted averaging of spatial similarity and frequency similarity to determine comprehensive similarity;

[0056] Step 602: After weighting the first background image spatial feature using the comprehensive similarity as a weighted score, the third background image spatial feature is enhanced to determine the final background image spatial feature of the current frame.

[0057] Based on the content of the above embodiment, as an optional embodiment, the target detection method based on frequency domain enhancement provided by the present invention further includes:

[0058] Step 7: Determine the enhanced image features of the current frame based on the third foreground image spatial features and the third background image spatial features;

[0059] The third foreground image spatial feature and the third background image spatial feature are both foreground image spatial features after frequency information enhancement.

[0060] Step 8: Obtain the enhanced image of the current frame based on the enhanced image features of the current frame.

[0061] Optionally, in the above embodiment, both the spatial similarity and the frequency similarity may be represented by cosine similarity.

[0062] The following example takes the colonoscopy video as the video to be tested and the intestinal polyps as the detection target. Figure 1 and Figure 2 , the technical solution of the present invention is further analyzed and explained from three aspects (modules) of frequency extraction, analysis and enhancement.

[0063] (1) Frequency domain information extraction (can be designed as a functional module, namely, a frequency domain information extraction module):

[0064] For continuous colonoscopy video frames, the image information is first extracted through a convolutional neural network to obtain spatial feature information. For the spatial feature map, the spatial information is converted into frequency information using a Fourier transform (FFT). Let the Fourier transform of the image's spatial feature f(x, y) be F(u, v), where x and y represent the horizontal and vertical coordinates in the spatial domain, respectively, and u and v represent the horizontal and vertical coordinates in the frequency domain, respectively.

[0065] The Fourier transform output includes real and imaginary parts, representing the distribution of the feature map at different frequencies. By observing the frequency distribution of the spectrogram, we can clearly distinguish between low-frequency, high-frequency, and medium-frequency components. Low-frequency regions are often located in the center of the spectrogram, while high-frequency regions are located at the edges. The contributions of different frequency components of an image to specific tasks vary. Low-frequency information provides the overall outline of the image, while high-frequency components represent edges and details.

[0066] In the frequency domain, low frequencies mainly contain the overall outline or general semantic information of the image, while high frequencies contain edge and detail information. The main target polyp images in colonoscopy videos often contrast sharply with low-resolution, nearly flat textures, and the polyps appear raised in the image, different from the colonoscopy mucosa. However, polyps and intestinal mucosa are highly similar in appearance. When analyzed from the image and spatial features, they are very hidden, so a new perspective is needed to provide additional auxiliary detection based on spatial features. Spectrum-based methods, which focus on targets in different spectral bands, are a good way to distinguish foreground and background.

[0067] We can observe that after the convolution stage, the mid-frequency subband between the high and low subbands accounts for an increasingly larger proportion of the entire spectrum. Therefore, when analyzing the frequency information of image features after convolution, in addition to considering the high-frequency components that focus on details and the low-frequency components that provide general semantic information, the mid-frequency subband, which contains rich global information about polyps, should also be considered.

[0068] Therefore, for the frequency domain features F(u,v) obtained by FFT, low-frequency filters, intermediate-frequency filters and high-frequency filters are designed respectively to separate the different frequency bands of the spectrum information.

[0069] The low-frequency filter uses a small-scale Gaussian filter to focus on the center of the spectrum, preserving primarily the low-frequency information of the image and suitable for capturing overall contours and smooth backgrounds. The high-frequency filter uses a larger-scale Gaussian filter to filter the center of the spectrum, retaining only the high-frequency information in the periphery to enhance edges and detail. The mid-frequency filter subtracts two Gaussian filters of different scales to preserve information in the middle of the spectrum. This helps capture local details and transition information, enhances the contrast between foreground and background, and enables the model to more clearly identify the complete structure of an object.

[0070] The design goal of the low-frequency filter is to preserve the center area of ​​the spectrum. The low-frequency filter is a Gaussian filter with a small radius. The implementation principle is as follows:

[0071]

[0072] Among them, the parameter σ low Used to control the range of low-frequency components.

[0073] The purpose of high-frequency filter design is to suppress the central area of ​​the spectrum (low-frequency part) in the frequency domain and retain the edge area of ​​the spectrum (high-frequency part). The basic principle formula is as follows:

[0074]

[0075] Among them, the parameter σ high Used to control the radius of the high-pass filter, that is, the frequency selection range of the filter.

[0076] The intermediate frequency filter is constructed by interpolating two Gaussian high-pass filters of different radii to preserve the information in the intermediate frequency region. The specific implementation principle is as follows:

[0077] H mid (u,v)=H high1 (u,v)-H hiigh2 (u,v), (3)

[0078] Filter H hiigh1 (u,v) and H hiigh2 The radii of (u,v) are σ hiigh1 and σ hiigh2 , and satisfy σ hiigh1 <σ hiigh2 .

[0079] (2) Frequency domain information analysis (can be designed as a functional module, i.e., frequency domain information analysis module):

[0080] Usually, since the video is continuous, the foreground targets of the images in two adjacent frames often have a high similarity. By using the similarity relationship between the foreground of the previous frame and the foreground of the current frame, the degree of feature enhancement of the current frame using the previous frame is controlled, and the foreground features of the current frame are enhanced, which can better help the image complete the detection of the foreground target in the current frame.

[0081] To mitigate the significant differences in spatial features when the scene changes rapidly or the foreground object shifts significantly, which can lead to large errors in foreground information interaction controlled solely by spatial similarity, spectral information can effectively maintain the relative stability of foreground frequency information when there is a lot of environmental interference such as lighting, noise, and motion. Introducing spectral information to assist in the interaction of foreground information between adjacent frames can effectively compensate for the limitations of spatial feature similarity reference.

[0082] The specific implementation is to first input the image frames of the video into the network frame by frame in the form of I[i], so as to maintain the order of the images well. For each specific image I[i], except for the first frame itself being selected for foreground interaction, the subsequent frames all interact with the foreground of its previous frame I[i-1].

[0083] In order to achieve only foreground information interaction, during the training process, the GT image frame is used to generate a mask to multiply the image to obtain the foreground and background of each frame; when there is no GT annotation, the prediction result of the previous frame is used as the GT frame of the current frame to obtain the foreground from the image.

[0084] For adjacent image frames I[i] and I[i-1], we first use the same frequency feature extraction module and the same FFT to map the spatial feature information to the frequency domain, thereby obtaining the frequency features of the foreground. We then calculate the spatial similarity and frequency similarity respectively.

[0085] For the extracted foreground spatial features (i.e., foreground image spatial features), the cosine similarity is calculated. The calculation formula of spatial cosine similarity is as follows:

[0086]

[0087] Among them, f cur , f prev Represents the foreground spatial features of the current frame and the previous frame respectively. The similarity reflects the similarity of the spatial features. The higher the value, the more similar the two features are in the spatial domain.

[0088] The calculation of frequency similarity is based on the frequency domain feature representation of the current frame and the previous frame obtained by FFT. First, based on the foreground spatial feature f cur , f prev Get their respective frequency distribution F cur and F prev , and then the frequency similarity can be calculated by the following formula:

[0089]

[0090] This frequency similarity measures the consistency between two frames in different frequency components. A higher value indicates that the frequency domain features are more similar.

[0091] The comprehensive similarity evaluation combines the results of spatial similarity and frequency domain similarity to generate the final evaluation result of foreground similarity of adjacent frames. Here, a weighted approach is used to integrate spatial and frequency similarities. The specific calculation method is as follows:

[0092] combined_similarity=α·cosine_similarity+(1-α)·frequency_similarity, (6)

[0093] Here, α is a weight coefficient. In the specific implementation, α=0.8. This comprehensive similarity is used as the final indicator of foreground similarity between adjacent frames to assist the interaction of foreground information between previous and next frames.

[0094] The combined similarity score combined_similarity is used to measure the degree to which the current frame refers to the foreground information of the previous frame. The higher the score, the more similar the previous frame is to the current frame, and the higher the degree of reference when using the previous frame to enhance the current frame. In the specific implementation, cimbined_simliarity is used as a weighted score to combine the foreground feature f of the previous frame with the foreground feature f of the previous frame. pre-r After weighting (x,y) we get f pre-ref (x,y), which is subsequently used to enhance the feature information of the current frame:

[0095] f pre-ref (x,y)=combined_similarity*f pre-r (x,y), (7)

[0096]

[0097] where f pre-ref (x,y) represents the feature map after weighting the foreground spatial feature information of the previous frame using combined_similarity; Represents the feature map of the current frame after the previous frame information is enhanced; f cur-r (x,y) represents the original spatial features of the current frame, f cur-r * (x,y) is the feature map of the original spatial features after high-frequency-intermediate frequency information enhancement.

[0098] (3) Frequency domain information enhancement (can be designed as a functional module, i.e., frequency domain information enhancement module):

[0099] Figure 3 This is a schematic diagram of the frequency enhancement module verifying the high-frequency, medium-frequency, and low-frequency components and focusing on the image information in an embodiment of the present invention. Figure 3 The frequency information enhancement module mainly utilizes the difference in focus between low-frequency and high-frequency information. Low-frequency information is concentrated in the central area of ​​the image spectrum, usually representing the overall outline, background and smooth area characteristics of the image. Low-frequency contains a large range of gradients and smooth changes in the image, which can represent the global structure and general semantic characteristics of the image; high-frequency information is located in the peripheral area of ​​the image spectrum, usually containing rapid changes and detail information in the image, such as object edges, textures and tiny details. These high-frequency components reflect the detail information in the image.

[0100] Through the above analysis, high-frequency components are used to enhance foreground information, while low-frequency components are used to enhance background information. This strengthens the image's understanding of foreground and background information, helping to better distinguish between them. Because the proportion of mid-frequency components increases after multiple convolutions, and they contain a variety of spatial details, foreground image feature enhancement is actually performed simultaneously with high- and mid-frequency components.

[0101] Specifically, for the spatial features f(x, y) obtained from the image, the same design as in frequency analysis is used in training and testing to use the real annotation mask or the network prediction frame of the previous frame as the mask to distinguish the image frame features into the background image spatial features f b (x,y) and foreground image spatial features f r (x, y). Then, the foreground and background features are transformed into the spectrum domain through FFT to obtain the frequency domain features F of the background and foreground of the current frame. b (u,v) and F r (u,v).

[0102] Then, for the background image feature F b (u,v) and foreground image features F r (u,v) are enhanced by low-frequency filtering and medium-high frequency filtering respectively. The implementation ideas for the foreground and background are the same, but the filters used are different. The following takes low-frequency filtering to enhance the spatial features of the background image as an example to illustrate the actual operation process:

[0103] The low-pass filter is used to enhance the low-frequency components, which helps to enrich the features with global semantic information. At the same time, the noise suppression characteristics of low frequencies can be used to better eliminate the interference of noise information in the background.

[0104] Specifically, the present invention generates a b The mapping coefficients H of (u, v) with the same spatial size are used to construct a Gaussian low-pass filter. The coefficients have a value of 0 at the center and increase in a Gaussian pattern towards the surrounding areas. Given the known information that the center of the FFT spectrum is high frequency and the surrounding areas are low frequency, the coefficients H well simulate a Gaussian low-pass filter.

[0105] Before updating the spectral domain, note that the spectral features are complex numbers for FFT operations. In order to make the complex features compatible with the neural layer, the complex number is split into the real part F br (u,v) and the imaginary part F bi (u, v), by connecting the imaginary part and the real part along the channel dimension to form a new two-channel tensor. In order to update the features in the spectral domain, a 1×1 convolution with a ReLU activation function is used, and then the result is divided into the real part and the imaginary part along the channel dimension, and the result is converted back to the complex number to obtain the filtered background frequency feature. The spectrum features are transferred back to the spatial domain through two-dimensional IFFT operation to obtain the new background image features after low-frequency enhancement. Finally, the updated features With the initial image space tensor f b (x,y) are added to output enhanced background features.

[0106] The foreground feature enhancement is similar to the background, except that the filters are replaced with high-frequency filters and medium-frequency filters, and the medium-frequency and high-frequency components are extracted and enhanced separately.

[0107] Finally, the unified enhanced image features of the foreground and background of the current frame are obtained as follows:

[0108] f enhanced (x,y)

[0109] =IFFT(H low (u,v)·F b (u,v))+IFFT(H high (u,v)·F r (u,v)+H mid (u,v)·F r (u,v)), (9)

[0110] in,

[0111] F b (u,v)=FFT(f b (x,y)), F r (u,v)=FFT(f r (x,y), (10)

[0112] Based on the contents of the above embodiments, the present invention verifies the advantages of the technical solution of the present invention through design experiments, and the experimental results are shown in the following table:

[0113]

[0114] Among them, F-Similarity represents the measurement of whether frequency similarity is introduced when the feature information of the previous frame is used to enhance the current frame; HM R-en represents whether high-frequency-medium-frequency information is used to enhance foreground features; L B-en represents whether low-frequency information is used to enhance background features; P, R, and Error represent the detection accuracy, recall rate, and false detection rate, respectively.

[0115] In a second aspect, the present invention further provides a target detection system based on frequency domain enhancement, comprising:

[0116] a first processing unit, configured to calculate a similarity of spatial features of foreground images of adjacent frames in the video to be detected as spatial similarity; wherein the adjacent frames include a previous frame and a current frame, the spatial features of the foreground image of the previous frame are first spatial features of the foreground image, and the spatial features of the foreground image of the current frame are second spatial features of the foreground image;

[0117] a second processing unit, configured to convert spatial features of foreground images of adjacent frames into frequency domain features of foreground images, and calculate similarities of the frequency domain features of the foreground images of adjacent frames as frequency similarities; wherein the frequency domain features of the foreground image of the previous frame are the first frequency domain features of the foreground image, and the frequency domain features of the foreground image of the current frame are the second frequency domain features of the foreground image;

[0118] a third processing unit, configured to filter the frequency domain features of the second foreground image using a preset medium-high frequency filter, and convert the filtered frequency domain features of the foreground image into spatial features of a third foreground image;

[0119] The fourth processing unit is configured to determine a final foreground image spatial feature of the current frame according to the spatial similarity, the frequency similarity, the first foreground image spatial feature, and the third foreground image spatial feature, so as to enhance the detection of the foreground object.

[0120] It should be noted that the target detection system based on frequency domain enhancement provided by the embodiment of the present invention can execute the target detection method based on frequency domain enhancement described in any of the above embodiments during specific operation, which will not be described in detail in this embodiment.

[0121] Figure 4 Schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the target detection method based on frequency domain enhancement.

[0122] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the target detection method based on frequency domain enhancement provided in the above-mentioned embodiments.

[0123] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the target detection method based on frequency domain enhancement provided in the above embodiments.

[0124] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A target detection method based on frequency domain enhancement, characterized in that: include: Calculating the similarity of the foreground image spatial features of adjacent frames in the video to be detected as the spatial similarity; wherein the adjacent frames include the previous frame and the current frame, the foreground image spatial features of the previous frame are the first foreground image spatial features, and the foreground image spatial features of the current frame are the second foreground image spatial features; Converting the spatial features of the foreground images of adjacent frames into frequency domain features of the foreground images, and calculating the similarity of the frequency domain features of the foreground images of adjacent frames as the frequency similarity; wherein the frequency domain features of the foreground image of the previous frame are the first frequency domain features of the foreground image, and the frequency domain features of the foreground image of the current frame are the second frequency domain features of the foreground image; Using a preset medium-high frequency filter to filter the frequency domain features of the second foreground image, and converting the filtered frequency domain features of the foreground image into spatial features of the third foreground image; Determining a final foreground image spatial feature of the current frame based on the spatial similarity, the frequency similarity, the first foreground image spatial feature, and the third foreground image spatial feature to enhance foreground object detection; Determining a final foreground image spatial feature of a current frame according to the spatial similarity, the frequency similarity, the first foreground image spatial feature, and the third foreground image spatial feature includes: Perform weighted averaging of spatial similarity and frequency similarity to determine comprehensive similarity; After weighted processing is performed on the first foreground image spatial feature using the comprehensive similarity as a weighted score, the third foreground image spatial feature is enhanced to determine the final foreground image spatial feature of the current frame.

2. The target detection method based on frequency domain enhancement according to claim 1, characterized in that: Also includes: Filtering the second background image frequency domain feature using a preset low-frequency filter, and converting the filtered background image frequency domain feature into a third background image spatial feature; wherein the second background image frequency domain feature is the background image frequency domain feature of the current frame; Determining a final background image spatial feature of the current frame based on the spatial similarity, the frequency similarity, the first background image spatial feature, and the third background image spatial feature to weaken background interference; The first background image spatial feature is the background image spatial feature of the previous frame.

3. The target detection method based on frequency domain enhancement according to claim 2, characterized in that: Determining the final background image spatial feature of the current frame according to the spatial similarity, the frequency similarity, the first background image spatial feature, and the third background image spatial feature, including: Perform weighted averaging of spatial similarity and frequency similarity to determine comprehensive similarity; After weighted processing is performed on the first background image spatial feature using the comprehensive similarity as a weighted score, the third background image spatial feature is enhanced to determine the final background image spatial feature of the current frame.

4. The target detection method based on frequency domain enhancement according to claim 2, characterized in that: in, The low frequency filter Expressed as: ； High-frequency filter Expressed as: ； IF filter Expressed as: ； Among them, u and v represent the horizontal and vertical coordinates in the frequency domain respectively, and the parameters Used to control the range of low-frequency components, parameters Used to control the radius of the high-pass filter, and are two Gaussian filters with different radii.

5. The target detection method based on frequency domain enhancement according to claim 2, characterized in that: Also includes: determining an enhanced image feature of the current frame according to the third foreground image spatial feature and the third background image spatial feature; Obtain an enhanced image of the current frame according to the enhanced image features of the current frame.

6. The target detection method based on frequency domain enhancement according to claim 1, characterized in that: The video to be detected is a colonoscopy video, and the target to be detected is intestinal polyps.

7. A target detection system based on frequency domain enhancement, characterized in that: include: a first processing unit, configured to calculate a similarity of spatial features of foreground images of adjacent frames in the video to be detected as spatial similarity; wherein the adjacent frames include a previous frame and a current frame, the spatial features of the foreground image of the previous frame are first spatial features of the foreground image, and the spatial features of the foreground image of the current frame are second spatial features of the foreground image; a second processing unit, configured to convert spatial features of foreground images of adjacent frames into frequency domain features of foreground images, and calculate similarities of the frequency domain features of the foreground images of adjacent frames as frequency similarities; wherein the frequency domain features of the foreground image of the previous frame are the first frequency domain features of the foreground image, and the frequency domain features of the foreground image of the current frame are the second frequency domain features of the foreground image; a third processing unit, configured to filter the frequency domain features of the second foreground image using a preset medium-high frequency filter, and convert the filtered frequency domain features of the foreground image into spatial features of a third foreground image; a fourth processing unit, configured to determine a final foreground image spatial feature of a current frame based on the spatial similarity, the frequency similarity, the first foreground image spatial feature, and the third foreground image spatial feature, so as to enhance detection of foreground objects; Determining a final foreground image spatial feature of a current frame according to the spatial similarity, the frequency similarity, the first foreground image spatial feature, and the third foreground image spatial feature includes: Perform weighted averaging of spatial similarity and frequency similarity to determine comprehensive similarity; After weighted processing is performed on the first foreground image spatial feature using the comprehensive similarity as a weighted score, the third foreground image spatial feature is enhanced to determine the final foreground image spatial feature of the current frame.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the target detection method based on frequency domain enhancement according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the target detection method based on frequency domain enhancement according to any one of claims 1 to 6 are implemented.

Images