Target detection method and system based on frequency domain enhancement

Abstract

The invention provides a target detection method and system based on frequency domain enhancement, and belongs to the technical field of computer vision, and the method comprises the steps: calculating the similarity of foreground image spatial features of adjacent frames in a to-be-detected video, and taking the similarity as spatial similarity; converting the foreground image spatial features of the adjacent frames into foreground image frequency domain features, and calculating the similarity of the foreground image frequency domain features of the adjacent frames as frequency similarity; performing filtering processing on the second foreground image frequency domain feature by using a preset medium-high frequency filter, and converting the filtered foreground image frequency domain feature into a third foreground image spatial feature; and according to the spatial similarity, the frequency similarity, the first foreground image spatial feature and the third foreground image spatial feature, determining a final foreground image spatial feature of the current frame so as to strengthen detection of a foreground target. According to the method, the semantic relation between the image pixel features and the target is better established, feature enhancement is realized, and the detection precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a target detection method and system based on frequency domain enhancement. Background Art

[0003] Colonoscopy is the gold standard for diagnosing colorectal cancer and precancerous lesions, and is the cornerstone of CRC prevention. Diagnosing and treating precancerous polyps through colonoscopy and removing them before they turn into cancer can significantly reduce the incidence and mortality of CRC and improve the survival rate of patients. Although colonoscopy provides information on the appearance and location of polyps, it requires expensive labor resources and has a high misdiagnosis rate of up to 27% due to the following reasons: First, colonoscopy screening is highly professional, the accuracy of diagnosis is heavily dependent on the doctor's experience, and manual screening consumes a lot of time and cost; on the other hand, polyps show significant differences in color, size and shape, and polyps are similar in appearance to surrounding tissues, with unclear boundaries and strong concealment.

[0004] Therefore, it is necessary to develop an automated and accurate colorectal polyp detection system to reduce the burden of manual screening and the occurrence of misdiagnosis. Currently, traditional methods and colonoscopy target detection methods based on deep learning are only designed at the image level, and few works focus on the advantages of frequency domain information in enhancing image features to achieve better target detection. Summary of the invention

[0005] The present invention provides a target detection method and system based on frequency domain enhancement, which is used to solve the defect that most of the existing technologies ignore the potential of frequency domain analysis in feature enhancement, realize the division of frequency spectrum information obtained from image features into different frequency ranges, and specifically use the frequency domain features of different frequency bands to enhance the foreground and background of the image respectively.

[0006] In a first aspect, the present invention provides a target detection method based on frequency domain enhancement, comprising: calculating the similarity of foreground image spatial features of adjacent frames in a video to be detected as spatial similarity; wherein the adjacent frames include a previous frame and a current frame, the foreground image spatial features of the previous frame are first foreground image spatial features, and the foreground image spatial features of the current frame are second foreground image spatial features;

[0007] The spatial features of the foreground images of adjacent frames are converted into frequency domain features of the foreground images, and the similarity of the frequency domain features of the foreground images of adjacent frames is calculated as the frequency similarity; wherein the frequency domain features of the foreground images of the previous frame are the first frequency domain features of the foreground images, and the frequency domain features of the foreground images of the current frame are the second frequency domain features of the foreground images;

[0008] Using a preset medium-high frequency filter to filter the frequency domain features of the second foreground image, and converting the filtered frequency domain features of the foreground image into a third foreground image spatial feature;

[0009] According to the spatial similarity, the frequency similarity, the first foreground image spatial feature and the third foreground image spatial feature, the final foreground image spatial feature of the current frame is determined to enhance the detection of the foreground object.

[0010] According to a target detection method based on frequency domain enhancement provided by the present invention, the method also includes: using a pre-set low-frequency filter to filter the frequency domain features of the second background image, and converting the filtered background image frequency domain features into third background image spatial features; wherein the second background image frequency domain features are the background image frequency domain features of the current frame; based on spatial similarity, frequency similarity, the first background image spatial features and the third background image spatial features, determining the final background image spatial features of the current frame to weaken background interference; wherein the first background image spatial features are the background image spatial features of the previous frame.

[0011] According to a target detection method based on frequency domain enhancement provided by the present invention, the final foreground image spatial features of the current frame are determined according to spatial similarity, frequency similarity, first foreground image spatial features and third foreground image spatial features, including: taking a weighted average of spatial similarity and frequency similarity to determine comprehensive similarity; after weighting the first foreground image spatial features using the comprehensive similarity as a weighted score, enhancing the third foreground image spatial features to determine the final foreground image spatial features of the current frame.

[0012] According to a target detection method based on frequency domain enhancement provided by the present invention, the final background image spatial features of the current frame are determined according to spatial similarity, frequency similarity, first background image spatial features and third background image spatial features, including: taking a weighted average of the spatial similarity and the frequency similarity to determine the comprehensive similarity; after weighting the first background image spatial features using the comprehensive similarity as a weighted score, enhancing the third background image spatial features to determine the final background image spatial features of the current frame.

[0013] According to a target detection method based on frequency domain enhancement provided by the present invention, the spatial similarity and the frequency similarity are characterized by using pre-similarity.

[0014] According to a frequency domain enhancement-based target detection method provided by the present invention, the method also includes: determining enhanced image features of the current frame based on a third foreground image spatial feature and a third background image spatial feature; and obtaining an enhanced image of the current frame based on the enhanced image features of the current frame.

[0015] According to a target detection method based on frequency domain enhancement provided by the present invention, the video to be detected is a colonoscopy video, and the target to be detected is an intestinal polyp.

[0016] In a second aspect, the present invention further provides a target detection system based on frequency domain enhancement, comprising:

[0017] The first processing unit is used to calculate the similarity of the foreground image spatial features of adjacent frames in the video to be detected as the spatial similarity; wherein the adjacent frames include a previous frame and a current frame, the foreground image spatial features of the previous frame are first foreground image spatial features, and the foreground image spatial features of the current frame are second foreground image spatial features;

[0018] The second processing unit is used to convert the spatial features of the foreground images of adjacent frames into frequency domain features of the foreground images, and calculate the similarity of the frequency domain features of the foreground images of adjacent frames as the frequency similarity; wherein the frequency domain features of the foreground images of the previous frame are the first frequency domain features of the foreground images, and the frequency domain features of the foreground images of the current frame are the second frequency domain features of the foreground images;

[0019] A third processing unit is used to filter the frequency domain features of the second foreground image using a preset medium-high frequency filter, and convert the filtered frequency domain features of the foreground image into a third foreground image spatial feature;

[0020] The fourth processing unit is used to determine the final foreground image spatial feature of the current frame according to the spatial similarity, the frequency similarity, the first foreground image spatial feature and the third foreground image spatial feature to enhance the detection of the foreground target.

[0021] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the target detection method based on frequency domain enhancement as described above are implemented.

[0022] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the frequency domain enhancement-based target detection methods described above.

[0023] The target detection method and system based on frequency domain enhancement provided by the present invention have the following beneficial effects compared with the prior art:

[0024] (1) The present invention makes full use of the frequency domain information within the frame. By performing targeted analysis on the extracted frequency domain information and constraining the image features based on the frequency information, the semantic connection between the image pixel features and the target is better established, feature enhancement is achieved, and detection accuracy is improved.

[0025] (2) The present invention starts from the perspective of segmented analysis of different frequency bands of the image feature spectrum in the target detection scenario, considers the differences in the areas of interest using information in different frequency bands, and proposes a complete set of frequency extraction, analysis and enhancement methods.

[0026] (3) The foreground similarity analysis designed by the present invention integrates frequency similarity analysis. Compared with the method that only considers spatial feature similarity, the foreground similarity analysis integrates frequency information similarity and spatial similarity for weighted analysis, which can better improve the effectiveness of adjacent frame references in dynamic scenes and complex backgrounds.

[0027] (4) The present invention specifically utilizes high-frequency, medium-frequency and low-frequency information to enhance image features based on the foreground and background information of the image, highlighting foreground details, weakening background interference, and improving the detection effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 It is a schematic diagram of a framework for enhancing colonoscopy target detection using spectrum information designed in an embodiment of the present invention;

[0030] Figure 2 It is a schematic diagram of a process for enhancing colonoscopy target detection by using spectrum information designed in an embodiment of the present invention;

[0031] Figure 3 It is a schematic diagram of focusing on image information of high-frequency, medium-frequency and low-frequency components for a frequency enhancement module in an embodiment of the present invention;

[0032] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] It should be noted that, in the description of the embodiments of the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0035] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0036] First, the overall process of the present invention is described:

[0037] Step 1: Calculate the similarity of the spatial features of the foreground images of adjacent frames in the video to be detected as the spatial similarity.

[0038] The video to be detected may be a colonoscopy video, the adjacent frames include a previous frame and a current frame, the foreground image spatial feature of the previous frame is a first foreground image spatial feature, and the foreground image spatial feature of the current frame is a second foreground image spatial feature.

[0039] The foreground image spatial features are extracted from the foreground image of the image frame using a convolutional neural network; among them, the spatial similarity is used to indicate the similarity of the foreground objects between two frames in the spatial domain.

[0040] Step 2: The spatial features of the foreground images of adjacent frames are converted into frequency domain features of the foreground images, and the similarity of the frequency domain features of the foreground images of adjacent frames is calculated as the frequency similarity.

[0041] The image frequency domain features can be obtained by converting the image space features based on Fourier transform, wherein the foreground image frequency domain features of the previous frame are the first foreground image frequency domain features, and the foreground image frequency domain features of the current frame are the second foreground image frequency domain features.

[0042] Frequency similarity is used to reflect the similarity of foreground objects between two frames in the frequency domain.

[0043] Step 3: Filter the frequency domain features of the second foreground image using a preset medium-high frequency filter, and convert the filtered frequency domain features of the foreground image into the third foreground image spatial features.

[0044] Among them, the medium-high frequency filter refers to a medium frequency filter and a high frequency filter. The present invention can realize the conversion of spatial characteristics and frequency domain characteristics through Fourier transformation and inverse Fourier transformation.

[0045] Step 4: Determine the final foreground image spatial features of the current frame based on the spatial similarity, the frequency similarity, the first foreground image spatial features, and the third foreground image spatial features to enhance the detection of the foreground target.

[0046] Optionally, step 4 includes:

[0047] Step 401: Perform weighted averaging on the spatial similarity and the frequency similarity to determine the comprehensive similarity.

[0048] Among them, the weights can be adjusted according to the specific application scenarios or experimental results to ensure that the comprehensive similarity can accurately reflect the changes of the foreground targets in the spatial domain and frequency domain.

[0049] Step 402: After weighting the first foreground image spatial feature by taking the comprehensive similarity as a weighted score, the third foreground image spatial feature is enhanced to determine the final foreground image spatial feature of the current frame.

[0050] Based on the content of the above embodiment, as an optional embodiment, the target detection method based on frequency domain enhancement provided by the present invention further includes:

[0051] Step 5: Utilize a preset low-frequency filter to filter the frequency domain features of the second background image, and convert the filtered frequency domain features of the background image into the third background image spatial features.

[0052] The second background image frequency domain feature is the background image frequency domain feature of the current frame, which is obtained by Fourier transforming the background image spatial feature, and will not be described here. The background image spatial feature is extracted from the background image of the image frame using a convolutional neural network.

[0053] Step 6: Determine the final background image spatial feature of the current frame according to the spatial similarity, the frequency similarity, the first background image spatial feature and the third background image spatial feature to weaken the background interference.

[0054] The first background image spatial feature is the background image spatial feature of the previous frame.

[0055] Optionally, step 6 includes:

[0056] Step 601: performing weighted averaging of spatial similarity and frequency similarity to determine comprehensive similarity;

[0057] Step 602: After weighting the first background image spatial feature by taking the comprehensive similarity as a weighted score, the third background image spatial feature is enhanced to determine the final background image spatial feature of the current frame.

[0058] Based on the content of the above embodiment, as an optional embodiment, the target detection method based on frequency domain enhancement provided by the present invention further includes:

[0059] Step 7: Determine the enhanced image features of the current frame according to the third foreground image spatial features and the third background image spatial features;

[0060] The third foreground image spatial feature and the third background image spatial feature are both foreground image spatial features after frequency information enhancement.

[0061] Step 8: Obtain an enhanced image of the current frame based on the enhanced image features of the current frame.

[0062] Optionally, in the above embodiment, both the spatial similarity and the frequency similarity may be characterized by using cosine similarity.

[0063] The following example takes the colonoscopy video as the video to be tested and the intestinal polyp as the target for testing. Figure 1 and Figure 2 , the technical solution of the present invention is further analyzed and explained from three aspects (modules) of frequency extraction, analysis and enhancement.

[0064] (1) Frequency domain information extraction (can be designed as a functional module, namely, a frequency domain information extraction module):

[0065] For the image frames of continuous colonoscopy videos, the image information is first extracted through a convolutional neural network to obtain the spatial feature information of the image frame. For the spatial feature map, the spatial information is converted into frequency information using Fourier transform FFT. Let the Fourier transform result of the spatial feature f(x, y) of the image be F(u, v), where x and y represent the horizontal and vertical coordinates in the spatial domain, respectively, and u and v represent the horizontal and vertical coordinates in the frequency domain, respectively.

[0066] The output of the Fourier transform includes real and imaginary parts, which represent the distribution of the feature map at different frequencies. By observing the frequency distribution of the spectrum, we can clearly distinguish the low-frequency, high-frequency and medium-frequency components, where the low-frequency area is often located in the center of the spectrum and the high-frequency area is located at the edge. The components of different frequencies of the image contribute differently to specific tasks. The low-frequency information provides the overall outline of the image, while the high-frequency components represent the edges and details in the image.

[0067] In the frequency domain, low frequencies mainly contain the overall contour or general semantic information of the image, while high frequencies contain edge and detail information. The main target polyp images in colonoscopy videos often contrast sharply with low-resolution, nearly flat textures, and the polyps appear raised in the image, which is different from the colonoscopy mucosa. However, polyps are highly similar to intestinal mucosa in appearance, and are highly hidden when analyzed from the image and spatial features, so a new perspective is needed for additional auxiliary detection based on spatial features. Spectrum-based methods are a good way to distinguish foreground and background from the perspective of focusing on targets in different spectrum bands.

[0068] It can be observed that after the convolution stage, the amplitude ratio of the middle subband frequency of the high and low subbands to the entire spectrum is getting higher and higher. Therefore, when using the frequency information of the image features after convolution for analysis, in addition to considering the high-frequency components that focus on details and the low-frequency components of general semantic information, the middle-frequency subband containing rich global information of polyps also needs to be taken into consideration.

[0069] Therefore, for the frequency domain features F(u,v) obtained by FFT, low-frequency filters, intermediate-frequency filters and high-frequency filters are designed respectively to separate the different frequency bands of the spectrum information.

[0070] The low-frequency filter uses a small-scale Gaussian filter to focus on the center area of ​​the spectrum, mainly retaining the low-frequency information of the image, which is suitable for capturing the overall outline and smooth background. The high-frequency filter uses a larger-scale Gaussian filter to filter the center part of the spectrum, retaining only the high-frequency information around it to enhance the edge and detail features. The mid-frequency filter retains the information in the middle part of the spectrum by subtracting two Gaussian filters of different scales, which helps capture local details and transition information, enhances the contrast between the foreground and the background, and enables the model to more clearly identify the complete structure of the object.

[0071] The design goal of the low-frequency filter is to preserve the central area of ​​the spectrum. The low-frequency filter is a Gaussian filter with a small radius. The implementation principle is as follows:

[0072]

[0073] Among them, the parameter σ low Used to control the range of low-frequency components.

[0074] The purpose of high-frequency filter design is to suppress the central area of ​​the spectrum (low-frequency part) in the frequency domain and retain the edge area of ​​the spectrum (high-frequency part). The basic principle formula is as follows:

[0075]

[0076] Among them, the parameter σ high Used to control the radius of the high-pass filter, that is, the frequency selection range of the filter.

[0077] The intermediate frequency filter is constructed by interpolating two Gaussian high-pass filters of different radii to retain the information in the intermediate frequency region. The specific implementation principle is as follows:

[0078] H mid (u,v)=H high1 (u,v)-H hiigh2 (u,v), (3)

[0079] Filter H hiigh1 (u,v) and H hiigh2 The radii of (u,v) are σ hiigh1 and σ hiigh2 , and satisfy σ hiigh1 <σ hiigh2 .

[0080] (2) Frequency domain information analysis (can be designed as a functional module, namely, frequency domain information analysis module):

[0081] Usually, since the video is continuous, the foreground targets of two adjacent frames often have high similarity. By using the similarity relationship between the foreground of the previous frame and the foreground of the current frame, the degree of feature enhancement of the current frame using the previous frame is controlled, and the foreground features of the current frame are enhanced, which can better help the image complete the detection of foreground targets in the current frame.

[0082] To alleviate the problem that the spatial information exhibits great differences when the scene changes rapidly or the foreground target moves significantly, the foreground information interaction using only spatial similarity control will have large errors. Spectral information can keep the foreground frequency information relatively stable when there is a lot of environmental interference information such as lighting, noise and motion. Introducing spectral information to assist the foreground information interaction of adjacent frames can make up for the limitations of spatial feature similarity reference.

[0083] The specific implementation is to first input the image frames of the video into the network frame by frame in the form of I[i], so as to maintain the order of the images well. For each specific frame of image I[i], except for the first frame selecting itself for foreground interaction, the subsequent frames all interact with its previous frame I[i-1] foreground interaction.

[0084] In order to achieve only foreground information interaction, during the training process, the GT image frame is used to generate a mask to multiply the image to obtain the foreground and background of each frame; when there is no GT annotation, the prediction result of the previous frame is used as the GT frame of the current frame to obtain the foreground from the image.

[0085] For adjacent image frames I[i] and I[i-1], the spatial feature information is first mapped to the frequency domain using the same frequency feature extraction module and the same FFT, thereby obtaining the frequency features of the foreground. The spatial similarity and frequency similarity are then calculated respectively.

[0086] For the extracted foreground spatial features (i.e., foreground image spatial features), the cosine similarity is calculated. The calculation formula of spatial cosine similarity is as follows:

[0087]

[0088] Among them, f cur , f prev Represent the foreground spatial features of the current frame and the previous frame respectively. The similarity reflects the similarity of the spatial features. The higher the value, the more similar the two features are in the spatial domain.

[0089] The calculation of frequency similarity is based on the frequency domain feature representation of the current frame and the previous frame obtained by FFT. First, based on the foreground spatial feature f cur , f prev Get their respective frequency distribution F cur and F prev , then the frequency similarity can be calculated by the following formula:

[0090]

[0091] This frequency similarity measures the consistency between two frames in different frequency components. The higher the value, the more similar the frequency domain features are.

[0092] The comprehensive evaluation of similarity combines the results of spatial similarity and frequency domain similarity to generate the final evaluation result of foreground similarity of adjacent frames. Here, a weighted method is used to integrate spatial and frequency similarities. The specific calculation method is as follows:

[0093] combined_similarity=α·cosine_similarity+(1-α)·frequency_similarity, (6)

[0094] Wherein, α is a weight coefficient, and in the specific implementation, α=0.8. This comprehensive similarity is used as the final indicator of the foreground similarity of adjacent frames, and is used to assist the foreground information interaction of the previous and next frames.

[0095] The combined similarity score combined_similarity is used to measure the degree to which the current frame refers to the foreground information of the previous frame. The higher the score, the more similar the previous frame is to the current frame, and the higher the degree of reference is when the previous frame is used to enhance the current frame. In specific implementation, cimbined_simliarity is used as a weighted score to weight the foreground feature f of the previous frame. pre-r After weighting (x,y), we get f pre-ref (x,y), which is used to enhance the feature information of the current frame later:

[0096] f pre-ref (x,y)=combined_similarity*f pre-r (x,y), (7)

[0097]

[0098] where f pre-ref (x, y) represents the feature map after the foreground spatial feature information of the previous frame is weighted using combined_similarity; represents the feature map of the current frame after the previous frame information is enhanced; f cur-r (x, y) represents the original spatial features of the current frame, f cur-r * (x,y) is the feature map of the original spatial features after high-frequency-mid-frequency information enhancement.

[0099] (3) Frequency domain information enhancement (can be designed as a functional module, namely, frequency domain information enhancement module):

[0100] Figure 3 FIG. 1 is a schematic diagram of a frequency enhancement module for verifying the focus of image information of high frequency, medium frequency and low frequency components in an embodiment of the present invention, see Figure 3 The frequency information enhancement module mainly utilizes the difference in the focus of low-frequency and high-frequency information. The low-frequency information is concentrated in the central area of ​​the image spectrum, usually representing the overall contour, background and smooth area characteristics of the image. The low frequency contains a large range of gradients and smooth changes in the image, which can represent the global structure and general semantic characteristics of the image; the high-frequency information is located in the peripheral area of ​​the image spectrum, usually containing rapid changes and detail information in the image, such as object edges, textures and tiny details. These high-frequency components reflect the detail information in the image.

[0101] After the above analysis, the high-frequency components are used to enhance the foreground information, and the low-frequency components are used to enhance the background information, thereby enhancing the image's understanding of the foreground and background information and better helping the image distinguish between the foreground and background information. Since the proportion of the intermediate frequency components increases after multiple convolutions and contains a variety of spatial detail information, the high-frequency and intermediate frequencies are combined for simultaneous enhancement when actually enhancing the foreground image features.

[0102] Specific implementation First, for the spatial features f(x, y) obtained from the image, the same design as in frequency analysis is used in training and testing, respectively, to use the real annotation mask or the network prediction frame of the previous frame as the mask to distinguish the image frame features into the background image spatial features f b (x, y) and the foreground image spatial feature f r (x, y). Then, the foreground and background features are transformed into the spectrum domain through FFT to obtain the frequency domain features F of the background and foreground of the current frame. b (u,v) and F r (u,v).

[0103] Then, for the background image feature F b (u,v) and foreground image features F r (u,v) are enhanced by low-frequency filtering and medium-high frequency filtering respectively. The implementation ideas for the foreground and background are the same, but the filters used are different. The following takes low-frequency filtering to enhance the spatial characteristics of the background image as an example to illustrate the actual operation process:

[0104] The low-frequency components are enhanced by low-pass filters, which helps to enrich the features with global semantic information. At the same time, the noise suppression characteristics of low frequencies can be used to better eliminate the interference of noise information in the background.

[0105] Specifically, the present invention generates a b (u,v) has the same spatial size mapping coefficient H to construct a Gaussian low-pass filter. The center of the coefficient has a value of 0 and increases in a Gaussian manner. According to the known information that the center of the spectrum after FFT is high frequency and the surrounding is low frequency, the coefficient H simulates a Gaussian low-pass filter very well.

[0106] Before updating the spectral domain, note that the spectral features are complex numbers for FFT operations. In order to make the complex features compatible with the neural layer, the complex number is split into the real part F br (u,v) and the imaginary part F bi (u,v), and a new two-channel tensor is formed by concatenating the imaginary and real parts along the channel dimension. To update the features in the spectral domain, a 1×1 convolution with a ReLU activation function is used, and then the result is divided into real and imaginary parts along the channel dimension, and the result is converted back to a complex number to obtain the filtered background frequency features. The spectrum features are transferred back to the spatial domain through two-dimensional IFFT operation to obtain the new background image features after low-frequency enhancement. Finally, the updated features With the initial image space tensor f b (x,y) are added to output enhanced background features.

[0107] The foreground feature enhancement is similar to the background, except that the filters are replaced with high-frequency filters and medium-frequency filters, and the medium-frequency and high-frequency components are extracted and enhanced separately.

[0108] Finally, the unified enhanced image features of the foreground and background of the current frame are obtained as follows:

[0109] f enhanced (x,y)

[0110] =IFFT(H low (u,v)·F b (u,v))+IFFT(H high (u,v)·F r (u,v)+H mid (u,v)·F r (u,v)), (9)

[0111] in,

[0112] F b (u,v)=FFT(f b (x,y)), F r (u,v)=FFT(f r (x,y), (10)

[0113] Based on the contents of the above embodiments, the present invention verifies the advantages of the technical solution of the present invention through design experiments, and the experimental results are shown in the following table:

[0114]

[0115] Among them, F-Similarity represents the measurement of whether frequency similarity is introduced when the feature information of the previous frame is used to enhance the current frame; HM R-en represents whether high-frequency-medium-frequency information is used to enhance the foreground features; L B-en represents whether low-frequency information is used to enhance the background features; P, R, and Error represent the detection accuracy, recall rate, and false detection rate, respectively.

[0116] In a second aspect, the present invention further provides a target detection system based on frequency domain enhancement, comprising:

[0117] The first processing unit is used to calculate the similarity of the foreground image spatial features of adjacent frames in the video to be detected as the spatial similarity; wherein the adjacent frames include a previous frame and a current frame, the foreground image spatial features of the previous frame are first foreground image spatial features, and the foreground image spatial features of the current frame are second foreground image spatial features;

[0118] The second processing unit is used to convert the spatial features of the foreground images of adjacent frames into frequency domain features of the foreground images, and calculate the similarity of the frequency domain features of the foreground images of adjacent frames as the frequency similarity; wherein the frequency domain features of the foreground images of the previous frame are the first frequency domain features of the foreground images, and the frequency domain features of the foreground images of the current frame are the second frequency domain features of the foreground images;

[0119] A third processing unit is used to filter the frequency domain features of the second foreground image using a preset medium-high frequency filter, and convert the filtered frequency domain features of the foreground image into a third foreground image spatial feature;

[0120] The fourth processing unit is used to determine the final foreground image spatial feature of the current frame according to the spatial similarity, the frequency similarity, the first foreground image spatial feature and the third foreground image spatial feature to enhance the detection of the foreground target.

[0121] It should be noted that the target detection system based on frequency domain enhancement provided in the embodiment of the present invention can execute the target detection method based on frequency domain enhancement described in any of the above embodiments during specific operation, which will not be elaborated in this embodiment.

[0122] Figure 4 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the target detection method based on frequency domain enhancement.

[0123] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the target detection method based on frequency domain enhancement provided in the above-mentioned embodiments.

[0124] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the target detection method based on frequency domain enhancement provided in the above-mentioned embodiments.

[0125] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A target detection method based on frequency domain enhancement, characterized in that: include: Calculate the similarity of the foreground image spatial features of adjacent frames in the video to be detected as the spatial similarity; wherein the adjacent frames include a previous frame and a current frame, the foreground image spatial features of the previous frame are the first foreground image spatial features, and the foreground image spatial features of the current frame are the second foreground image spatial features; The spatial features of the foreground images of adjacent frames are converted into frequency domain features of the foreground images, and the similarity of the frequency domain features of the foreground images of adjacent frames is calculated as the frequency similarity; wherein the frequency domain features of the foreground images of the previous frame are the first frequency domain features of the foreground images, and the frequency domain features of the foreground images of the current frame are the second frequency domain features of the foreground images; Using a preset medium-high frequency filter to filter the frequency domain features of the second foreground image, and converting the filtered frequency domain features of the foreground image into a third foreground image spatial feature; According to the spatial similarity, the frequency similarity, the first foreground image spatial feature and the third foreground image spatial feature, the final foreground image spatial feature of the current frame is determined to enhance the detection of the foreground object.

2. The target detection method based on frequency domain enhancement according to claim 1, characterized in that: Also includes: The frequency domain features of the second background image are filtered using a preset low-frequency filter, and the frequency domain features of the background image after filtering are converted into the third background image spatial features; wherein the frequency domain features of the second background image are the frequency domain features of the background image of the current frame; Determine the final background image spatial feature of the current frame according to the spatial similarity, the frequency similarity, the first background image spatial feature and the third background image spatial feature to weaken background interference; The first background image spatial feature is the background image spatial feature of the previous frame.

3. The target detection method based on frequency domain enhancement according to claim 1, characterized in that: Determining the final foreground image spatial feature of the current frame according to the spatial similarity, the frequency similarity, the first foreground image spatial feature, and the third foreground image spatial feature, including: The spatial similarity and frequency similarity are weighted averaged to determine the comprehensive similarity; After weighted processing is performed on the first foreground image spatial feature by taking the comprehensive similarity as a weighted score, the third foreground image spatial feature is enhanced to determine the final foreground image spatial feature of the current frame.

4. The target detection method based on frequency domain enhancement according to claim 2, characterized in that: Determining the final background image spatial feature of the current frame according to the spatial similarity, the frequency similarity, the first background image spatial feature, and the third background image spatial feature, including: The spatial similarity and frequency similarity are weighted averaged to determine the comprehensive similarity; After weighted processing is performed on the first background image spatial feature by taking the comprehensive similarity as a weighted score, the third background image spatial feature is enhanced to determine the final background image spatial feature of the current frame.

5. The target detection method based on frequency domain enhancement according to claim 2, characterized in that: in, The low frequency filter H low (u,v) is expressed as: High frequency filter H high (u,v) is expressed as: IF filter H mid (u,v) is expressed as: H mid (u,v)=H high1 (u,v)-H high2 (u,v); Among them, u and v represent the horizontal and vertical coordinates in the frequency domain respectively, and the parameter σ low Used to control the range of low-frequency components, parameter σ high Used to control the radius of the high-pass filter. H high1 (u,v) and H high2 (u,v) are two Gaussian filters with different radii.

6. The target detection method based on frequency domain enhancement according to claim 2, characterized in that: Also includes: Determining enhanced image features of the current frame according to the third foreground image spatial features and the third background image spatial features; According to the enhanced image features of the current frame, an enhanced image of the current frame is obtained.

7. The target detection method based on frequency domain enhancement according to claim 1, characterized in that: The video to be detected is a colonoscopy video, and the target to be detected is intestinal polyps.

8. A target detection system based on frequency domain enhancement, characterized in that: include: The first processing unit is used to calculate the similarity of the foreground image spatial features of adjacent frames in the video to be detected as the spatial similarity; wherein the adjacent frames include a previous frame and a current frame, the foreground image spatial features of the previous frame are first foreground image spatial features, and the foreground image spatial features of the current frame are second foreground image spatial features; The second processing unit is used to convert the spatial features of the foreground images of adjacent frames into frequency domain features of the foreground images, and calculate the similarity of the frequency domain features of the foreground images of adjacent frames as the frequency similarity; wherein the frequency domain features of the foreground images of the previous frame are the first frequency domain features of the foreground images, and the frequency domain features of the foreground images of the current frame are the second frequency domain features of the foreground images; A third processing unit is used to filter the frequency domain features of the second foreground image using a preset medium-high frequency filter, and convert the filtered frequency domain features of the foreground image into a third foreground image spatial feature; The fourth processing unit is used to determine the final foreground image spatial feature of the current frame according to the spatial similarity, the frequency similarity, the first foreground image spatial feature and the third foreground image spatial feature to enhance the detection of the foreground target.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the target detection method based on frequency domain enhancement as described in any one of claims 1 to 7 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the target detection method based on frequency domain enhancement as claimed in any one of claims 1 to 7 are implemented.

Images