Visual feature modeling method, device, storage medium and computer program product
By mapping images to the frequency domain and using the underdamped wave equation for oscillation propagation, the problems of high computational complexity and loss of high-frequency details in traditional visual feature modeling methods are solved, thereby improving computational efficiency and accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- PEKING UNIV SHENZHEN GRADUATE SCHOOL
- Filing Date
- 2026-03-12
- Publication Date
- 2026-06-16
AI Technical Summary
Traditional visual feature modeling methods are computationally complex and lose high-frequency details, making it difficult to meet real-time requirements and visual task performance.
The image to be processed is mapped to the frequency domain, and the underdamped wave equation is used to oscillate and propagate each frequency component. The damping coefficient and propagation speed are adjusted to obtain the frequency domain features, which are then mapped to the spatial domain to form a visual feature map.
The computational complexity was reduced from O(N²) to O(N log N), while high-frequency detail information was preserved, thus improving the accuracy of visual feature modeling.
Smart Images
Figure CN122223355A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer vision technology, and in particular to visual feature modeling methods, devices, storage media and computer program products. Background Technology
[0002] Traditional visual feature modeling methods (such as Convolutional Neural Networks (CNNs) and Transformers with self-attention mechanisms) face two main problems. First, they have high computational complexity: especially in high-resolution images or dense prediction tasks, existing self-attention mechanisms have a global computational complexity of O(N). 2 This leads to a sharp increase in computational demands, making it difficult to meet real-time requirements. Secondly, high-frequency details are lost during information transmission: high-frequency details are easily smoothed out during transmission, resulting in the loss of important information such as image edges and textures, thus affecting the effectiveness of visual tasks.
[0003] Therefore, how to improve the accuracy of visual feature modeling while reducing computational complexity has become a technical problem that this application urgently needs to solve. Summary of the Invention
[0004] The main objective of this application is to provide a visual feature modeling method, device, storage medium, and computer program product, aiming to solve the technical problem of how to improve the accuracy of visual feature modeling while reducing computational complexity.
[0005] To achieve the above objectives, this application proposes a visual feature modeling method, the method comprising: Obtain the image to be processed; Map the image to be processed to the frequency domain; The frequency components in the frequency domain are oscillated and propagated using an underdamped wave equation, and the damping coefficient and propagation speed of the underdamped wave equation are adjusted during the oscillation propagation to obtain the frequency domain characteristics. The frequency domain features are mapped to the spatial domain to obtain a visual feature map.
[0006] In one embodiment, the step of mapping the image to be processed to the frequency domain includes: Extract the initial semantic field from the image to be processed; The initial semantic field is mapped from the spatial domain to the frequency domain based on the two-dimensional Fourier transform, resulting in different frequency components located in the frequency domain.
[0007] In one embodiment, the step of using an underdamped wave equation to oscillate and propagate each frequency component in the frequency domain, and adjusting the damping coefficient and propagation speed of the underdamped wave equation during the oscillation propagation to obtain frequency domain characteristics includes: The propagation characteristic value of each frequency component in the frequency domain is calculated based on the underdamped wave equation to perform oscillation propagation. In the oscillation propagation, the propagation velocity and damping coefficient in the underdamped wave equation are adjusted according to the frequency component. The frequency domain characteristics are determined based on the propagation characteristic values of each frequency component.
[0008] In one embodiment, the step of calculating the propagation characteristic value of each frequency component in the frequency domain based on the underdamped wave equation includes: The damped oscillation frequency of each frequency component in the frequency domain is calculated based on the preset propagation speed and the preset damping coefficient. The cosine oscillation term and the sine oscillation term are calculated based on the damped oscillation frequency and the preset propagation time, respectively. An exponential decay factor is generated based on the propagation time. The propagation characteristic value of the frequency component is calculated based on the cosine oscillation term, the sine oscillation term, the exponential decay factor, and the underdamped wave equation.
[0009] In one embodiment, the step of determining the frequency domain features based on the propagation characteristic values of each frequency component includes: The propagation characteristic values of each frequency component are filled into the frequency domain according to the coordinate position of the frequency component to obtain the frequency domain features.
[0010] In one embodiment, the step of mapping the frequency domain features to the spatial domain to obtain a visual feature map includes: Perform an inverse Fourier transform on the frequency domain features to obtain the feature matrix in the spatial domain; The visual feature map is determined based on the feature matrix.
[0011] In one embodiment, after the step of mapping the frequency domain features to the spatial domain to obtain a visual feature map, the method further includes: The visual feature map is input into a pre-trained neural network model to perform the target visual task.
[0012] In addition, to achieve the above objectives, this application also proposes a visual feature modeling device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the visual feature modeling method as described above.
[0013] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the visual feature modeling method described above.
[0014] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the visual feature modeling method described above.
[0015] One or more technical solutions proposed in this application have at least the following technical effects: After mapping the image to be processed to the frequency domain, the underdamped wave equation is used in the frequency domain to perform independent oscillation propagation processing on each frequency component, avoiding global pairwise calculations in the spatial domain, thus reducing the computational complexity from O(N) to O(N). 2 The computational complexity is reduced to O(N log N). At the same time, by adjusting the damping coefficient and propagation speed of the underdamped wave equation, the propagation behavior of high-frequency and low-frequency components can be differentiated, so that high-frequency detail information is fully preserved during propagation, avoiding the over-smoothing problem of traditional propagation methods, thereby improving the accuracy of visual feature modeling. Attached Figure Description
[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This is a flowchart illustrating the first embodiment of the visual feature modeling method of this application. Figure 2 This is a flowchart illustrating the second embodiment of the visual feature modeling method of this application. Figure 3 This is a flowchart illustrating the third embodiment of the visual feature modeling method of this application. Figure 4 This is a schematic diagram of the module structure of the visual feature modeling device according to an embodiment of this application; Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the visual feature modeling method in the embodiments of this application.
[0019] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0020] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.
[0021] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.
[0022] The main solution of this application embodiment is: to acquire an image to be processed; to map the image to be processed to the frequency domain; to use an underdamped wave equation to oscillate and propagate each frequency component in the frequency domain, and to adjust the damping coefficient and propagation speed of the underdamped wave equation during the oscillation propagation to obtain frequency domain features; and to map the frequency domain features to the spatial domain to obtain a visual feature map.
[0023] In this embodiment, for ease of description, the visual feature modeling system will be used as the execution subject in the following description.
[0024] This application's embodiments take into account two main problems faced by traditional visual feature modeling methods (such as Convolutional Neural Networks (CNNs) and Transformers with self-attention mechanisms): First, high computational complexity, especially in high-resolution images or dense prediction tasks, where existing self-attention mechanisms have a global computational complexity of O(N). 2 This leads to a sharp increase in computational demands, making it difficult to meet real-time requirements. Secondly, high-frequency details are lost during information transmission: high-frequency details are easily smoothed out during transmission, resulting in the loss of important information such as image edges and textures, thus affecting the effectiveness of visual tasks.
[0025] Therefore, this application provides a solution that maps the image to be processed to the frequency domain, and then uses an underdamped wave equation in the frequency domain to perform independent oscillation propagation processing on each frequency component, avoiding global pairwise calculations in the spatial domain, thereby reducing the computational complexity from O(N) to O(N). 2 The computational complexity is reduced to O(N log N). At the same time, by adjusting the damping coefficient and propagation speed of the underdamped wave equation, the propagation behavior of high-frequency and low-frequency components can be differentiated, so that high-frequency detail information is fully preserved during propagation, avoiding the over-smoothing problem of traditional propagation methods, thereby improving the accuracy of visual feature modeling.
[0026] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or visual feature modeling system capable of performing the above functions. The following description uses a visual feature modeling system as an example to illustrate this embodiment and the subsequent embodiments.
[0027] Based on this, embodiments of this application provide a visual feature modeling method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the visual feature modeling method of this application.
[0028] In this embodiment, the visual feature modeling method includes steps S10 to S40: Step S10: Obtain the image to be processed; It should be noted that, in the embodiments of this application, the image to be processed refers to the original input data that needs to be modeled for visual features. The image to be processed is usually stored in digital form and is composed of a pixel array, with each pixel containing color or grayscale information.
[0029] It is understandable that the specific implementation method for acquiring the image to be processed can be flexibly selected according to different application scenarios. In one possible implementation, the system connects to a camera or image sensor through an image acquisition interface to acquire image data of the current scene in real time and perform necessary preprocessing, such as format conversion and size normalization. In another possible implementation, the system reads pre-saved image files from a storage medium, supporting common image formats such as JPEG, PNG, and BMP, and converts them into a unified internal representation through a decoding module.
[0030] Step S20: Map the image to be processed to the frequency domain; It should be noted that, in the embodiments of this application, mapping the image to be processed to the frequency domain refers to the process of converting the image from a spatial domain representation to a frequency domain representation through mathematical transformation. In the spatial domain representation, the image is described by the distribution of the grayscale or color values of pixels on a two-dimensional plane; in the frequency domain representation, the image is described by sinusoidal wave components of different frequencies and their amplitude and phase.
[0031] The purpose of mapping to the frequency domain is to separate high-frequency and low-frequency information in the image, creating conditions for subsequent frequency-time decoupling propagation based on the underdamped wave equation. Low-frequency information typically corresponds to smooth regions and the overall structure of the image, while high-frequency information corresponds to details such as edges and textures. By processing each frequency component independently in the frequency domain, targeted preservation of high-frequency details can be achieved, avoiding the loss of details caused by over-smoothing during feature propagation in traditional spatial domain methods.
[0032] It should be noted that mapping the image to be processed to the frequency domain can be achieved by performing a two-dimensional Fourier transform on the image. Specifically, the system first extracts an initial semantic field from the image to be processed. This initial semantic field can be the original pixel value matrix of the image, or it can be a preliminary feature map extracted by a shallow convolutional neural network. The initial semantic field exists in the form of a two-dimensional matrix, where each element represents a feature value at a spatial location.
[0033] Furthermore, the system applies a two-dimensional discrete Fourier transform to the initial semantic field, converting it from the spatial domain to the frequency domain. The transformed result is a complex matrix of the same size as the original image, where each element corresponds to a complex value of a specific spatial frequency component, containing the amplitude and phase information of that frequency component. This complex matrix is the set of different frequency components in the frequency domain, with each coordinate position uniquely corresponding to a specific frequency combination.
[0034] Step S30: The underdamped wave equation is used to oscillate and propagate each frequency component in the frequency domain, and the damping coefficient and propagation speed of the underdamped wave equation are adjusted during the oscillation propagation to obtain the frequency domain characteristics. It should be noted that, in the embodiments of this application, the underdamped wave equation refers to a partial differential equation describing the ability of a vibration system to oscillate even under damping. Its mathematical form includes a second-order time derivative term, a first-order damping derivative term, and a second-order spatial derivative term. Applying this equation in the frequency domain allows for independent simulation of the time evolution of each frequency component.
[0035] Oscillation propagation refers to the evolution of each frequency component over time according to the wave equation, manifested as periodic changes in amplitude and gradual decay. The damping coefficient is a parameter that controls the rate of oscillation decay; the larger the damping coefficient, the faster the amplitude decays. The propagation speed is a parameter that controls the rate of wave propagation; the greater the propagation speed, the wider the spatial range of wave propagation within the same time period.
[0036] Understandably, by adjusting the damping coefficient and propagation speed, differentiated processing of different frequency components can be achieved: a smaller damping coefficient allows high-frequency details to be preserved for a longer period, while a larger propagation speed allows low-frequency information to spread rapidly. The purpose of obtaining frequency domain features is to obtain a frequency domain representation enhanced by the wave propagation mechanism. This representation preserves high-frequency details while achieving effective propagation of global information, providing high-quality input for the subsequent inverse transform back to the spatial domain.
[0037] Specifically, the underdamped wave equation is used to propagate each frequency component in the frequency domain. This equation can physically decouple frequency and time during propagation, while adjusting the frequency attenuation and propagation speed to ensure that high-frequency information is preserved more fully.
[0038] Step S40: Map the frequency domain features to the spatial domain to obtain a visual feature map.
[0039] It should be noted that, in this embodiment, mapping frequency domain features to the spatial domain refers to the process of restoring the frequency domain representation after wave propagation processing to the spatial domain representation through inverse transformation. The frequency domain feature is a complex matrix containing the amplitude and phase information of each frequency component after oscillation propagation.
[0040] The visual feature map is the final output spatial domain feature map. Its size is the same as the original image to be processed or adjusted as needed. The feature value at each location reflects the semantic information of that location after fluctuation enhancement. The purpose of mapping frequency domain features to the spatial domain is to reintegrate the results of independent processing in the frequency domain into a spatially interpretable feature representation, so that it can be used for subsequent visual tasks such as classification, detection, and segmentation.
[0041] It is understandable that by mapping frequency domain features to the inverse transformation of the spatial domain, the preservation of high-frequency details and the propagation of low-frequency semantics during wave propagation can be reflected in the spatial domain, forming a high-quality feature map that combines rich details and semantic consistency.
[0042] This embodiment provides a visual feature modeling method. After mapping the image to be processed to the frequency domain, it uses an underdamped wave equation in the frequency domain to perform independent oscillation propagation processing on each frequency component, avoiding global pairwise calculations in the spatial domain, thereby reducing the computational complexity from O(N) to O(N). 2 The computational complexity is reduced to O(N log N). At the same time, by adjusting the damping coefficient and propagation speed of the underdamped wave equation, the propagation behavior of high-frequency and low-frequency components can be differentiated, so that high-frequency detail information is fully preserved during propagation, avoiding the over-smoothing problem of traditional propagation methods, thereby improving the accuracy of visual feature modeling.
[0043] In one feasible implementation, step S20 may include steps S21-S22: Step S21: Extract the initial semantic field from the image to be processed; It should be noted that, in the embodiments of this application, the initial semantic field refers to the basic feature representation extracted from the image to be processed to represent the image content. Its form is usually a two-dimensional matrix, and each element in the matrix corresponds to the feature value of a specific spatial location in the image.
[0044] The purpose of extracting the initial semantic field is to transform the raw pixel-level information into a semantic-level representation that is more suitable for subsequent frequency domain analysis and wave propagation, providing an input foundation for the entire visual feature modeling process. By extracting the initial semantic field, visual patterns in the image, such as edges, textures, and color distribution, can be initially captured, while filtering out some noise and redundant information, thereby improving the efficiency and accuracy of subsequent processing.
[0045] Specifically, the image to be processed is first acquired, which exists in the form of a digital matrix. In one feasible implementation, the pixel value matrix of the image to be processed is directly used as the initial semantic field. In this case, the initial semantic field is the original image itself, and the grayscale or color value of each pixel directly constitutes the feature representation.
[0046] In another implementation, the system preprocesses the image to be processed to obtain an initial semantic field. The preprocessing operations may include image size normalization, color space conversion, histogram equalization, noise reduction filtering, etc., to enhance the key features of the image and suppress interference.
[0047] Step S22: Based on the two-dimensional Fourier transform, the initial semantic field is mapped from the spatial domain to the frequency domain to obtain different frequency components located in the frequency domain.
[0048] It should be noted that, in the embodiments of this application, the two-dimensional Fourier transform refers to a mathematical tool for transforming a two-dimensional signal from the spatial domain to the frequency domain. Its essence is to decompose the signal into the superposition of sinusoidal components of different frequencies and directions.
[0049] The spatial domain refers to the natural representation of image features using pixel coordinates, where the value at each location reflects the brightness or feature intensity of that point. The frequency domain refers to the representation of image features using frequency coordinates, where the value at each point reflects the amplitude and phase of the corresponding frequency component.
[0050] Frequency components are the basic units in the frequency domain. Each component corresponds to a specific combination of spatial frequencies. Low-frequency components correspond to the smooth areas and overall structure of an image, while high-frequency components correspond to the edges, textures, and other details of the image.
[0051] The purpose of mapping the initial semantic field to the frequency domain is to separate the different frequency information mixed together in the spatial domain, so that the underdamped wave equation can be used to process each frequency component independently, achieving frequency-time decoupled oscillation propagation. Through frequency domain transformation, the system can selectively adjust the propagation behavior of different frequency components, preserving high-frequency details while ensuring the effective diffusion of low-frequency global information, thereby overcoming the defect of traditional spatial domain methods that easily lose details during feature propagation.
[0052] Specifically, the initial semantic field is usually a multi-channel two-dimensional feature map. The system first performs symmetrical padding on the feature map of each channel to reduce boundary effects. Then, it independently applies a two-dimensional fast Fourier transform to each channel to convert the pixel features in the spatial domain into complex representations in the frequency domain. The result of the transformation is a complex tensor with the same size as the input, where each coordinate position corresponds to a specific spatial frequency component. The complex value at that position contains the amplitude and phase information of that frequency component. Finally, the system stacks the frequency domain representations of all channels along the channel dimension to obtain complete frequency domain features, thereby obtaining different frequency components located in the frequency domain.
[0053] Based on the first embodiment of this application, a second embodiment of this application is proposed. In the second embodiment of this application, content that is the same as or similar to that in the first embodiment described above can be referred to the above description and will not be repeated hereafter.
[0054] Based on this, please refer to Figure 2 , Figure 2 This is a flowchart illustrating the second embodiment of this application, as shown below. Figure 2 As shown, in this embodiment, the step S30, which uses an underdamped wave equation to oscillate and propagate each frequency component in the frequency domain, and adjusts the damping coefficient and propagation speed of the underdamped wave equation during the oscillation propagation to obtain the frequency domain characteristics, may include steps S31-S32: Step S31: Calculate the propagation characteristic value of each frequency component in the frequency domain based on the underdamped wave equation to perform oscillation propagation. In the oscillation propagation, the propagation velocity and damping coefficient in the underdamped wave equation are adjusted according to the frequency component. Specifically, in the frequency domain, the underdamped wave equation is used to oscillate and propagate each frequency component, and the propagation characteristic value of each frequency component after oscillation propagation is calculated. The specific calculation formula (1) is as follows: (1) in, Represents a frequency component Propagation characteristic value after propagation time t, This represents the initial characteristics of a frequency component. Represents the initial velocity field. F represents the damping coefficient, and F represents the Fourier transform. Represents frequency components The damped oscillation frequency.
[0055] Step S32: Determine the frequency domain features based on the propagation characteristic values of each frequency component.
[0056] It should be noted that, in the embodiments of this application, the frequency domain feature refers to the complete frequency domain representation obtained by combining the propagation feature values of all frequency components according to their original positions. This frequency domain feature is a complex tensor with the same size as the input frequency domain representation, where each element corresponds to the updated value of a specific frequency component after propagation through the underdamped wave equation.
[0057] The purpose of determining the frequency domain features based on the propagation characteristic values of each frequency component is to reintegrate the independently calculated results of each frequency component into a unified frequency domain representation, providing input for the subsequent inverse Fourier transform. This integration process maintains the integrity of the frequency domain structure, ensuring that the inverse transform can correctly recover the spatial domain features, while preserving the independent control effect on each frequency component during wave propagation.
[0058] Specifically, the system maintains an empty complex tensor with the same size as the original frequency domain representation. Then, it iterates through the coordinates of each frequency component and fills the corresponding propagation feature value into that position. After filling all frequency components, the system obtains a complete complex tensor, which is the updated frequency domain feature.
[0059] In this embodiment, by introducing an underdamped wave equation to perform independent and parallel oscillation propagation calculations on each frequency component in the frequency domain, and combining the damping coefficient and propagation speed adaptively adjusted according to the frequency components, complete decoupling control of frequency and time is achieved. This mechanism can ensure the effective diffusion of low-frequency global semantic information while preserving high-frequency details (such as edges and textures), avoiding the over-smoothing problem in traditional spatial domain propagation methods, and reducing complexity through parallel computation in the frequency domain. Thus, while significantly improving the accuracy and flexibility of feature modeling, computational efficiency is greatly optimized.
[0060] In one feasible implementation, step S31 may include steps S311 to S314: Step S311: Calculate the damped oscillation frequency of each frequency component in the frequency domain based on the preset propagation speed and preset damping coefficient. Understandably, in the process of calculating the propagation characteristic value for oscillation propagation, different propagation speeds and damping coefficients can be selected according to each different frequency component.
[0061] Specifically, under underdamped conditions, the damped oscillation frequency is defined as: (2) in, Represents frequency components The damped oscillation frequency, This represents the damping coefficient.
[0062] Step S312: Calculate the cosine oscillation term and the sine oscillation term according to the damped oscillation frequency and the preset propagation time, respectively; Referring to formula (1), the cosine oscillation term is specifically expressed as: The sinusoidal oscillation term is specifically represented as: .
[0063] in, Represents frequency components The initial features are subjected to a Fourier transform, where t represents the propagation time. This indicates that a Fourier transform is performed on the initial velocity field.
[0064] Step S313: Generate an exponential decay factor based on the propagation time; Referring to formula (1), the exponential decay factor is expressed as: This can be understood as using an exponential decay factor to control the exponential decay of the overall characteristics. If the damping coefficient varies with frequency, then each frequency component corresponds to a different decay factor; if the damping coefficient is uniform globally, then all frequency components share the same decay factor value.
[0065] Step S314: Calculate the propagation characteristic value of the frequency component based on the cosine oscillation term, the sine oscillation term, the exponential decay factor, and the underdamped wave equation.
[0066] Referring to formula (1), calculate the propagation characteristic value of the frequency component according to the above formula (1). .
[0067] In one feasible implementation, step S32 may include step S321: Step S321: Fill the frequency domain with the propagation characteristic values of each frequency component according to the coordinate position of the frequency component to obtain the frequency domain features.
[0068] It should be noted that, in the embodiments of this application, the frequency domain feature refers to the complete frequency domain representation obtained by combining the propagation feature values of all frequency components according to their original positions. This frequency domain feature is a complex tensor with the same size as the input frequency domain representation, where each element corresponds to the updated value of a specific frequency component after propagation through the underdamped wave equation.
[0069] The purpose of determining the frequency domain features based on the propagation characteristic values of each frequency component is to reintegrate the independently calculated results of each frequency component into a unified frequency domain representation, providing input for the subsequent inverse Fourier transform.
[0070] Specifically, the system maintains an empty complex tensor with the same size as the original frequency domain representation. Then, it iterates through the coordinates of each frequency component and fills the corresponding propagation feature value into that position. After filling all frequency components, the system obtains a complete complex tensor, which is the updated frequency domain feature.
[0071] Based on the first and / or second embodiments of this application, a third embodiment of this application is proposed. In the third embodiment of this application, content that is the same as or similar to the first and / or second embodiments described above can be referred to the above description and will not be repeated hereafter.
[0072] In this embodiment, step S40, which maps the frequency domain features to the spatial domain to obtain a visual feature map, may include steps S41-S42: Step S41: Perform an inverse Fourier transform on the frequency domain features to obtain the feature matrix in the spatial domain; The system first acquires frequency domain features, typically a multi-channel complex tensor, with each channel corresponding to a set of frequency domain representations. The system independently performs a two-dimensional inverse Fourier transform on each channel, employing a fast Fourier transform algorithm to improve computational efficiency. The transformed result is a complex tensor with the same size as the input frequency domain features, where each element contains both real and imaginary parts. This inversely transformed complex tensor is used as the feature matrix in the spatial domain, where the real part represents the reconstructed spatial feature values.
[0073] Step S42: Determine the visual feature map based on the feature matrix.
[0074] First, the real part of the feature matrix is extracted, as the imaginary part is usually numerical noise and can be ignored. Extracting the real part yields a real tensor, which is the preliminary visual feature map. Then, the system performs necessary post-processing on this feature map according to application requirements, including but not limited to: range normalization, channel number adjustment, spatial size adjustment, and non-linear activation. The post-processed feature map is the final visual feature map, which can be input into subsequent neural network modules or used directly for task output.
[0075] In this embodiment, by performing an inverse Fourier transform on the frequency domain features enhanced by propagation through the underdamped wave equation and extracting the real part, a lossless and efficient restoration from the frequency domain processing result to the spatial domain interpretable feature map is achieved. This process not only completes the mapping of frequency domain enhancement information to the spatial domain, enabling the protection of high-frequency details and the diffusion of low-frequency semantics during wave propagation to be reflected in the spatial features, but also ensures that the output feature map has a stable numerical distribution and is compatible with the format of downstream tasks through real part extraction and post-processing operations, thereby providing high-quality feature input with both rich detail and semantic consistency for subsequent visual tasks.
[0076] Based on the above embodiments of this application, a fourth embodiment of this application is proposed. In the fourth embodiment of this application, content that is the same as or similar to the above embodiments can be referred to the above description, and will not be repeated hereafter.
[0077] Based on this, please refer to Figure 3 , Figure 3 This is a schematic flowchart of the fourth embodiment of this application, as shown below. Figure 3 As shown, after step S40, which maps the frequency domain features to the spatial domain to obtain the visual feature map, step S50 is further included: Step S50: Input the visual feature map into the pre-trained neural network model to perform the target visual task.
[0078] It should be noted that, in the embodiments of this application, the pre-trained neural network model refers to a deep neural network that has been trained on a large dataset and has the ability to process specific visual tasks. Its parameters have converged to an optimal state, and it can extract high-level semantics from input features and generate task output. The target visual task refers to a specific computer vision application, including but not limited to image classification, object detection, image segmentation, pose estimation, scene recognition, etc.
[0079] Specifically, the system first acquires a visual feature map, typically a three-dimensional tensor containing spatial height, spatial width, and the number of feature channels. This feature map is then used as input to a pre-trained neural network model. The model's structure varies depending on the target visual task: for image classification, the model typically includes global average pooling layers and fully connected layers, ultimately outputting a class probability distribution; for object detection, the model typically includes a region proposal network and a detection head, outputting the object's bounding box and class; for image segmentation, the model typically includes a decoder and a pixel-level classifier, outputting the class label for each pixel. The system performs forward propagation computation, processing the input feature map layer by layer, ultimately obtaining the task output.
[0080] Furthermore, it should be noted that the pre-trained neural network model can be combined with the embodiments of this application in various ways. In one possible implementation, the system directly replaces the input image of the original model with a visual feature map; that is, the model originally received an RGB image, but now receives a feature map generated by the embodiments of this application. In this case, the first layer of the model needs to be adjusted to match the number of channels of the feature map. In another possible implementation, the system inputs the visual feature map as an additional feature branch in parallel with the original image, and combines the two through a feature fusion module (such as concatenation or weighted summation) to enrich the feature input of the model. In yet another possible implementation, the system uses the feature extraction module of the embodiments of this application as a pre-processing module of the pre-trained model, inserting it into the shallow part of the model, for example, replacing the original stem layer or the first few convolutional layers, thereby achieving plug-and-play feature enhancement. In yet another possible implementation, the system adaptively adjusts the visual feature map for different target tasks, for example, adjusting the channel dimension through 1×1 convolution, or adjusting the spatial resolution through bilinear interpolation, to match the input requirements of the downstream model. In another possible implementation, the system adopts a multi-task learning strategy, using the same visual feature map to support multiple downstream tasks simultaneously, thereby improving computational efficiency and generalization ability by sharing feature representations.
[0081] Specifically, taking image classification tasks as an example, pre-trained neural network models typically use deep convolutional neural networks (CNNs) as their core structure. This network consists of multiple stacked layers: early convolutional layers extract low-level visual features such as edges and textures; mid-level convolutional layers combine object parts; and high-level convolutional layers generate semantically abstract feature maps. Each convolutional layer is often followed by a batch normalization layer to accelerate convergence, and the ReLU activation function is used to introduce non-linearity. The feature maps are transformed into one-dimensional feature vectors through global average pooling layers, and finally mapped to the class prediction distribution through fully connected layers.
[0082] During training, the model employs a mini-batch stochastic gradient descent strategy. Specifically, each iteration randomly samples a batch of images and their labels, performing forward propagation to calculate the cross-entropy loss. Then, backpropagation is performed via an automatic differentiation mechanism, calculating the gradient of the loss with respect to each parameter layer by layer. The optimizer (such as momentum-driven SGD or Adam) adjusts all trainable parameters accordingly, including the kernel weights of the backbone network, the batch normalization scaling factor, and user-defined physically learnable parameters (such as propagation time and damping coefficient). The training data also undergoes data augmentation operations such as random cropping and horizontal flipping to improve the model's generalization ability. After dozens of iterations, training stops when the validation set accuracy no longer improves, and the final saved model possesses powerful feature extraction capabilities. During the inference phase, the input image only requires one forward propagation to obtain the prediction result.
[0083] In this embodiment, by inputting the high-quality visual feature map generated by the embodiments of this application into the pre-trained neural network model, the seamless connection and collaborative work between the frequency domain fluctuation propagation enhancement feature and the existing mature task model are realized. This process enables the model to make full use of the high-frequency detail information and diffused global semantic information retained in the preceding steps, and significantly improve the performance of the target visual task (such as image classification, object detection, and image segmentation) without modifying the main structure of the model. At the same time, through the end-to-end training mechanism, the feature extraction module can adaptively optimize according to the task objective, and finally achieve higher task accuracy and stronger generalization ability.
[0084] For example, in order to help understand the implementation process of the visual feature modeling method in the above embodiments of this application, the visual feature modeling method described in this application is described below: Input features are converted into a frequency domain representation: First, the input image or feature map is converted into a frequency domain representation through a two-dimensional Fourier transform in order to model different frequency components. This process provides a semantic field in the frequency domain for subsequent wave propagation.
[0085] Frequency propagation is performed using the underdamped wave equation: This equation is used to propagate various frequency components in the frequency domain. It can physically decouple frequency from time during propagation, while simultaneously adjusting the frequency attenuation and propagation speed to ensure that high-frequency information is preserved more fully.
[0086] Frequency and time decoupling control mechanism: During propagation, damping terms and propagation speed are used to control the propagation rate of each frequency component, thereby ensuring the propagation of global information while avoiding the loss of high-frequency details. This process is achieved through the time decay factor in the formula, providing the ability to preserve and control high-frequency information.
[0087] Output frequency domain features and return to the spatial domain: Finally, the feature map in the frequency domain is transformed back to the spatial domain through inverse Fourier transform, and further processed by a convolutional neural network to complete the final visual feature modeling.
[0088] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the visual feature modeling method of this application. Any simple transformations based on this technical concept are within the protection scope of this application.
[0089] This application also provides a visual feature modeling device, please refer to... Figure 4 The visual feature modeling device includes: Acquisition module 10 is used to acquire the image to be processed; Mapping module 20 is used to map the image to be processed to the frequency domain; The oscillation propagation module 30 is used to perform oscillation propagation on each frequency component in the frequency domain using an underdamped wave equation, and to adjust the damping coefficient and propagation speed of the underdamped wave equation during the oscillation propagation to obtain frequency domain characteristics. The inverse mapping module 40 is used to map the frequency domain features to the spatial domain to obtain a visual feature map.
[0090] The visual feature modeling apparatus provided in this application, employing the visual feature modeling method in the above embodiments, can solve the technical problem of visual feature modeling. Compared with the prior art, the beneficial effects of the visual feature modeling apparatus provided in this application are the same as those of the visual feature modeling method provided in the above embodiments, and other technical features in the visual feature modeling apparatus are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.
[0091] This application provides a visual feature modeling device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the visual feature modeling method in Embodiment 1 above.
[0092] The following is for reference. Figure 5 The diagram illustrates a structural schematic of a visual feature modeling device suitable for implementing embodiments of this application. The visual feature modeling device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The visual feature modeling device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.
[0093] like Figure 5As shown, the visual feature modeling device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the visual feature modeling device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the visual feature modeling device to communicate wirelessly or wiredly with other devices to exchange data. While the figures show visual feature modeling devices with various systems, it should be understood that implementing or having all of the systems shown is not required. More or fewer systems may be implemented alternatively.
[0094] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.
[0095] The visual feature modeling device provided in this application, employing the visual feature modeling method in the above embodiments, can solve the technical problem of visual feature modeling. Compared with the prior art, the beneficial effects of the visual feature modeling device provided in this application are the same as those of the visual feature modeling method provided in the above embodiments, and other technical features in this visual feature modeling device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.
[0096] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.
[0097] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
[0098] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the visual feature modeling method in the above embodiments.
[0099] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.
[0100] The aforementioned computer-readable storage medium may be included in the visual feature modeling device; or it may exist independently and not be assembled into the visual feature modeling device.
[0101] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by a visual feature modeling device, the visual feature modeling device: acquires an image to be processed; maps the image to be processed to the frequency domain; uses an underdamped wave equation to oscillate and propagate each frequency component in the frequency domain, and adjusts the damping coefficient and propagation speed of the underdamped wave equation during the oscillation propagation to obtain frequency domain features; and maps the frequency domain features to the spatial domain to obtain a visual feature map.
[0102] Computer program code for performing the operations of this application can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, and C++, and conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0103] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0104] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.
[0105] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described visual feature modeling method, thereby solving the technical problem of visual feature modeling. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the visual feature modeling method provided in the above embodiments, and will not be repeated here.
[0106] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the visual feature modeling method described above.
[0107] The computer program product provided in this application can solve the technical problem of visual feature modeling. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the visual feature modeling method provided in the above embodiments, and will not be repeated here.
[0108] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.
Claims
1. A visual feature modeling method, characterized in that, The visual feature modeling method includes: Obtain the image to be processed; Map the image to be processed to the frequency domain; The frequency components in the frequency domain are oscillated and propagated using an underdamped wave equation, and the damping coefficient and propagation speed of the underdamped wave equation are adjusted during the oscillation propagation to obtain the frequency domain characteristics. The frequency domain features are mapped to the spatial domain to obtain a visual feature map.
2. The visual feature modeling method as described in claim 1, characterized in that, The step of mapping the image to be processed to the frequency domain includes: Extract the initial semantic field from the image to be processed; The initial semantic field is mapped from the spatial domain to the frequency domain based on the two-dimensional Fourier transform, resulting in different frequency components located in the frequency domain.
3. The visual feature modeling method as described in claim 1, characterized in that, The steps of using an underdamped wave equation to oscillate and propagate each frequency component in the frequency domain, and adjusting the damping coefficient and propagation speed of the underdamped wave equation during the oscillation propagation to obtain frequency domain characteristics include: The propagation characteristic value of each frequency component in the frequency domain is calculated based on the underdamped wave equation to perform oscillation propagation. In the oscillation propagation, the propagation velocity and damping coefficient in the underdamped wave equation are adjusted according to the frequency component. The frequency domain characteristics are determined based on the propagation characteristic values of each frequency component.
4. The visual feature modeling method as described in claim 3, characterized in that, The step of calculating the propagation characteristic value of each frequency component in the frequency domain based on the underdamped wave equation includes: The damped oscillation frequency of each frequency component in the frequency domain is calculated based on the preset propagation speed and the preset damping coefficient. The cosine oscillation term and the sine oscillation term are calculated based on the damped oscillation frequency and the preset propagation time, respectively. An exponential decay factor is generated based on the propagation time. The propagation characteristic value of the frequency component is calculated based on the cosine oscillation term, the sine oscillation term, the exponential decay factor, and the underdamped wave equation.
5. The visual feature modeling method as described in claim 3, characterized in that, The step of determining the frequency domain features based on the propagation characteristic values of each frequency component includes: The propagation characteristic values of each frequency component are filled into the frequency domain according to the coordinate position of the frequency component to obtain the frequency domain features.
6. The visual feature modeling method as described in claim 1, characterized in that, The step of mapping the frequency domain features to the spatial domain to obtain a visual feature map includes: Perform an inverse Fourier transform on the frequency domain features to obtain the feature matrix in the spatial domain; The visual feature map is determined based on the feature matrix.
7. The visual feature modeling method as described in claim 1, characterized in that, The step of mapping the frequency domain features to the spatial domain to obtain a visual feature map further includes: The visual feature map is input into a pre-trained neural network model to perform the target visual task.
8. A visual feature modeling device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the visual feature modeling method as described in any one of claims 1 to 7.
9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the visual feature modeling method as described in any one of claims 1 to 7.
10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the visual feature modeling method as described in any one of claims 1 to 7.