An image processing method, apparatus, device and storage medium

Abstract

The application discloses an image processing method and device, equipment and a storage medium, and relates to the technical field of computer vision. Image data is input into a feature modeling network for feature extraction to obtain a feature map. Target region detection is performed based on the feature map to determine a target position. The feature modeling network comprises a local distortion modeling branch and a global state space modeling branch that receive the same input feature. The local distortion modeling branch processes the input feature based on convolution and deformable convolution to generate a spatial offset prior field and a local enhanced feature. The global state space modeling branch processes the input feature based on a state space model and introduces the spatial offset prior field to dynamically modulate the global state space modeling process to generate a global context feature. The application can solve the problem that a traditional convolutional neural network based on a regular receptive field cannot stably extract target features for images that have distortion, stretching and other problems.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to an image processing method, apparatus, device and storage medium. Background Technology

[0002] With the rapid popularization of intelligent connected vehicles and crowdsourced map data collection, vehicle-mounted perception systems are placing higher demands on their ability to identify dynamic targets and protect privacy in road scenes. Vehicle-mounted cameras, as a crucial component of the perception system, are widely deployed around the front, rear, left, and right of vehicles. Especially with the application of wide-angle and fisheye lenses, they can achieve full coverage of pedestrians, non-motorized vehicles, and other targets outside the vehicle. Images collected while the vehicle is in motion often contain clearly identifiable facial information of pedestrians. These sensitive areas need to be detected and de-identified in a timely manner before uploading or processing to meet the compliance requirements of personal information protection regulations.

[0003] Compared to static surveillance or mobile images, in-vehicle fisheye images exhibit significant structural complexity. On one hand, fisheye lenses are prone to strong deformation in edge regions, causing stretching or compression of target shapes in the image. On the other hand, pedestrians outside the vehicle are often small targets, and facial areas are frequently affected by factors such as angular occlusion, lighting variations, and pose misalignment, making it difficult for traditional convolutional neural networks based on regular receptive fields to stably extract target features. This spatial structural uncertainty poses extremely high challenges to target detection and subsequent privacy processing. Summary of the Invention

[0004] This application provides an image processing method, apparatus, device, and storage medium to solve the problem in the related art that traditional convolutional neural networks based on rule-based receptive fields are unable to stably extract target features from images that have undergone distortion, stretching, or changes in target proportions.

[0005] In a first aspect, embodiments of this application provide an image processing method, including: Image data is input into a feature modeling network for feature extraction to obtain a feature map; Target region detection is performed based on the feature map to determine the target location; The feature modeling network includes a local distortion modeling branch and a global state space modeling branch set in parallel. The local distortion modeling branch and the global state space modeling branch receive the same input features. The local distortion modeling branch processes the input features based on convolution and deformable convolution to generate a spatial offset prior field and local enhancement features. The global state space modeling branch processes the input features based on a state space model and introduces the spatial offset prior field to dynamically modulate the global state space modeling process and generate global context features. The feature map is obtained based on the local enhancement features and the global context features.

[0006] In conjunction with the first aspect, in one implementation, the local distortion modeling branch includes an offset generation subnetwork and an adaptive sampling subnetwork; Both the offset generation subnetwork and the adaptive sampling subnetwork receive the same input features; The offset generation subnetwork includes several convolutional layers for encoding input features to generate the spatial offset prior field. The adaptive sampling subnetwork includes several deformable convolutional layers, which are used to dynamically sample the input features using the spatial offset prior field to generate the local enhanced features.

[0007] In conjunction with the first aspect, in one embodiment, the local distortion modeling branch further includes a local refinement subnetwork connected after the adaptive sampling subnetwork. The local refinement subnetwork includes several depthwise separable convolutional layers for refining the output features of the adaptive sampling subnetwork to generate the local enhancement features.

[0008] In conjunction with the first aspect, in one implementation, the global state space modeling branch includes several layers of state space models connected in sequence. Each layer of the state space model processes the input features and introduces the spatial offset prior field to dynamically modulate the global state space modeling process of that layer of state space model and generate output features.

[0009] In conjunction with the first aspect, in one implementation, image data is input into a feature modeling network for feature extraction to obtain a feature map, including: Image data is input into a convolutional coding network for feature extraction to obtain a basic feature map; The basic feature map is input into the feature modeling network for feature extraction to obtain the feature map.

[0010] In conjunction with the first aspect, in one embodiment, the convolutional coding network includes several sequentially connected convolutional layers for encoding input features to generate the basic feature map.

[0011] In conjunction with the first aspect, in one implementation, the feature modeling network includes several layers of sequentially connected modeling subnetworks; Each modeling sub-network layer includes a local distortion modeling branch and a global state space modeling branch set in parallel, as well as a feature fusion module that obtains a feature map based on the local enhancement features and global context features generated by the modeling sub-network layer. The feature modeling network further includes a downsampling layer disposed between two adjacent modeling sub-networks. The downsampling layer is used to downsample the feature map of the upper modeling sub-network in the two adjacent modeling sub-networks and use it as the input feature of the lower modeling sub-network.

[0012] In conjunction with the first aspect, in one implementation, target region detection is performed based on the feature map to determine the target location, including: A multi-scale detection head is used to detect target regions on feature maps of corresponding scales to obtain target locations and confidence scores at different scale levels. Based on confidence scores, target locations at different scale levels are aggregated to obtain the final target location.

[0013] In conjunction with the first aspect, in one implementation, target locations at different scale levels are aggregated based on confidence scores to obtain the final target location, including: The confidence score is used to filter target locations at different scale levels; The nonmaximum suppression method is used to remove redundancy from the target locations at different scales after screening in order to obtain the final target locations.

[0014] In conjunction with the first aspect, in one embodiment, the method further includes: performing reversible desensitization processing on the target location to obtain a desensitized image and key information.

[0015] Secondly, embodiments of this application provide an image processing apparatus, the image processing apparatus comprising: The feature extraction module is used to input image data into the feature modeling network for feature extraction to obtain a feature map; The target detection module is used to detect target regions based on the feature map to determine the target location; The feature modeling network includes a local distortion modeling branch and a global state space modeling branch set in parallel. The local distortion modeling branch and the global state space modeling branch receive the same input features. The local distortion modeling branch processes the input features based on convolution and deformable convolution to generate a spatial offset prior field and local enhancement features. The global state space modeling branch processes the input features based on a state space model and introduces the spatial offset prior field to dynamically modulate the global state space modeling process and generate global context features. The feature map is obtained based on the local enhancement features and the global context features.

[0016] Thirdly, embodiments of this application provide an image processing apparatus, the image processing apparatus including a processor, a memory, and an image processing program stored in the memory and executable by the processor, wherein when the image processing program is executed by the processor, it implements the steps of the image processing method as described in any of the preceding claims.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing an image processing program, wherein when the image processing program is executed by a processor, it implements the steps of the image processing method as described in any of the preceding claims.

[0018] The beneficial effects of the technical solution provided in this application include: This application provides an image processing method, apparatus, device, and storage medium. By utilizing parallel local distortion modeling branches and global state space modeling branches, this application ensures that the network can still stably extract target features even under severe distortion or scale changes, thereby significantly improving the accuracy and reliability of target region detection and effectively overcoming the limitations of traditional technologies. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic flowchart of an embodiment of the image processing method of this application; Figure 2 A schematic diagram of the feature modeling network for this application; Figure 3 This is a schematic diagram of the detection of the target area in this application; Figure 4 This is a schematic diagram of the detection head provided in an embodiment of this application; Figure 5 This is a schematic diagram of the functional modules of an embodiment of the image processing apparatus of this application; Figure 6 This is a schematic diagram of the hardware structure of the image processing device involved in the embodiments of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0023] In a first aspect, embodiments of this application provide an image processing method.

[0024] In one embodiment, reference is made to Figure 1 , Figure 1 This is a schematic flowchart of an embodiment of the image processing method of this application. Figure 1 As shown, the image processing methods include: S101: Input the image data into the feature modeling network for feature extraction to obtain the feature map.

[0025] In step S101, the feature modeling network includes a local distortion modeling branch and a global state space modeling branch set in parallel. The local distortion modeling branch and the global state space modeling branch receive the same input features. The local distortion modeling branch processes the input features based on convolution and deformable convolution to generate a spatial offset prior field and local enhancement features. The global state space modeling branch processes the input features based on the state space model and introduces the spatial offset prior field to dynamically modulate the global state space modeling process and generate global context features. The feature map is obtained based on the local enhancement features and the global context features.

[0026] In step S101, the image data can be image data acquired by a fisheye camera. Due to the wide field of view of fisheye lenses, the acquired images generally suffer from distortion, stretching, and changes in target proportions. It is understood that the aforementioned image data can also be data acquired by other types of cameras that exhibit distortion, stretching, and changes in target proportions. In other words, the feature modeling network provided in this application can process images with distortion, stretching, and changes in target proportions.

[0027] Understandably, when using a forward-facing wide-angle fisheye camera for data acquisition, the image resolution can be 1920×1080, with a frame rate of 30fps. The camera is mounted in the front of the vehicle, covering the complete field of view of the forward close-up and pedestrian areas. All acquired images undergo basic denoising and normalization preprocessing before input to improve model robustness and feature stability.

[0028] Deformable convolution, by introducing learnable convolutional offsets, can dynamically adjust the position and shape of the receptive field, thereby effectively aligning target regions with structural distortions or scale changes in the input image. This mechanism exhibits stronger deformation robustness than traditional convolution when dealing with fisheye distortion images, especially showing significant advantages in recognizing faces with non-standard poses. However, deformable convolution still falls under the category of local modeling and lacks the ability to understand the overall context of the image, making it difficult to form a stable and consistent response, particularly when processing targets that appear only briefly in consecutive frames.

[0029] Based on this, this application further introduces a state-space model. In this embodiment, the aforementioned state-space model includes, but is not limited to, the visual Mamba model.

[0030] Although the Transformer architecture is widely used in visual tasks such as object detection and image segmentation due to its powerful global modeling capabilities, and its self-attention mechanism can capture long-distance dependencies and enhance target context understanding, its computational complexity increases quadratically with the input size. Especially when processing high-resolution fisheye images, the computational and memory overhead is enormous, making it difficult to deploy in real time on automotive edge devices and limiting its practicality.

[0031] Visual Mamba models are a type of selective structured state-space model (SSM) that achieves long-range dependency modeling with linear computational complexity. Its core features are efficiently captured by tokenization, scanning, and selective state updates.

[0032] The visual Mamba model is based on a state-space model. While possessing linear computational complexity, it can still effectively establish global dependencies. Moreover, the visual Mamba model has low computational overhead, high inference efficiency, and good perception of long sequences. The visual Mamba model has been verified to be comparable to or even better than the standard Transformer in terms of modeling capabilities.

[0033] S102: Target region detection is performed based on the feature map to determine the target location.

[0034] Traditional convolutional neural networks, due to their fixed receptive fields, cannot effectively align their static sampling grids with the deformed target features when faced with image distortion, stretching, or changes in target scale. This leads to the loss of key information or interference from background noise, resulting in unstable feature extraction. This application solves this problem by constructing a parallel dual-branch coupled architecture, addressing the issue from the perspective of feature perception and modeling mechanisms.

[0035] First, the local distortion modeling branch generates a spatial offset prior field from the input features through convolutional encoding. Then, this spatial offset prior field and the input features are jointly input into a deformable convolutional layer for processing. This mechanism allows the sampling location to no longer be limited to a fixed grid, but rather to actively fit the geometry of the distorted target based on the spatial offset prior field, thereby accurately capturing local details. Because the sampling points move with the deformation, it solves the problem that a fixed receptive field cannot adapt to geometric deformation, ensuring high sensitivity to stretched or deformed targets and adapting to targets of different proportions.

[0036] Secondly, the global state space modeling branch introduces a visual Mamba model to capture long-range dependencies. Crucially, it receives the spatial offset prior field generated by the local distortion modeling branch. This coupling mechanism enables the global modeling process to perceive local geometric changes and dynamically adjust state space parameters. Because the global modeling is guided by local distortion information, it avoids misclassifying background noise as features in distorted regions, thus enhancing the robustness of global semantics.

[0037] Finally, by fusing local enhancement features with global contextual features of perceptual distortion, the generated feature map combines detail accuracy with semantic integrity.

[0038] As can be seen, this application utilizes parallel local distortion modeling branches and global state space modeling branches to ensure that the network can stably extract target features even under severe distortion or scale changes, thereby significantly improving the accuracy and reliability of target region detection. This effectively overcomes the limitations of traditional techniques, lays a solid foundation for subsequent processing, and achieves high-precision detection in complex scenes, meeting the application requirements of complex environments such as automotive. In summary, this application achieves adaptability and robustness in feature extraction through deep interaction between local and global methods, solving the problem of traditional methods struggling to stably extract target features under complex geometric transformations.

[0039] Furthermore, in one embodiment, image data is input into a feature modeling network for feature extraction to obtain a feature map, including: S201: Input the image data into the convolutional coding network for feature extraction to obtain the basic feature map.

[0040] S202: Input the basic feature map into the feature modeling network for feature extraction to obtain the feature map.

[0041] Raw image data contains a large amount of redundant texture, illumination variation, and noise information. If directly fed into the local and global modeling branches, the model will be forced to perform offset modeling and state propagation in the low-level pixel domain, making it difficult to stably extract effective information related to facial structure and contextual semantics. This embodiment uses a convolutional coding network for preprocessing to complete the representation transformation from the raw image domain to the high-dimensional feature domain. Because convolutional coding extracts basic feature representations that combine local texture, structural edges, and preliminary semantic information, it provides a stable, compact, and learnable input foundation for the alignment operation of the subsequent local distortion modeling branch and the propagation process of the global state space modeling branch. This not only reduces noise interference but also makes offset generation and state updates more focused on semantic features rather than pixel fluctuations, thereby significantly improving the stability and semantic expressiveness of feature extraction. This ensures accurate capture of target features even in complex lighting and distortion scenarios, enhancing the robustness and adaptability of the overall solution in complex in-vehicle environments.

[0042] In step S201, the convolutional coding network includes several sequentially connected convolutional layers used to encode input features to generate the basic feature map. Image data serves as the input feature of the first convolutional layer, and the input features of each subsequent convolutional layer are the output features of the layer above. The output feature of the last convolutional layer is the basic feature map.

[0043] For example, the convolutional coding network includes three convolutional layers connected in sequence, namely a 3×3 convolutional layer, a 7×7 convolutional layer, and a 3×3 convolutional layer.

[0044] Furthermore, in one embodiment, see... Figure 2 As shown, the local distortion modeling branch includes an offset generation subnetwork and an adaptive sampling subnetwork; both the offset generation subnetwork and the adaptive sampling subnetwork receive the same input features, such as... Figure 2 In this context, the basic feature map serves as the input feature for both the offset generation subnetwork and the adaptive sampling subnetwork.

[0045] The offset generation subnetwork includes several sequentially connected convolutional layers for encoding the input features to generate the spatial offset prior field; it can be understood that the base feature map ( Figure 2 The first convolutional layer uses F0 as its input feature, and each subsequent convolutional layer uses the output feature of the previous convolutional layer as its input feature. The output feature of the last convolutional layer is the spatial offset prior field. Figure 2 (denoted as Δp).

[0046] The adaptive sampling sub-network includes several deformable convolutional layers used to dynamically sample the input features using the spatial offset prior field to generate the local enhanced features. It can be understood that the base feature map ( Figure 2 (denoted as F0) and spatial offset prior fields ( Figure 2 The first deformable convolutional layer (denoted as Δp) serves as the input feature, and the input features of each subsequent deformable convolutional layer are the output features of the previous deformable convolutional layer. The output feature of the last deformable convolutional layer (Δp) is used as the input feature. Figure 2 The Chinese character is written as F. a (Also known as alignment features) can be used as local enhancement features ( Figure 2 The Chinese character is written as F. L ).

[0047] Furthermore, in one embodiment, see... Figure 2 As shown, the local distortion modeling branch further includes a local refinement subnetwork connected after the adaptive sampling subnetwork. This local refinement subnetwork comprises several depthwise separable convolutional layers, used to refine the output features of the adaptive sampling subnetwork to generate the local enhancement features. It can be understood that without the local refinement subnetwork, the output features of the last deformable convolutional layer ( Figure 2 The Chinese character is written as F. a It can be directly used as a local enhancement feature. Figure 2 The Chinese character is written as F. L To enhance the representation of small-scale facial edges, contours, and textures, a local refinement subnetwork can be added, and the output features of the last deformable convolutional layer ( Figure 2 The Chinese character is written as F. a The first depthwise separable convolutional layer serves as the input feature, and the input features of each subsequent depthwise separable convolutional layer are the output features of the previous depthwise separable convolutional layer. The output features of the last depthwise separable convolutional layer can be used as local enhancement features. Figure 2 The Chinese character is written as F. L ).

[0048] Combination Figure 2 To better understand, the local distortion modeling branch is illustrated as an example. Image data is processed by a convolutional coding network to obtain the basic feature map. , This refers to the spatial location index in the base feature map, i.e., the coordinates of each grid point. The local distortion modeling branch includes an offset generation subnetwork based on the base feature map. For each spatial location, the input features are... Both can generate spatial offset prior fields corresponding to a 3×3 sampling template (this is just an example of a 3×3 sampling template size; the actual sampling template size can be adjusted as needed). It can be understood that a 3×3 sampling template represents a local sampling template centered at the spatial location (x,y). Since a 3×3 sampling template contains 9 sampling points, therefore... In each spatial location Each outputs 9 corresponding two-dimensional displacements, which can be represented as (Equivalent to 18 output channels). Spatial offset prior field Used to drive the adaptive sampling subnetwork to process the basic feature map The sampling position is dynamically adjusted from a regular grid to a position that better fits the distorted structure, thereby achieving local geometric alignment and obtaining alignment features. The corresponding adaptive sampling subnetwork can be represented as:

[0049] in, The convolution weights for the k-th sampling point in the 3×3 sampling template are... This represents the spatial position of the k-th sampling point in the 3×3 sampling template. The standard position offset, in this case using a 3×3 sampling template, is K=9, k=1,2...,K. This represents the spatial offset prior field for the k-th sampling point in the 3×3 sampling template. Indicates spatial location The actual sampled coordinates after correction by the standard position offset and spatial offset prior field of the k-th sampling point in the 3×3 sampling template. This represents the coordinate sampling operation on the base feature map.

[0050] The offset-guided alignment sampling described above effectively compensates for the radial stretching and scaling changes caused by fisheye imaging, ensuring that subsequent local feature modeling is based on structural alignment. Furthermore, the local distortion modeling branch utilizes a local thinning subnetwork to perform local structural thinning on the aligned features, enhancing the edge, contour, and texture representation of small-scale faces to obtain enhanced local features. Therefore, the output of the local distortion modeling branch includes the spatial offset prior field output from the intermediate process. Local enhancement features of the final output Spatial offset prior fields It serves both for local alignment and for distortion-driven modulation in subsequent global modeling, while Used to provide geometrically stable local representations.

[0051] See Figure 2As shown, further, in one embodiment, the global state space modeling branch includes several layers of sequentially connected state space models. Each layer of the state space model processes the input features and introduces the spatial offset prior field to dynamically modulate the global state space modeling process of that layer's state space model, generating output features. It can be understood that the basic feature map ( Figure 2 The first layer of the state space model (denoted as F0) serves as the input feature. The input features of each subsequent layer are the output features of the previous layer. The output features of the last layer are the global context features. Figure 2 The Chinese character is written as F. G ).

[0052] Combination Figure 2 As shown, to better understand, the global state space modeling branch is illustrated exemplarily. This branch uses a state space modeling structure to perform long-range dependency propagation on features, capturing the global context with linear complexity and suppressing complex background interference. Image data is processed by a convolutional coding network to obtain the basic feature map. . The basic feature map Expanded into a sequence according to the preset scanning order , For state representation, For output, its update process can be represented as:

[0053] in , Here, C represents the state transition and input projection parameters, respectively, and C represents the state projection parameters. To enable explicit global propagation of the spatial differences in fisheye distortion, this application differs from traditional state update methods that use fixed parameters. In this embodiment... , The spatial migration prior field output from the local distortion modeling branch is dynamically modulated. Specifically, the spatial migration prior field corresponding to sequence position t is... Recorded as ,but , The calculation is as follows:

[0054]

[0055] in, , It is a lightweight mapping function (linear projection) used to map the spatial offset prior field into parameter increments, so that the state update exhibits different propagation preferences in different distortion regions: in areas with significant distortion, it strengthens the propagation of local continuity and structural consistency to avoid cross-regional mis-aggregation caused by distortion; in areas with weak distortion, it enhances the cross-regional semantic aggregation capability to improve the separation of face regions from complex backgrounds.

[0056] The output of the global state space modeling branch is defined as the global context feature. It is used to characterize global semantic information after long-range dependency propagation and background suppression.

[0057] Finally, local enhancement features With global context features Feature maps are obtained after linear projection fusion. ( Figure 2 The representation is denoted as F, forming a unified representation that combines geometric stability and semantic separation, providing a structurally consistent input basis for subsequent detection and desensitization.

[0058] As can be seen, the feature modeling network of this application adopts a parallel dual-branch coupled structure, consisting of a local distortion modeling branch and a global state space modeling branch, and achieves cross-branch parameter-level collaboration through a distortion-driven modulation mechanism. This design addresses the typical contradictions under vehicle-mounted fisheye imaging conditions: on the one hand, pedestrian faces in images exhibit significant radial geometric stretching, proportional changes, and edge distortion, leading to local structural misalignment; on the other hand, complex road backgrounds and distant small targets bring long-range interference, making it difficult to obtain stable semantic separation by relying solely on local convolution. To this end, this application decouples "local geometric adaptation" and "global context modeling" into parallel branches, and achieves collaborative enhancement through structural coupling, improving the reliability of subsequent detection and desensitization from the source.

[0059] Furthermore, in one embodiment, the feature modeling network includes several layers of modeling sub-networks connected in sequence; each layer of the modeling sub-network includes a local distortion modeling branch and a global state space modeling branch set in parallel, as well as a feature fusion module that obtains a feature map based on the local enhancement features and global context features generated by the modeling sub-network of that layer. The feature modeling network further includes a downsampling layer disposed between two adjacent modeling sub-networks. The downsampling layer is used to downsample the feature map of the upper modeling sub-network in the two adjacent modeling sub-networks and use it as the input feature of the lower modeling sub-network.

[0060] Understandably, the basic feature map ( Figure 2The first layer of the modeling subnetwork (denoted as F0) serves as the input feature. The input features of each subsequent layer of the modeling subnetwork are the output features of the previous layer of the modeling subnetwork after being processed by the downsampling layer. Since the downsampling process has been performed, the output features of each layer of the modeling subnetwork, i.e., the feature map, have different scales. Therefore, the feature maps of each layer of the modeling subnetwork constitute a multi-scale feature map.

[0061] This embodiment constructs a hierarchical multi-scale feature extraction architecture by connecting adjacent modeling sub-networks through downsampling layers. Since each layer integrates local distortion correction and global semantic modeling in parallel, the network can balance geometric alignment and context awareness at different scales. As the layers deepen, the downsampling operation gradually abstracts semantics, preserving both subtle textures and understanding macroscopic layouts, effectively adapting to the varying sizes of pedestrians in vehicular scenes. This avoids missed detections at a single scale, ensuring stable output of high-precision detection results under complex distortions and enhancing the model's generalization ability to multi-scale targets.

[0062] See Figure 3 As shown, further, in one embodiment, target region detection based on the feature map to determine the target location includes: S301: Employs a multi-scale detection head to detect target regions on feature maps of corresponding scales to obtain target locations and confidence scores at different scale levels.

[0063] S302: Based on confidence scores, aggregate target locations at different scale levels to obtain the final target location.

[0064] This embodiment utilizes multi-scale feature maps generated by a hierarchical modeling sub-network, combined with a multi-scale detection head for parallel detection. Because different scale feature maps are adept at capturing target features of varying sizes, they can comprehensively cover pedestrians at different distances in vehicular scenarios, effectively avoiding missed detections at a single scale. Subsequently, results from different scales are aggregated based on confidence scores, eliminating redundant detection boxes across scales and retaining only the most reliable target locations. This mechanism not only adapts to complex environments with drastically changing target scales but also improves detection accuracy by selecting high-confidence results, ensuring the accuracy of subsequent privacy processing and significantly enhancing the system's robustness and reliability in dynamic scenarios.

[0065] It is understandable that the aforementioned target detection can be aimed at specific objects that need to be detected, such as faces, animals, bridges, or other sensitive information such as military units or government agency locations. The specific target can be determined according to the detection needs.

[0066] See Figure 3 As shown, further, in one embodiment, target locations at different scale levels are aggregated based on confidence scores to obtain the final target location, including: S401: Use the confidence score to filter target locations at different scale levels.

[0067] S402: Using the nonmaximum suppression method, redundancy is eliminated from the target locations at different scale levels after screening to obtain the final target location.

[0068] This embodiment eliminates low-quality false positives through confidence screening, ensuring target reliability; furthermore, non-maximum suppression eliminates cross-scale redundancy, avoiding duplicate detections. This makes the target location unique and accurate, preventing conflicts in privacy protection processing and significantly improving detection accuracy and system robustness, meeting the stability requirements of automotive scenarios.

[0069] To make it easier to understand, an example will be used below.

[0070] Suppose there are 3 people in an image. At a certain scale, there might be 10 bounding boxes generated at the locations of these 3 people (e.g., 4 overlapping boxes near the 1st person, 3 near the 2nd person, and 3 near the 3rd person), representing the target locations. Each bounding box corresponds to a confidence score, representing how likely the model considers "this to be a face." A pre-set confidence threshold is used; bounding boxes with confidence scores below this threshold are discarded, and only bounding boxes with confidence scores at or above the confidence threshold are retained.

[0071] After the above screening, although the remaining detection boxes all have high confidence, many may still overlap on the same face. The purpose of non-maximum suppression is to remove duplicates, retaining only the best-performing detection box. Specifically: Sorting: Sort all remaining detection boxes from highest to lowest confidence score.

[0072] Select the highest: Take the detection box with the highest confidence score (let's say Box A) and keep it.

[0073] Calculate overlap: Calculate the overlap between Box A and all other remaining detection boxes.

[0074] Suppression: If the overlap between a detection box and Box A exceeds the set overlap threshold (e.g., 0.5), it means they refer to the same target, and the overlapping box is deleted.

[0075] Loop: From the remaining boxes, select the one with the highest confidence score (let's say Box B), and repeat the above until all detection boxes have been processed.

[0076] In this application, see Figure 4As shown, the detection head preferably adopts a lightweight structure of "shared feature transformation + branch prediction": first, the feature map is further integrated and denoised through a shared convolutional transformation module, and then the prediction results are output into the classification prediction branch and the regression prediction branch respectively. The classification prediction branch is used to output the confidence score. The regression prediction branch is used to output the prediction box offset parameters and form the final prediction box based on the reference box.

[0077] in, Indicates the prediction box. The reference box refers to the initial candidate box preset at the corresponding feature position of the detection head, which is used to provide the initial position and size benchmark for the regression of the predicted box; This represents the predicted bounding box offset parameter output by the detection head, used to describe the geometric adjustment of the predicted bounding box relative to the reference box. These represent the horizontal and vertical offsets of the predicted bounding box center point relative to the reference bounding box center point, respectively. and These represent the scaling adjustments of the predicted bounding box relative to the reference bounding box in the width and height directions, respectively. "+" indicates regression correction based on the reference bounding box; as an example, this could be a direct addition element-by-element.

[0078] This structure can effectively improve the detection head's ability to perceive targets of different scales, and enhance the accuracy and robustness of detection of distant, small-sized, and pose-unstable face regions.

[0079] Furthermore, in one embodiment, the method further includes: performing reversible desensitization processing on the target location to obtain a desensitized image and key information.

[0080] The reversible desensitization process can employ any of the configurable paths such as perturbation mapping, frequency domain embedding, and image replacement to generate desensitized images and key information.

[0081] The anonymized images and key information are output in a structured manner and uploaded to the backend monitoring system or cloud platform for identity verification and subsequent recovery scheduling.

[0082] When the legal authorization conditions are met, the desensitized areas of the desensitized image are restored based on the recorded key information to reconstruct the original facial image and achieve controllable recovery of privacy information.

[0083] For example, in perturbation mapping encryption, the desensitization function is defined as follows:

[0084] in, This represents the original encrypted area, such as the face. For disturbance parameters, This is the key control function. Represents the desensitization function. This is the desensitized result, i.e., the desensitized image. In authorized scenarios, this can be achieved through an inverse function. Perform restoration:

[0085] It is evident that key information includes at least perturbation parameters, key control functions, desensitization functions, and inverse functions.

[0086] Taking facial recognition as an example, after facial recognition is completed, this application integrates a reversible desensitization mechanism that supports multiple technical paths. Without relying on fixed rules or image post-processing, it performs encryption perturbation or camouflage processing on the detected facial regions, while retaining necessary information for subsequent recovery under authorized conditions. This strategy is not limited to a specific implementation method and supports multiple scalable paths, including perturbation mapping, frequency domain embedding, and image replacement, to meet the application needs of different privacy protection and data availability scenarios. Through the synergistic integration of spatial adaptability and global modeling mechanisms at the architectural level, the current method achieves efficient detection and controllable desensitization of complex pedestrian facial targets in fisheye images while maintaining the overall model's lightweight nature. It is suitable for typical scenarios with strict privacy compliance requirements, such as autonomous driving visual acquisition and crowdsourced map data preprocessing.

[0087] The feature modeling network and multi-scale detection head in this application are trained using supervised learning. The training data can be obtained from road scene images collected by vehicle-mounted camera systems, preferably including pedestrian facial targets under fisheye or wide-angle imaging conditions, and covering different lighting, weather, road environment, target scale, pose changes, occlusion, and obvious distortion at the edge of the field of view. Before training, the original images can be preprocessed with distortion correction, scale normalization, cropping, and data augmentation, and the pedestrian facial regions in the images can be labeled with bounding boxes and categories. During training, the corrected input image is convolutionally encoded to obtain a basic feature map, which is then input into the local distortion modeling branch and the global state space modeling branch for joint feature modeling. The outputs of the two branches are fused and input into the detection head to output the facial bounding box position and confidence. The network parameters are iteratively optimized based on the bounding box regression loss, target classification loss, and optional offset constraint loss, thereby obtaining a training model that can be used for subsequent face detection and privacy desensitization processing.

[0088] Secondly, embodiments of this application also provide an image processing apparatus.

[0089] In one embodiment, reference is made to Figure 5 , Figure 5 This is a schematic diagram of the functional modules of an embodiment of the image processing apparatus of this application. Figure 5As shown, the image processing apparatus includes: The feature extraction module is used to input image data into the feature modeling network for feature extraction to obtain a feature map; The target detection module is used to detect target regions based on the feature map to determine the target location; The feature modeling network includes a local distortion modeling branch and a global state space modeling branch set in parallel. The local distortion modeling branch and the global state space modeling branch receive the same input features. The local distortion modeling branch processes the input features based on convolution and deformable convolution to generate a spatial offset prior field and local enhancement features. The global state space modeling branch processes the input features based on a state space model and introduces the spatial offset prior field to dynamically modulate the global state space modeling process and generate global context features. The feature map is obtained based on the local enhancement features and the global context features.

[0090] Furthermore, in one embodiment, the feature extraction module is used to input image data into a feature modeling network for feature extraction to obtain a feature map, including: Image data is input into a convolutional coding network for feature extraction to obtain a basic feature map; The basic feature map is input into the feature modeling network for feature extraction to obtain the feature map.

[0091] Furthermore, in one embodiment, the target detection module is used to perform target region detection based on the feature map to determine the target location, including: A multi-scale detection head is used to detect target regions on feature maps of corresponding scales to obtain target locations and confidence scores at different scale levels. Based on confidence scores, target locations at different scale levels are aggregated to obtain the final target location.

[0092] Furthermore, in one embodiment, the target detection module is used to aggregate target locations at different scale levels based on confidence scores to obtain the final target location, including: The confidence score is used to filter target locations at different scale levels; The nonmaximum suppression method is used to remove redundancy from the target locations at different scales after screening in order to obtain the final target locations.

[0093] Furthermore, in one embodiment, the image processing apparatus further includes a reversible desensitization module, which is used to perform reversible desensitization processing on the target location to obtain a desensitized image and key information.

[0094] The functions of each module in the above-mentioned image processing device correspond to the steps in the above-mentioned image processing method embodiments, and their functions and implementation processes will not be described in detail here.

[0095] Thirdly, embodiments of this application provide an image processing device, which may be a personal computer (PC), a laptop computer, a server, or other device with data processing capabilities.

[0096] Reference Figure 6 , Figure 6 This is a schematic diagram of the hardware structure of the image processing device involved in the embodiments of this application. In the embodiments of this application, the image processing device may include a processor, a memory, a communication interface, and a communication bus.

[0097] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0098] Communication interfaces include input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting devices within the image processing equipment, as well as interfaces used for interconnecting the image processing equipment with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.

[0099] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0100] The processor can be a general-purpose processor, which can call the image processing program stored in the memory and execute the image processing method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the image processing program is called can be referred to in the various embodiments of the image processing method of this application, and will not be repeated here.

[0101] Those skilled in the art will understand that Figure 6 The hardware structure shown does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0102] Fourthly, embodiments of this application also provide a computer-readable storage medium.

[0103] The present application has a computer-readable storage medium storing an image processing program, wherein when the image processing program is executed by a processor, it implements the steps of the image processing method described above.

[0104] The method implemented when the image processing program is executed can be referred to in various embodiments of the image processing method of this application, and will not be repeated here.

[0105] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0106] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0107] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0108] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0109] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0110] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0111] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An image processing method, characterized in that, include: Image data is input into a feature modeling network for feature extraction to obtain a feature map; Target region detection is performed based on the feature map to determine the target location; The feature modeling network includes a parallel local distortion modeling branch and a global state space modeling branch, both of which receive the same input features. The local distortion modeling branch includes an offset generation sub-network and an adaptive sampling sub-network, both receiving the same input features. The offset generation sub-network includes several convolutional layers for encoding the input features to generate a spatial offset prior field. The adaptive sampling sub-network includes several deformable convolutional layers for dynamically sampling the input features using the spatial offset prior field to generate locally enhanced features. The global state space modeling branch processes the input features based on a state space model and introduces the spatial offset prior field to dynamically modulate the global state space modeling process, generating global context features. The feature map is obtained based on the locally enhanced features and the global context features.

2. The image processing method as described in claim 1, characterized in that, The local distortion modeling branch also includes a local refinement subnetwork connected after the adaptive sampling subnetwork. The local refinement subnetwork includes several depthwise separable convolutional layers, which are used to refine the output features of the adaptive sampling subnetwork to generate the local enhancement features.

3. The image processing method as described in claim 1, characterized in that, The global state space modeling branch includes several layers of state space models connected in sequence. Each layer of the state space model processes the input features and introduces the spatial offset prior field to dynamically modulate the global state space modeling process of that layer of state space model and generate output features.

4. The image processing method as described in claim 1, characterized in that, Image data is input into a feature modeling network for feature extraction to obtain feature maps, including: Image data is input into a convolutional coding network for feature extraction to obtain a basic feature map; The basic feature map is input into the feature modeling network for feature extraction to obtain the feature map.

5. The image processing method as described in claim 4, characterized in that, The convolutional coding network comprises several sequentially connected convolutional layers, used to encode input features to generate the basic feature map.

6. The image processing method as described in claim 1, characterized in that, The feature modeling network includes several layers of modeling subnetworks connected in sequence; Each modeling sub-network layer includes a local distortion modeling branch and a global state space modeling branch set in parallel, as well as a feature fusion module that obtains a feature map based on the local enhancement features and global context features generated by the modeling sub-network layer. The feature modeling network further includes a downsampling layer disposed between two adjacent modeling sub-networks. The downsampling layer is used to downsample the feature map of the upper modeling sub-network in the two adjacent modeling sub-networks and use it as the input feature of the lower modeling sub-network.

7. The image processing method as described in claim 6, characterized in that, Target region detection based on the feature map to determine the target location includes: A multi-scale detection head is used to detect target regions on feature maps of corresponding scales to obtain target locations and confidence scores at different scale levels. Based on confidence scores, target locations at different scale levels are aggregated to obtain the final target location.

8. The image processing method as described in claim 7, characterized in that, Based on confidence scores, target locations at different scale levels are aggregated to obtain the final target locations, including: The confidence score is used to filter target locations at different scale levels; The nonmaximum suppression method is used to remove redundancy from the target locations at different scales after screening in order to obtain the final target locations.

9. The image processing method as described in claim 1, characterized in that, The method further includes: performing reversible desensitization processing on the target location to obtain a desensitized image and key information.

10. An image processing apparatus, characterized in that, The image processing device includes: The feature extraction module is used to input image data into the feature modeling network for feature extraction to obtain feature maps; The target detection module is used to detect target regions based on the feature map to determine the target location; The feature modeling network includes a parallel local distortion modeling branch and a global state space modeling branch, both of which receive the same input features. The local distortion modeling branch includes an offset generation sub-network and an adaptive sampling sub-network, both receiving the same input features. The offset generation sub-network includes several convolutional layers for encoding the input features to generate a spatial offset prior field. The adaptive sampling sub-network includes several deformable convolutional layers for dynamically sampling the input features using the spatial offset prior field to generate locally enhanced features. The global state space modeling branch processes the input features based on a state space model and introduces the spatial offset prior field to dynamically modulate the global state space modeling process, generating global context features. The feature map is obtained based on the locally enhanced features and the global context features.

11. An image processing device, characterized in that, The image processing device includes a processor, a memory, and an image processing program stored in the memory and executable by the processor, wherein when the image processing program is executed by the processor, it implements the steps of the image processing method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an image processing program, wherein when the image processing program is executed by a processor, it implements the steps of the image processing method as described in any one of claims 1 to 9.

Images