A method for processing a partially damaged image based on STP-CP

Abstract

This invention discloses a method for processing partially damaged images based on STP-CP, belonging to the field of image processing technology. This invention combines matrix semi-tensor product (STP) with convolution operations to construct a cross-dimensional inner product operation to replace the traditional convolution inner product. It directly performs convolution calculations on the effective pixels of the damaged image, eliminating the need for zero-padding or interpolation to fill missing regions. A mask is used to distinguish between effective and invalid pixels to complete the convolution calculation. By defining a cross-dimensional inner product, this invention enables direct operation between receptive fields of different dimensions and the convolution kernel, effectively avoiding false edges, false textures, and feature distortions introduced by padding operations. It preserves the true texture features of the image, improving the accuracy and robustness of partially damaged images in tasks such as feature extraction, target recognition, and image restoration. This method is computationally stable and simple to implement, and can be widely applied in scenarios such as autonomous driving visual perception, remote sensing images, medical images, surveillance footage, and the transmission of damaged images.

Description

Technical Field

[0001] This invention relates to the field of image processing, specifically a method for processing partially damaged images based on STP-CP. Background Technology

[0002] In actual image acquisition, transmission, and storage, images often suffer from partial pixel loss or invalidity due to occlusion, noise, data loss, or local damage, resulting in partially damaged images. These images lack a complete and regular pixel structure, making them unsuitable for direct convolution operations. Current technologies that most closely address partially damaged images generally employ zero-padding to process the missing areas, forcibly normalizing the image into a regular matrix before performing convolutional feature extraction. This approach has significant drawbacks: zero-padding artificially introduces invalid black pixels, disrupting the original grayscale distribution and local texture continuity, leading to unexpected distortion of overall features and failing to reflect the true information of the image; zero values ​​at the missing edges form false edges and textures with surrounding valid pixels, causing the convolution kernel to incorrectly extract interfering features and resulting in severe distortion of normal features in the neighborhood; simultaneously, it cannot distinguish between real black pixels and padded zero values, easily leading to feature confusion. Furthermore, simple zero padding does not utilize prior image structure for reasonable estimation, and missing information is directly discarded. Moreover, the feature error introduced by shallow padding will be passed down layer by layer in the network and accumulate continuously, which will significantly reduce the accuracy and robustness of subsequent feature extraction, image restoration and target recognition tasks. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention aims to overcome the deficiencies of existing zero-padding-based convolution techniques when processing partially damaged images, and proposes a partially damaged image processing method based on STP-CP. This invention eliminates the need for zero-padding or interpolation to fill pixel gaps in damaged areas. It uses a mask to filter effective pixels, bypasses mask propagation, and directly skips damaged areas, performing convolution calculations only on effective pixels. This preserves original effective features, avoids artifacts and errors, and achieves full and efficient utilization of effective information. Furthermore, it replaces traditional convolution inner product operations with cross-dimensional inner products constructed using matrix semi-tensor products, allowing convolution to directly apply to effective pixels without zero-padding or explicit mask propagation in missing areas. This fundamentally avoids false edges, false textures, and feature distortion, preserving the true texture and feature distribution of the image. This significantly improves the feature extraction accuracy, robustness, and overall processing effect of damaged images, meeting the practical needs of robust visual processing in scenarios such as autonomous driving, remote sensing images, medical images, surveillance footage, and the transmission of damaged images.

[0004] To solve the aforementioned technical problem, the present invention adopts the following technical solution: a partially damaged image processing method based on STP-CP. This method, for a given partially damaged image matrix and convolution kernel matrix, retains the original effective pixels of the partially damaged image without performing filling or padding on the damaged area; using the size of the convolution kernel matrix as the sliding window size, it slides on the partially damaged image matrix with preset horizontal and vertical steps to extract the data of the effective area within the sliding window, and finally obtains the effective pixel area matrix. The effective pixel region matrix is ​​vectorized by stacking columns to obtain the effective region vector. At the same time, the convolution kernel matrix is ​​vectorized by stacking columns to obtain the convolution kernel vector. Perform a cross-dimensional inner product operation between the effective region vector and the convolution kernel vector to obtain the convolution output value at the current window position. After traversing all window positions, the complete convolution feature map is finally obtained. The cross-dimensional inner product operation includes: for vectors of different dimensions, first expanding the vectors to the same dimension by using the least common multiple, then performing the inner product operation and normalizing.

[0005] Furthermore, the formula for the cross-dimensional inner product operation is as follows: , where x and y are vectors to be used for the cross-dimensional inner product. This represents a cross-dimensional inner product of vectors x and y. The general inner product, where m is the dimension of vector x, n is the dimension of vector y, and t is the least common multiple of m and n. Indicates the Kronecker product. , It is a length of , A column vector of all 1s.

[0006] Furthermore, the vectorization process involves converting the matrix into a vector by stacking columns.

[0007] Furthermore, the partially damaged image matrix identifies the valid region and the damaged region through a mask matrix. Only the valid region participates in the convolution operation, with 1 representing the valid region and 0 representing the damaged region. When the window slides, the data of the valid region within the sliding window is extracted based on the mask matrix, and finally the valid pixel region matrix is ​​obtained.

[0008] Furthermore, when the window slides to the image boundary, the boundary area is only used for schematic illustration and does not participate in the calculation, and no zero-fill operation is performed on the boundary area.

[0009] Furthermore, this method can be used for autonomous driving, remote sensing images, medical images, surveillance footage, and transmission of damaged images.

[0010] The beneficial effects of this invention are as follows: Compared with existing traditional convolution techniques that employ zero-padding, interpolation completion, or masking, the technical solution described in this invention has the following beneficial effects: 1. Eliminate false pixels and false edge interference at the source to improve feature quality.

[0011] Existing technologies require zero-filling of missing image regions, which artificially introduces invalid pixels, creating false edges and textures at damaged boundaries, leading to distorted feature extraction. This invention eliminates the need for any filling of missing regions, performing calculations directly based on valid pixels. This completely avoids feature interference from false pixels, preserving the original image texture and true feature distribution, significantly improving the accuracy and reliability of feature extraction.

[0012] 2. Avoid the accumulation of errors layer by layer and improve the robustness of the algorithm.

[0013] Traditional padding methods introduce feature errors that propagate and amplify layer by layer in the network, reducing the stability of subsequent recognition, detection, and repair tasks. This invention, however, does not rely on padding or mask propagation, eliminating the error introduction process at its source. Even under complex degradation conditions such as local missing features, occlusion, and transmission corruption, it can still stably output effective features, significantly improving the robustness of image processing.

[0014] 3. Simplify the calculation process, improve processing efficiency, and reduce computational overhead.

[0015] Existing technologies require additional preprocessing steps such as padding, mask generation, and invalid value removal, increasing the computational complexity and number of steps. This invention directly extracts the effective region and performs cross-dimensional inner product operations, eliminating multiple preprocessing steps. The computational flow is simpler, the computational load is lower, and it effectively improves image processing speed and overall operating efficiency.

[0016] 4. It has a wider range of applications and is more adaptable to damaged images.

[0017] Traditional convolution is only suitable for regular, complete images, and its effect on images with local damage or irregular regions is poor. This invention can directly process various degraded images such as those with local missing parts, occlusions, and transmission corruption, and is insensitive to the form, location, and size of the damage. Its versatility and practicality are significantly better than existing methods.

[0018] 5. More suitable for real-time processing of autonomous driving and in-vehicle edge devices.

[0019] In autonomous driving visual perception scenarios, this invention can maintain stable output under conditions such as dirty lenses, rain obstruction, and image transmission packet loss, thereby improving the accuracy of target detection and image recognition. At the same time, due to its lightweight operation and lower processing time, it can meet the real-time processing needs of platforms with limited computing power, such as vehicle terminals and embedded devices, making it more practical. Attached Figure Description

[0020] Figure 1 This is a flowchart of the method; Figure 2 This is a schematic diagram of effective region extraction and window sliding in the embodiment. Detailed Implementation

[0021] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0022] Example 1 This embodiment discloses a method for processing partially damaged images based on STP-CP, such as... Figure 1 As shown, it includes the following steps: S01. Obtain the partially damaged image matrix Set the convolution kernel matrix For partially damaged image matrices No need to fill in missing positions; keep the original valid data unchanged and select the horizontal step size. and vertical step size Using the size of the convolution kernel matrix as the window size, in partially damaged image matrices... Slide the window upwards with given vertical and horizontal steps, extract the data of the effective region within the window, and form a matrix V according to the region. V is the effective pixel region matrix (the process is as follows). Figure 2 As shown in the diagram, to facilitate visualization of the image boundary region, the layout space required for window sliding is preserved in the schematic, represented as... , This space is for display purposes only, does not participate in calculations, and is not filled with zero values.

[0023] S02. Effective region vectorization: Arrange the elements of each effective region in matrix V column-wise to obtain the matrix. , It is a region vector arranged in columns.

[0024] S03. Calculate the convolution value using cross-dimensional inner product, and then convert the region vector... With the vectorized convolution kernel matrix Substituting into the cross-dimensional inner product formula defined earlier... Perform calculations to obtain the convolution value at the current window position. By iterating through all windows, a complete convolutional feature map is finally obtained.

[0025] The formula for calculating the cross-dimensional inner product is: , where x and y are vectors to be used for the cross-dimensional inner product. This represents a cross-dimensional inner product of vectors x and y. The general inner product, where m is the dimension of vector x, n is the dimension of vector y, and t is the least common multiple of m and n. Indicates the Kronecker product. , It is a length of , A column vector of all 1s.

[0026] The cross-dimensional vector operations defined in this embodiment construct a mathematical framework that can uniformly process vectors of different dimensions, ensuring the consistency and stability of the calculation results of addition, subtraction, inner product, and distance between vectors of different dimensions. It provides a reliable mathematical foundation for unfilled convolution and irregular data processing, and the operation results have good convergence and reproducibility.

[0027] This technical solution eliminates the need for zero-filling, avoiding interference from false pixels and pseudo-edges. This operation, based directly on effective pixels, preserves the true texture and feature distribution. It is suitable for images with partial defects, occlusions, or transmission corruption. Errors do not accumulate with the network, significantly improving the robustness of feature extraction, recognition, and repair. In practical applications, it is suitable for real-time processing in autonomous driving and in-vehicle edge devices.

[0028] Example 2 This embodiment discloses a method for processing partially damaged images based on STP-CP, and the specific implementation steps are as follows: 1. Autonomous vehicles acquire road images captured by onboard cameras. These images may be partially damaged due to rain, dirty lenses, light interference, or other reasons. 2. For the damaged images mentioned above, no zero-filling or masking is performed; the valid pixel information is directly preserved on the original image. 3. Using the given size of the convolution kernel matrix as the window size, slide the window according to the set horizontal and vertical strides, and extract the effective pixel regions in the image sequentially according to the window. 4. Stack the extracted valid pixel regions column by column to obtain the corresponding region vector; 5. Substitute the above-mentioned region vector and column-stacked convolution kernel matrix into the cross-dimensional inner product defined in this invention to perform the operation and obtain the convolution output value at the current window position; 6. By traversing all the sliding windows in the above manner, a complete convolutional feature map is finally obtained, thus completing the feature extraction of the partially damaged image.

[0029] When applied to autonomous driving visual perception scenarios, this invention addresses the issue of partially damaged images by using no zero-padding or interpolation for feature extraction. Instead, it extracts convolutional features solely from the valid pixels in the image, effectively avoiding false edges and textures caused by padding data. This preserves the true information of the image and significantly improves the accuracy and robustness of target detection and image recognition in autonomous driving scenarios.

[0030] The above description is merely the basic principle and preferred embodiment of the present invention. Improvements and substitutions made by those skilled in the art based on the present invention are within the scope of protection of the present invention.

Claims

1. A method for processing partially damaged images based on STP-CP, characterized in that: This method, given a partially damaged image matrix and a convolution kernel matrix, retains the original valid pixels of the partially damaged image without padding or completing the damaged areas; it uses the size of the convolution kernel matrix as the sliding window size and slides it on the partially damaged image matrix with preset horizontal and vertical steps to extract the data of the valid areas within the sliding window, and finally obtains the valid pixel area matrix. The effective pixel region matrix is ​​vectorized by stacking columns to obtain the effective region vector. At the same time, the convolution kernel matrix is ​​vectorized by stacking columns to obtain the convolution kernel vector. Perform a cross-dimensional inner product operation between the effective region vector and the convolution kernel vector to obtain the convolution output value at the current window position. After traversing all window positions, the complete convolution feature map is finally obtained. The cross-dimensional inner product operation includes: for vectors of different dimensions, first expanding the vectors to the same dimension by using the least common multiple, then performing the inner product operation and normalizing.

2. The partially damaged image processing method based on STP-CP according to claim 1, characterized in that: The formula for the cross-dimensional inner product operation is as follows: , where x and y are vectors to be used for the cross-dimensional inner product. This represents a cross-dimensional inner product of vectors x and y. The general inner product, where m is the dimension of vector x, n is the dimension of vector y, and t is the least common multiple of m and n. Indicates the Kronecker product. , It is a length of , A column vector of all 1s.

3. The partially damaged image processing method based on STP-CP according to claim 1, characterized in that: The vectorization process converts the matrix into a vector by stacking columns.

4. The partially damaged image processing method based on STP-CP according to claim 1, characterized in that: The partially damaged image matrix identifies the valid region and the damaged region through a mask matrix. Only the valid region participates in the convolution operation, with 1 indicating the valid region and 0 indicating the damaged region. When the window slides, the data of the valid region within the sliding window is extracted based on the mask matrix, and finally the valid pixel region matrix is ​​obtained.

5. The method for processing partially damaged images based on STP-CP according to claim 1, characterized in that: When the window slides to the image boundary, the boundary area is only used for illustration and does not participate in the calculation. No zero-fill operation is performed on the boundary area.

6. The method for processing partially damaged images based on STP-CP according to claim 1, characterized in that: This method can be used for autonomous driving, remote sensing images, medical images, surveillance footage, and transmission of damaged images.

Images