Sampling method, reconstruction method, communication node and storage medium

Abstract

The invention provides a sampling method, a reconstruction method, a communication node and a storage medium. The sampling method comprises the following steps: sampling to-be-sampled data at a first sampling rate; reconstructing a measured value obtained after sampling to obtain an initial image; determining the information amount of a plurality of image blocks in the initial image; allocating a second sampling rate based on the information amount of each image block, and determining an actual sampling parameter corresponding to each image block; and for each image block, sampling an area corresponding to the image block in the to-be-sampled data based on the actual sampling parameter of the image block to obtain a sampling result.

Description

Technical Field

[0001] The present application relates to the field of communication technology, for example, to a sampling method, a reconstruction method, a communication node and a storage medium. Background Art

[0002] Compressed sensing (CS), also known as sparse sampling, is an emerging signal processing technology. CS achieves efficient compression by sparsely representing and randomly projecting signals. Compared with the Nyquist sampling theorem, compressed sensing can effectively reconstruct compressed signals when the signals are sparse in certain transform domains, and can restore the original signals after compression. It has broad application prospects in data transmission, image compression, speech processing and other fields.

[0003] When current compressed sensing methods sample images, the information content in the sampled images is limited, and important information in the images is sampled inefficiently, which in turn potentially reduces the quality of image reconstruction. Summary of the invention

[0004] The present application provides a resource selection method, a communication node and a storage medium.

[0005] In a first aspect, the present application provides a sampling method, comprising:

[0006] Sampling the data to be sampled at a first sampling rate;

[0007] Reconstruct the measured values ​​obtained after sampling to obtain an initial image;

[0008] Determining the information amount of multiple image blocks in the initial image;

[0009] Allocating the second sampling rate based on the amount of information of each of the image blocks, and determining actual sampling parameters corresponding to each of the image blocks;

[0010] For each image block, a region corresponding to the image block in the data to be sampled is sampled based on actual sampling parameters of the image block to obtain a sampling result.

[0011] In a second aspect, the present application provides a reconstruction method, comprising:

[0012] Obtaining an initial reconstruction result of a sampling result, wherein the sampling result is a result obtained by sampling according to any one of the methods of claims 1 to 6;

[0013] The initial reconstruction result is reconstructed through a U-shaped network that crosses the connection to obtain a reconstructed result.

[0014] In a third aspect, the present application provides a communication node, including:

[0015] one or more processors;

[0016] A storage device for storing one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the embodiments of the present application.

[0018] In a fourth aspect, the present application provides a storage medium, which stores a computer program, and when the computer program is executed by a processor, the method provided in the embodiment of the present application is implemented.

[0019] With regard to the above embodiments and other aspects of the present application and their implementation, further description is provided in the accompanying drawings, detailed description and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of a sampling method provided in an embodiment of the present application;

[0021] Figure 2 A flowchart of a reconstruction method provided in an embodiment of the present application;

[0022] Figure 3a A schematic diagram of a network structure of a compressed sensing model provided in an embodiment of the present application;

[0023] Figure 3b A schematic diagram of a depth reconstruction framework based on residual blocks provided in an embodiment of the present application;

[0024] Figure 3c A schematic diagram of the network structure of a UNet network based on a residual block and a Swin Transformer block provided in an embodiment of the present application;

[0025] Figure 3d A network structure diagram of STCSNet provided in an embodiment of the present application;

[0026] Figure 4 A schematic diagram of the structure of a sampling device provided in an embodiment of the present application;

[0027] Figure 5 A schematic diagram of the structure of a reconstruction device provided in an embodiment of the present application;

[0028] Figure 6 A schematic diagram of the structure of a communication node provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present application more clear, the embodiments of the present application will be described in detail below with reference to the accompanying drawings. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other arbitrarily without conflict.

[0030] The steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions. Also, although a logical sequence is shown in the flowchart, in some cases, the steps shown or described can be performed in a sequence different from that shown here.

[0031] In the present application, the terms "first", "second", etc. are used to distinguish similar objects and are not necessarily used to describe a specific sequence or order.

[0032] Compressed sensing (CS), also known as sparse sampling, is an emerging signal processing technology. CS achieves efficient compression by sparsely representing and randomly projecting signals. Compared with the Nyquist sampling theorem, compressed sensing can effectively reconstruct compressed signals when the signals are sparse in certain transform domains, and can restore the original signals after compression. It has broad application prospects in data transmission, image compression, speech processing and other fields.

[0033] The process of compressed sensing processing problem can be roughly divided into sampling stage and reconstruction stage. The sampling stage is used to obtain measurement values, and the reconstruction stage is used to reconstruct the original signal from the obtained measurement values.

[0034] Compressed sensing algorithms are divided into two categories: traditional algorithms and deep learning algorithms. Traditional algorithms have problems of slow speed and poor reconstruction quality; deep learning algorithms are divided into two categories: non-iterative uninterpretable and iterative interpretable algorithms. Non-iterative algorithms usually use a deep learning network for reconstruction and are not interpretable; iterative algorithms usually expand traditional compressed sensing algorithms into deep learning networks for iteration and are interpretable.

[0035] The sampling matrix contains part of the information of the original signal, which can guide the compressed sensing reconstruction process. The non-iterative algorithm usually does not use the information of the sampling matrix in the reconstruction stage, and can achieve good results at low sampling rates. However, as the sampling rate increases, the importance of the sampling matrix becomes greater, and the performance of the non-iterative algorithm will deteriorate. The iterative algorithm usually expands the traditional compressed sensing algorithm into the deep learning network, and uses the information of the sampling matrix, so the iterative algorithm generally has better results, but the speed is relatively slower. That is, the interpretable algorithm generally has better results.

[0036] However, these algorithms ignore the different information contents in different regions of the image. Using uniform sampling to compress the image cannot contain the maximum information of the image, which will lead to oversampling of unimportant signals and inefficient sampling of important signals. Therefore, this application proposes an image information detection algorithm based on traditional methods, which allocates different numbers of measurement values ​​(i.e., different sampling rates) according to the information content of different regional blocks. In this way, better reconstruction effects can be obtained when the total number of measurement values ​​is constant.

[0037] The original image of the compressed sensing problem is not visible, so it is not advisable to process it directly on the original image. Some compressed sensing networks use deep learning algorithms to detect the amount of image information, but there is a gradient truncation problem, which makes the detection effect unsatisfactory. Based on this, this application guides the sampling rate allocation by using branches to obtain partial measurement values ​​for initial reconstruction. The proximal gradient descent algorithm based on the U-Net network is used for reconstruction.

[0038] Image information and video information have become the most common ways for people to obtain information. Therefore, how to better obtain and store image and video information has become a popular research direction. For the sampling of images and videos, data is first obtained according to Nyquist sampling, then the obtained data is compressed, and finally the compressed data is stored or transmitted. This method causes a great degree of resource waste. The compressed sensing theory can use random sampling to obtain discrete samples of the signal under the condition of far less than the Nyquist sampling rate, and then reconstruct the signal through a nonlinear reconstruction algorithm.

[0039] The non-adaptive CS method uses the same sampling rate for all blocks in the image. However, since the meaningful information in the image is not evenly distributed, such an operation will potentially reduce the reconstruction quality of the image; while the adaptive CS can adaptively allocate the sampling rate according to the content characteristics of the image, thereby improving the image reconstruction quality. Adaptive CS has the following disadvantages: 1. Its sampling matrix is ​​related to the sampling rate. One sampling matrix can only handle one sampling rate and cannot be applied to adaptive CS; 2. The calculation of image sampling rate allocation requires the original image, but in actual applications, only the measurement value can be obtained at the sampling end. Therefore, this application designs a dual-branch sampling structure at the sampling end to guide the allocation of sampling rates; 3. The reconstruction algorithm is mostly a black box operation mode, which is not interpretable and does not consider the dependency between the reconstruction features. Therefore, this application proposes a compressed sensing model based on the proximal gradient descent algorithm.

[0040] This patent uses an image information detection algorithm (also known as an image block information detection algorithm) to obtain image prior information for sampling rate allocation, uses an interpretable network for reconstruction, and implements it by expanding the proximal gradient mapping iterative algorithm (also known as the proximal gradient descent algorithm), where the proximal mapping is implemented using a residual block-based U-Net network. Leading results were achieved through a relatively lightweight network. The model is scalable and can train any sampling rate in the sampling rate range. The sampling rate range trained in this application is [0.1%, 100%].

[0041] This application can significantly reduce data transmission and storage costs while maintaining high quality by sparsely representing and compressing data. It can be applied to the following application areas:

[0042] Video surveillance and video streaming: By adopting compressed sensing technology, the bandwidth requirements of video streaming can be greatly reduced, thus saving a lot of network bandwidth and storage space. This is very valuable for video surveillance systems and real-time video transmission services.

[0043] Medical images and scans: Medical images and scan data usually have a lot of redundant information. By adopting compressed sensing deep learning algorithms, data transmission and storage requirements can be reduced while maintaining sufficient image quality. This is very critical for application scenarios such as remote diagnosis, medical image archiving and sharing.

[0044] Wireless communications and mobile applications: Compressed sensing deep learning algorithms have potential advantages in wireless communications, which can reduce transmission power consumption and bandwidth requirements. This is very valuable for mobile applications and wireless network optimization, such as video streaming, social media, and real-time gaming.

[0045] Data storage and cloud computing: With the popularity of cloud computing, processing and storing large-scale data has become increasingly important. The use of compressed sensing deep learning algorithms can effectively reduce data storage and transmission costs and improve the efficiency and scalability of cloud computing.

[0046] In an exemplary embodiment, Figure 1 A flow chart of a sampling method provided in an embodiment of the present application. The method can be applicable to the case of sampling the sampled data, the method can be performed by a sampling device, the sampling device can be implemented by software and / or hardware, and the sampling device can be integrated on a communication node. The communication node can be any device capable of implementing sampling.

[0047] like Figure 1 As shown, the sampling method provided by this application includes the following steps:

[0048] S110 , sampling the data to be sampled at a first sampling rate.

[0049] The data to be sampled can be understood as data to be sampled. The form of the data to be sampled is not limited here. For example, it can be an image or a measurement value of an image. The measurement value can be a value obtained after sampling the image and is not limited here. The number of measurement values ​​can be considered as the number of measurement values.

[0050] The data to be sampled may be sampled at a target sampling rate. The target sampling rate may be understood as the total sampling rate of the data to be sampled. In order to solve the technical problem of inefficient sampling of important information in an image, the present application may adopt two branches for sampling, one branch may sample the data to be sampled at a first sampling rate, and the other branch may sample the data to be sampled at a second sampling rate.

[0051] The values ​​of the first sampling rate and the second sampling rate can be determined based on the target sampling rate, such as the sum of the first sampling rate and the second sampling rate can be the target sampling rate, or can be different from the target sampling rate, such as less than the target sampling rate or greater than the target sampling rate. The values ​​of the first sampling rate and the second sampling rate are not limited, and the first sampling rate can be determined first, and then the second sampling rate is determined based on the target sampling rate and the first sampling rate.

[0052] In this step, the data to be sampled may be sampled at a first sampling rate to obtain a measurement value for reconstructing the initial image, and then the second sampling rate of the second branch may be allocated based on the amount of information in the initial image.

[0053] In this step, the data to be sampled can be sampled in blocks to obtain the measurement value of each block.

[0054] S120, reconstructing the measured values ​​obtained after sampling to obtain an initial image.

[0055] The initial image may be an image after initial reconstruction. After obtaining the sampled measurement values, this step may reconstruct the measurement values ​​to obtain the initial image, and the reconstruction means are not limited here. After obtaining the measurement values ​​of each block through the above steps, this step may reconstruct the measurement values ​​of each block to obtain the initial image of the block, which is used to analyze the amount of information of each block, and then allocate a sampling rate or the number of measurement values ​​to each block.

[0056] S130: Determine the information amount of multiple image blocks in the initial image.

[0057] The amount of information may refer to the content of information. After the initial image is determined, this step may determine the corresponding amount of information for each image block in the initial image. The method for determining the amount of information is not limited here, for example, the amount of information of each block may be determined based on the initial image and the corresponding image after mean filtering.

[0058] S140: Allocate the second sampling rate based on the amount of information of each of the image blocks, and determine actual sampling parameters corresponding to each of the image blocks.

[0059] The actual sampling parameters can be considered as the sampling parameters used in the process of sampling the data to be sampled. The actual sampling parameters can be the sampling rate or the number of measured values.

[0060] After determining the amount of information of each image block, this step can determine the sampling rate of the image block, or the number of measurement values, based on the amount of information of the image block. The sum of the number of measurement values ​​obtained after sampling each image block according to the determined sampling rate is equal to the number of measurement values ​​obtained after sampling the entire image according to the second sampling rate.

[0061] For example, the target sampling rate of an image is 25%, the first sampling rate is 20%, and the second sampling rate is 5%. After the second sampling rate is assigned to each image block in the image, the sampling rate of each image block may be 3%, 7%, etc., but the sum of the number of measurement values ​​obtained according to the sampling rates of different image blocks is equal to the sum of the measurement values ​​obtained at the second sampling rate of 5% of the entire image, or within a certain error range.

[0062] This step does not limit how to allocate the second sampling rate based on the amount of information, and the amount of information of the image block and the sampling rate may be proportional. The greater the amount of information, the greater the allocated sampling rate.

[0063] In this step, when allocating the second sampling rate, each image block may determine its own actual sampling parameter based on the information volume of each image block, or may determine its own actual sampling parameter based on the information volume allocation of each image block.

[0064] In one embodiment, each of the n image blocks determines its own actual sampling parameter based on its own information amount.

[0065] In one embodiment, after each image block in the n image blocks determines its own actual sampling parameter based on its own information amount, each actual sampling parameter may be fine-tuned by combining the second sampling rate and the actual sampling parameter of each image block.

[0066] In one embodiment, each of the n image blocks may be assigned a second sampling rate based on the corresponding amount of information to obtain respective actual sampling parameters. In this embodiment, when assigning the second sampling rate, the amount of information of each image block may be comprehensively considered.

[0067] S150 . For each image block, sample a region of the data to be sampled corresponding to the image block based on actual sampling parameters of the image block to obtain a sampling result.

[0068] The sampling result can be considered as the result obtained after sampling the data to be sampled. For example, the sampling result can be a measurement value obtained after sampling, or it can be an image obtained after sampling. The sampling result and the data to be sampled can be in the same form. For example, when the data to be sampled is a measurement value, the sampling result is a measurement value actually obtained by sampling the data to be sampled. The actual sampling can be considered as sampling from the data to be sampled and used as the sampling result. In this embodiment, sampling at the first sampling rate can be considered as reconstructing the initial image, thereby guiding the allocation of the sampling rate.

[0069] The area corresponding to the image block in the data to be sampled can be considered as the area corresponding to the image block in the initial image in the data to be sampled. The data to be sampled and the initial image can be different representations of data, and each image block in the initial image has corresponding data in the data to be sampled. After determining the amount of information in the image block, it can be used to guide the sampling of the measurement value of the corresponding area in the data to be sampled, thereby improving the sampling efficiency.

[0070] The sampling method provided in the present application adopts dual branches to sample the data to be sampled. One branch adopts a first sampling rate to sample the data to be sampled to obtain an initial image, and then allocates a second sampling rate to the second branch based on the amount of information of the initial image, so that the sampling result contains a high content of important information of the image, thereby realizing efficient sampling of important information in the image.

[0071] Based on the above embodiment, a variant embodiment of the above embodiment is proposed. It should be noted that in order to make the description concise, only the differences from the above embodiment are described in the variant embodiment.

[0072] In one embodiment, sampling the data to be sampled at a first sampling rate includes:

[0073] determining a first sampling rate;

[0074] The data to be sampled is sampled at the first sampling rate.

[0075] In this embodiment, when sampling the data to be sampled, a first sampling rate may be determined first, and then the data to be sampled may be sampled based on the first sampling rate to obtain a sampled measurement value.

[0076] The means for determining the first sampling rate is not limited and may be determined based on one or more of a target sampling rate, a target number of measurement values, and a size of an image block.

[0077] The target number of measured values ​​can be considered as the number of measured values ​​of the data to be sampled. The size of the image block can be considered as the size of each image block in the initial image.

[0078] In one embodiment, determining the first sampling rate includes:

[0079] The first sampling rate is determined based on the size of the image block in the matrix block sampling process, the target sampling rate of the data to be sampled, and the target number of measurement values ​​of the data to be sampled.

[0080] In this embodiment, the first sampling rate may be determined by the following formula:

[0081] B×B×ratio×γ; γ=round(0.8650-q / 4096). B can be used to determine the size of the image. ratio is the target sampling rate, and q is the target number of measurements.

[0082] In one embodiment, determining the amount of information of the plurality of image blocks in the initial image includes:

[0083] Performing mean filtering on the initial image to obtain a filtered image;

[0084] Determining information difference data between the initial image and the filtered image;

[0085] For each image block, the information amount of the image block is determined based on the information difference data of the image block.

[0086] The information difference data can be considered as data obtained by comparing the initial image and the filtered image. In this embodiment, the initial image and the filtered image can be subtracted by pixel and then the absolute value can be taken, and the information difference data can be determined based on the absolute value. In the determination process, the amount of information can be controlled by the control variable u.

[0087] In this embodiment, the corresponding information difference data may be determined for each image block. Then, for each image block, the sum of the information difference data of each image block is determined, and the determined sum value is determined as the corresponding information amount.

[0088] In one embodiment, taking image block a as an example, image block a can be mean filtered to obtain a filtered image, and then information difference data is determined based on image block a and the filtered image. For image block a, the sum of the corresponding information difference data is determined as the information amount of image block a.

[0089] In one embodiment, the actual sampling parameters include one or more of an actual sampling rate and an actual number of measurement values.

[0090] The actual sampling rate may be considered as the actual sampling rate, such as the sampling rate obtained by allocating the second sampling rate. The actual number of measured values ​​may be the actual number of measured values, such as the number of measured values ​​determined after allocating the second sampling rate.

[0091] In one embodiment, the sum of the first sampling rate and the second sampling rate is equal to the target sampling rate of the data to be sampled.

[0092] In the process of implementing dual-branch sampling, this embodiment meets the requirement of target sampling rate for the data to be sampled.

[0093] In one embodiment, reconstructing the measured values ​​obtained after sampling to obtain the initial image includes:

[0094] The measured values ​​after sampling are reconstructed through the sampling matrix to obtain the initial image.

[0095] In this embodiment, the initial image can be reconstructed based on the sampling matrix. Reconstruct the initial image. Among them, is the sampling matrix, and y is the measured value obtained after sampling.

[0096] In an exemplary embodiment, the present application also provides a reconstruction method, Figure 2 A flowchart of a reconstruction method provided in an embodiment of the present application. The reconstruction method is applicable to the case where the sampling result after sampling is reconstructed. The reconstruction method can be executed by a reconstruction device, and the reconstruction device can be integrated on a communication node. The communication node of the integrated sampling device and the communication node of the integrated reconstruction device can be the same node or different nodes, which is not limited here. The contents that are not yet detailed in this embodiment can be referred to the above embodiments, which will not be repeated here.

[0097] like Figure 2 As shown, the reconstruction method provided by the present application comprises the following steps:

[0098] S210: Obtain an initial reconstruction result of the sampling result.

[0099] In this embodiment, the sampling result may be a result after sampling, such as a result after sampling using the sampling method provided in this application.

[0100] The initial reconstruction result can be considered as the result obtained by performing initial reconstruction on the sampling result. The initial reconstruction is not limited here, and the initial reconstruction result can include an image and features extracted from the image.

[0101] S220 , reconstructing the initial reconstruction result through a U-shaped network that spans the connection to obtain a reconstructed result.

[0102] A cross connection can be understood as a cross-layer connection. The network structure of a U-shaped network can be U-shaped, and corresponding layers in the U-shaped network can be cross-connected. An exemplary U-shaped network can include three downsamplings and three upsamplings, the first downsampling can be cross-connected with the third upsampling, the second downsampling can be cross-connected with the second upsampling, and the third downsampling can be cross-connected with the first upsampling.

[0103] The reconstruction method provided in the present application first reconstructs the sampling result to obtain an initial reconstruction result, and then reconstructs the initial reconstruction result through a U-shaped network to obtain a reconstruction result, thereby realizing the reconstruction of the sampling result.

[0104] Based on the above embodiment, a variant embodiment of the above embodiment is proposed. It should be noted that in order to make the description concise, only the differences from the above embodiment are described in the variant embodiment.

[0105] In one embodiment, the initial reconstruction result is reconstructed by a U-shaped network across the connection to obtain a reconstructed result, including:

[0106] Iteratively performing gradient mapping processing on the initial reconstruction result to obtain a reconstructed result;

[0107] The gradient mapping process includes: gradient descent processing and mapping processing through a U-type network that spans the connection.

[0108] In this embodiment, the initial reconstruction result is iteratively processed through the U-shaped network with cross-connections, such as performing gradient mapping processing multiple times.

[0109] In the process of performing gradient mapping processing, gradient descent processing is performed first, and then mapping processing is performed through the U-shaped network across the connection.

[0110] The gradient descent process can be considered as performing gradient descent. The mapping process performed by the U-shaped network across the connection can be considered as performing proximal mapping.

[0111] In one embodiment, the mapping process is performed by spanning a connected U-shaped network, including:

[0112] The data processed by gradient descent and the initial reconstruction result are mapped by a U-shaped network based on residual blocks and cross connections.

[0113] In this embodiment, the U-type network is a network based on residual blocks and cross-connections. In this embodiment, the data processed by gradient descent can be converted into an image form and then input into the U-type network for mapping processing.

[0114] In one embodiment, the mapping process is performed by spanning a connected U-shaped network, including:

[0115] The data processed by gradient descent and the initial reconstruction result are mapped by a residual block, a moving window-based hierarchical visual self-attention model block (i.e., a SwinTransformer block) and a cross-connected U-shaped network.

[0116] The U-type network of this embodiment is a network based on residual blocks, Swin Transformer blocks and cross connections.

[0117] In this embodiment, the data after the gradient descent processing can be converted into an image form and then input into the U-type network for mapping processing.

[0118] In this embodiment, the Swin Transformer block can be in the layer where the U-type network performs downsampling and upsampling.

[0119] For example, the Swin Transformer block is used in the third downsampling and first upsampling layers.

[0120] In one embodiment, reconstructing the initial reconstruction result through the U-shaped network connected across to obtain the reconstructed result includes:

[0121] The initial reconstruction result is mapped through a moving window-based hierarchical visual self-attention model block and a U-shaped network with cross-connections to obtain a reconstructed result.

[0122] In this embodiment, the U-type network is a network based on Swin Transformer blocks and spanning connections.

[0123] In this embodiment, the data after the gradient descent processing can be converted into an image form and then input into the U-type network for mapping processing.

[0124] Swin Transformer blocks can be distributed in the layers of the U-shaped network for downsampling and upsampling. For example, use Swin Transformer blocks in the third downsampling and first upsampling layers.

[0125] In one embodiment, the gradient descent process includes:

[0126] Vectorize the initial reconstruction results;

[0127] The initial reconstruction result in vector form is subjected to gradient descent based on the sampling matrix and the set step size.

[0128] When performing the gradient descent process in this embodiment, the image in the initial reconstruction result may be converted into a vector form, that is, vectorized, and then the initial reconstruction result in the vector form may be subjected to gradient descent.

[0129] In this embodiment, the gradient descent process can be performed as follows x←x-ρφ T (φx-y'), where x is the initial reconstruction result in vector form, is the sampling matrix, ρ is the step size, and y' is the measurement value obtained by adaptive sampling based on the target sampling rate.

[0130] The present application is described exemplarily below. The sampling method and reconstruction method provided by the present application can be considered as a type of image compression sensing based on adaptive sampling rate allocation.

[0131] Figure 3a A schematic diagram of a network structure of a compressed sensing model provided in an embodiment of the present application is shown in FIG. Figure 3a , the present application provides a compressed sensing deep learning network IQCSNet for image information detection, which expands the proximal gradient descent algorithm into deep learning and has interpretability. The network is divided into three stages: sampling stage, initial reconstruction stage and deep reconstruction stage. In adaptive sampling, firstly, a fixed number of measurement values ​​1 are obtained by matrix block sampling to reconstruct an initial image (i.e., the sampled data is sampled at a first sampling rate; the measurement values ​​obtained after sampling are reconstructed to obtain the initial image), and then the image information is distributed using a mean filtering-based method (i.e., the amount of information of multiple image blocks in the initial image is determined) to obtain the sampling result 2; the reconstruction uses the proximal gradient descent algorithm, firstly performs gradient descent (GD), and then uses the lighter and more effective U-Net network framework based on residual blocks and cross connections to implement proximal mapping (PM), which has achieved good results.

[0132] Sampling stage: The sampling stage is performed by matrix block sampling. The matrix is ​​obtained from the training set through singular value decomposition (SVD), and the sampling matrix is ​​learnable. The model first obtains a certain fixed sampling value (i.e., the first sampling rate) to obtain image information (such as the measurement value obtained by sampling), and then allocates the sampling rate based on this, i.e., allocates the second sampling rate.

[0133] Since the original image is generally invisible when compressed sensing is processing problems, such as single-pixel imaging, the model must first obtain a certain number of basic measurement values ​​(such as measurement values ​​obtained based on the first sampling rate) to obtain an initial image to guide the sampling rate allocation. Here, the parameter γ is used to allocate the required fixed sampling value, that is, the first sampling rate.

[0134] γ=round(0.8650-q / 4096) (1)

[0135] The fixed sampling value can be calculated by the following formula (i.e., the first sampling rate is determined based on the size of the image block in the matrix block sampling process, the target sampling rate of the data to be sampled, and the target number of measurement values ​​of the data to be sampled): B×B×ratio×γ, where q is the target number of measurement values ​​(i.e., the number of measurement values ​​required for sampling), and q can be a given value, determined by a given block size B and a target sampling rate, such as q=B×B×target sampling rate. ratio is the target sampling rate, and B×B can indicate the size of the image block. 0.865 is a hyperparameter that can be determined by experience or experiment. 4096 can be considered as a parameter for adjusting the sampling rate distribution, which can be determined according to the size of the image block, or it can be an arbitrary value. The values ​​of 0.865 and 4096 are not limited here.

[0136] After the first sampling rate is determined, a simple initial reconstruction can be performed on the obtained measurement values ​​based on sampling at the first sampling rate.

[0137] In the present application, the measurement data obtained can be adaptively sampled to obtain the measurement value to complete the initial reconstruction.

[0138] The parameter γ is used to balance the quality of the initial reconstructed image and the effect of the sampling rate allocation. Therefore, the image information metric detection is implemented by Algorithm 1, i.e., the image block information detection algorithm. This algorithm is used to detect the information content of each block of the image as the basis for the measurement value quantity allocation (corresponding to the sampling rate allocation). The input of the image block information detection algorithm is a part of the fixed measurement value y (i.e., the measurement value obtained by sampling at the first sampling rate) and the sampling matrix The output is the information amount S of each image block. The image block information amount detection algorithm performs mean filtering on the initial image to obtain a filtered image; determines the information difference data between the initial image and the filtered image; for each image block, the specific operation of determining the information amount of the image block based on the information difference data of the image block is as follows:

[0139] Perform a simple initial reconstruction of the image Perform mean filtering on the image to determine the information difference data (i.e., INFO_DIFF = abs(X0-AVG_X0) × uu is a parameter that controls the amount of information), and S←sums INFO_DIFF block by block.

[0140] Among them, the initial reconstruction can be based on Determine the reconstructed image X0. AVG_X0 can be considered as the image after mean filtering, and the information difference data can be obtained by taking the absolute value of the difference between the reconstructed image and the image after mean filtering and multiplying it by u. After obtaining the information difference data of each block, the sum of the information difference data of each image block can be used as the information amount of the image block.

[0141] In the present disclosure, the sampling matrix can be obtained from the data set in the training phase and is known data.

[0142] After determining the amount of information of each image block, the remaining sampling rate, i.e., the second sampling rate, can be allocated by Algorithm 2, i.e., the measurement value allocation algorithm. The measurement value allocation algorithm inputs the amount of information S of each image block, the total measurement value data q allocated to all blocks, and the maximum number of measurement values ​​K that can be allocated to each image block (the maximum number of measurement values ​​cannot exceed the number of measurement values ​​of the full sampling of the corresponding block), and outputs the measurement value data result Q allocated according to the amount of information of each block.

[0143] Algorithm 2,ensures that blocks with more information content are assigned more,measurement values, and blocks with less information content are assigned fewer,measurement values.,This application directly uses the amount of information as a,measurement value reference, and then compresses the amount of information into the,desired target measurement value range, rather than allocating it proportionally.

[0144] The total number of measurements assigned to all blocks is the target number of measurements mentioned above. The maximum number of measurements cannot exceed the number of measurements contained in each image block to avoid repeated sampling after exceeding the limit.

[0145] The measurement value allocation algorithm can be qualitatively allocated, and the specific allocation is as follows:

[0146] Assume i is 0, T is 10, the initial value of Q is the information volume S of the corresponding image block, execute i+1, Q←round(clip(Q,0,K)), that is, the number of measured values ​​is corrected to an integer between 0 and K, and an error is generated. The error Δ=sum(Q)-q, and the error is determined to be equal to 0. If so, it can be considered that the total number of measured values ​​meets the conditions and exit the loop. If not, determine whether i is less than or equal to T. If so, the maximum number of iterations has not been exceeded, and the average number of measured values ​​is reduced, that is, Q←Q-Δ / LL is the number of blocks. If i exceeds T, it is considered that the maximum number of iterations has been exceeded, and the number of random measured values ​​is reduced. D←sample from the multinomial distribution, a total of Δ samples are taken, and each block is drawn from L blocks. The probability of each block being drawn is 1 / L, that is, Δ is randomly distributed to L blocks with a probability of 1 / L. ← can be understood as obtaining or equal to, which is determined based on the actual scene. Q←Q-sign(Δ)×D.

[0147] The number of measurements is allocated through the tested image. The block size is 32×32, the sampling rate is 0.25, and the average number of measurements per block is 256 when sampling in blocks. 256=32*32*0.25, that is, the number of measurements is determined by the block size and the sampling rate. Here, 0.25 is the total sampling rate (i.e., the target sampling rate), and the total sampling rate is the sum of the first sampling rate and the second sampling rate. The adaptive measurement value allocation results show that the number of measurements allocated in the image blocks with large information is large, such as the interval of the number of measurements can be [205,250), [250,300), [300,400), [400,1024). The qualitative allocation algorithm not only satisfies the requirement of keeping the total number of measurements unchanged, but also satisfies the requirement of allocating more measurements to blocks with more information such as texture, and vice versa, it is in line with the allocation idea. It can be seen from the sampling rate size diagram obtained by measurement that the number of measurements of image blocks with large information is larger. The obtained measurement value distribution result Q is used to calculate the sampling rate, and the sampling rate distribution image (i.e., the sampling result) is obtained. Q represents the number of measurement values ​​in different blocks. Q i Indicates the number of measurements assigned to the current block (i-th block), A is the sampling matrix, which is equivalent to the sampling matrix Indicates intercepting q from the sampling matrix i Rows are used to sample image blocks.

[0148] Q is the measurement value distribution result. Q divided by the total number of measurement values ​​of the corresponding block is the sampling rate. The sampling rate can be used to obtain the sampling rate distribution image. That is, intercept q from the sampling matrix i line, thus completing the adaptive sampling.

[0149] So the sampling process can be described as

[0150]

[0151] Indicates intercepting q from the sampling matrix i The rows are used to map the image block x i Sampling is performed. yi is the sampling result, and xi is the data to be sampled.

[0152] Initial reconstruction phase: The initial reconstruction is also processed block by block. The process can be described as

[0153]

[0154] In this stage, the sampling rate obtained at the sampling end is distributed to the image extraction features to obtain Q_feature.

[0155] The reconstruction phase of the U-Net network based on residual blocks and cross connections, that is, the deep reconstruction phase based on proximal gradient descent: The deep reconstruction is iteratively carried out through the proximal gradient mapping algorithm, also known as the proximal gradient descent algorithm (PGD). Each iteration is divided into two stages. The first stage is a trivial gradient descent step:

[0156]

[0157] The second stage is proximal mapping.

[0158]

[0159] Figure 3b A schematic diagram of a depth reconstruction framework based on residual blocks provided in an embodiment of the present application, Figure 3b is a stage of the unfolded PGD network, x = x-ρ×(Φ T Φx-Φ T y) is to perform gradient descent, reshape and fold (i.e., reshape and fold) after the descent, and then perform proximal mapping. Proximal mapping is followed by reshaping and unfolding. Figure 3b The first downsampling layer, i.e., the layer corresponding to DOWN1, and the third upsampling layer, i.e., the layer corresponding to UP1, are cross-connected.

[0160] Algorithm 3 is the proximal gradient descent algorithm. The input image X (the number of channels is 1), the feature Q_feature (the number of channels is 3) extracted from the sampling rate distribution map, and the sampling matrix Output the image reconstructed in UNet (i.e., U-shaped network), i.e., the reconstructed result. Since the sampling matrix is ​​known at the reconstruction end, the compressed sensing framework in this application is end-to-end, and the sampling matrix can be transmitted from the sampling end to the reconstruction end.

[0161] The initial reconstruction result may include an image X (ie, an initial image X) and a feature Q_feature extracted from a sampling rate distribution map.

[0162] The specific contents of the proximal gradient descent algorithm are as follows:

[0163] Step 1: Set i to 0 and the maximum number of iterations max_iterations to 7.

[0164] Step 2: Determine whether i ≤ max_iterations.

[0165] Step 3: If yes, convert the image X into vector form (i.e. x←reshape_and_unfold(X)), that is, vectorize the initial reconstruction result.

[0166] Step 4: Then perform gradient descent on the transformed vector based on the sampling matrix and step size, that is That is, the initial reconstruction result in vector form is subjected to gradient descent based on the sampling matrix and the set step size.

[0167] Step 5: Convert the result after gradient descent into image form, that is, X←reshape_and_fold(x).

[0168] Step 6: Convolve the converted image and feature Q_feature to obtain Head, that is, X_head←Head(X+Q_feature).

[0169] Then do the following:

[0170] X_down1←Conv(RESBlock(X)), X_down2←Conv(RESBlock(X_down1)), X_down3←Conv(RESBlock(X_down2)), anspose(X_body+X_down3)), X_up2←RESBlock(ConvTranspose(X_up3+X_down2)), X_up1←RESBlock(ConvTranspose(X_up2+X_down1)),

[0171] Since the first stage gradient descent process processes the vector of the image block, corresponding to steps 3 and 4 of Algorithm 3; the second stage is the proximal mapping process, which processes the complete image, corresponding to the operation after step 4 of Algorithm 3. Therefore, in the iterative process, the conversion from the first stage to the second stage requires a reshaping operation and a folding operation to obtain the image. Correspondingly, the conversion from the second stage to the first stage requires a reshaping operation and an unfolding operation to obtain the image vector.

[0172] The specific process of Algorithm 3 is as follows:

[0173] In the first stage, the gradient descent is performed block by block according to step 4 of Algorithm 3.

[0174] The second stage performs proximal mapping, that is, mapping the data processed by gradient descent and the initial reconstruction result through a U-shaped network based on residual blocks and cross connections. The specific processing process is as follows.

[0175] First, a convolutional neural network is used to extract the connection between image information and sampling rate distribution information into the feature space, where the sampling rate distribution information mapping plays a guiding role. This process is implemented by step 6 in Algorithm 3.

[0176] Secondly, the obtained features are input into the residual block and then downsampled by convolution. This process is performed three times, i.e., three downsamplings. This process is implemented by X_down1←Conv(RESBlock(X)), X_down2←Conv(RESBlock(X_down1)), and X_down3←Conv(RESBlock(X_down2)) in Algorithm 3.

[0177] Then, the features are extracted again for the final result of downsampling. This process is implemented by X_body←RESBlock(X_down3) in Algorithm 3.

[0178] In addition, the corresponding process of downsampling is performed, and upsampling is performed three times. This process first performs upsampling through convolution, and then connects the features just obtained with the features obtained by the downsampling process corresponding to the upsampling (such as up3 corresponding to down3) and sends them to the residual block. This process is implemented by X_up3←RESBlock(ConvTranspose(X_body+X_down3)), X_up1←RESBlock(ConvTranspose(X_up2+X_down1)) and X_up1←RESBlock(ConvTranspose(X_up2+X_down1)) in Algorithm 3.

[0179] Finally, the obtained features are connected with the features originally mapped to the feature space, and the number of channels is mapped to the original number of image channels through convolution. This process is implemented by X←Tail(X_up1+X_head) in Algorithm 3.

[0180] In one embodiment, a slight modification is made on the basis of the above embodiment. For the deep reconstruction module of the embodiment, that is, the U-Net network based on the residual block is used, and the global features of the image are not taken into account. In this embodiment, the Swin Transformer block is inserted into the U-Net network (i.e., the data processed by gradient descent and the initial reconstruction result are mapped by the residual block-based, moving window-based hierarchical visual self-attention model block and the U-type network across the connection). Considering the time and space complexity, the Swin Transformer block is inserted into the specific position of the U-Net network in this implementation case. Figure 3c A schematic diagram of the network structure of a UNet network based on a residual block and a Swin Transformer block provided in an embodiment of the present application, and the network structure is obtained as follows Figure 3c shown. Figure 3c The U-shaped network is more intuitively shown in FIG. The network is U-shaped. Figure 3c The curved arrows in the middle show the spanning connections. Figure 3c The Swin Transformer block is used in the third downsampling (corresponding to down3) and the first upsampling (corresponding to up3), and a cross connection is also performed. The cross connection can realize data transmission, such as transferring the downsampled features to the upsampling process.

[0181] The U-Net network structure is as follows: extract the head of the image to the feature space, that is, the Head part, downsample three times Down1, Down2, Dwon3, extract the middle part of the feature, that is, the Body part, upsample three times Up3, Up2, Up1, and restore the image features to the tail of the pixel space, that is, the Tail part. Among them, the Down3 and Up3 modules use two Swin Transformer blocks respectively.

[0182] Other stages of compressed sensing, such as the sampling stage and the initial reconstruction stage, may be the same as those in the above embodiment.

[0183] In one embodiment, the present application builds an end-to-end compressed sensing reconstruction network STCSNet based on Swin Transformer, which jointly optimizes a sampling network, an initial reconstruction network, and a deep reconstruction network. The sampling network learns the sampling matrix from the training image, which allows the compressed measurement value to retain more image structured information for better image reconstruction. The reconstruction network includes a linear initial reconstruction network and a nonlinear deep reconstruction network for learning the end-to-end mapping between compressed measurements and reconstructed images. The network is directly learned using a deep learning network and is not interpretable. Figure 3d A network structure diagram of STCSNet provided in an embodiment of the present application, Figure 3d The sampling stage, the initial reconstruction stage, and the reconstruction stage based on the Swin Transformer block and the U-Net network with cross-connection are shown. Among them, the short rectangle shows the sampling result, and the long rectangle shows the initial reconstruction result.

[0184] (1) The sampling stage is implemented through convolution operations. The sampling in this stage is learnable. The number of convolution kernel input channels is equal to the number of image channels, and the number of output channels is equal to the number of convolution kernels. Due to the use of block sampling, the number of convolution kernels is block size × block size × number of image channels × sampling rate. The convolution kernel size and step size are both block size, and padding is 0.

[0185] When sampling, block sampling is used, and the image X∈R H×W Divide into indivual Each non-overlapping image block is then reshaped into a vector x∈R N , use a learnable sampling matrix instead of a fixed random Gaussian matrix to obtain CS measurements, using a sampling subnetwork, denoted by F Φ (·), using a convolution kernel whose size and step size are the same as the block size, and the number of convolution kernels is the convolution operation of sampling rate × block size × block size, so the sampling process is

[0186] Y=F Φ (X) (6)

[0187] Y is the measured value obtained.

[0188] (2) The initial reconstruction stage is implemented through convolution and PixelShuffle operations. This stage is learnable. The number of input channels of the convolution kernel is equal to block size × block size × number of image channels × sampling rate, and the number of output channels is block size × block size × number of image channels. The convolution kernel size and step size are both 1, and padding is 0. PixelShuffle is to enlarge the feature resolution obtained by convolution by block size × block size, and reduce the number of channels by the corresponding multiple to the number of channels of the original image. Since this paper processes grayscale images, the number of image channels is assumed to be 1 below.

[0189] The initial reconstruction is to be restored from the measured values, which is also achieved through convolution operations, using convolution kernels with a step size of 1 and a number of kernels equal to block size × block size. The result is then reconstructed into an image through PixelShuffle. The process can be described as

[0190]

[0191] (3) Reconstruction phase based on Swin Transformer blocks and U-Net networks with cross connections (i.e., the initial reconstruction result is mapped through a moving window-based hierarchical visual self-attention model block and a U-net network with cross connections to obtain the reconstructed result): The deep reconstruction phase uses a U-Net network based on a moving window Swin Transformer block, which can make full use of global information, repeatedly mine network features, and extract image information. The U-Net network structure is as follows: extract the Head part of the image to the feature space, downsample three times Down1, Down2, Dwon3, extract the Body part of the feature, upsample three times Up3, Up2, Up1, and restore the image features to the Tail part of the pixel space, where the three downsampling, Body, and three upsampling modules use Swin Transformer blocks, and the corresponding Swin Transformer block numbers are 2, 2, 4, 2, 4, 2, and 2, respectively. The number of U-Net channels is 32, 64, 128, and 256, respectively.

[0192] (4) As for the sampling rate allocation strategy, since the present embodiment uses convolution for sampling, it is possible to consider obtaining all measurement values ​​through convolution, and then intercepting the required measurement values ​​for measurement value allocation.

[0193] The present application proposes a compressed sensing algorithm for adaptive sampling rate allocation, which does not require the use of original image information, obtains measurement values ​​at the sampling end, and reconstructs the obtained measurement values ​​at the reconstruction end, and the measurement value allocation algorithm has achieved very good results. The algorithm obtains some measurement values ​​through branches for initial reconstruction (i.e., sampling the sampled data at a first sampling rate; reconstructing the measurement values ​​obtained after sampling to obtain an initial image), and performs image information detection on the initial reconstruction result. First, the obtained image is mean filtered, and then the result is subtracted to obtain the difference in image information, based on which image measurement value allocation is performed (i.e., the initial image is mean filtered to obtain a filtered image; the information difference data of the initial image and the filtered image is determined; for each image block, the information amount of the image block is determined based on the information difference data of the image block). The reconstruction network uses a proximal descent algorithm (i.e., gradient mapping processing), and a U-Net network based on residual blocks and cross connections is used in the proximal mapping part. Not only is the model lightweight, but the reconstruction effect is also good.

[0194] In an exemplary embodiment, the present application provides a sampling device, Figure 4 This is a schematic diagram of the structure of a sampling device provided in an embodiment of the present application. The sampling device can be integrated on a communication node. Figure 4 As shown, the sampling device includes:

[0195] A first sampling module 410 is configured to sample the data to be sampled at a first sampling rate;

[0196] A reconstruction module 420 is configured to reconstruct the measurement values ​​obtained after sampling to obtain an initial image;

[0197] A first determination module 430 is configured to determine the information amount of multiple image blocks in the initial image;

[0198] A second determination module 440 is configured to allocate a second sampling rate based on the amount of information of each of the image blocks, and determine actual sampling parameters corresponding to each of the image blocks;

[0199] The second sampling module 450 is configured to sample, for each image block, a region corresponding to the image block in the data to be sampled based on actual sampling parameters of the image block to obtain a sampling result.

[0200] The sampling device provided in this embodiment is used to implement the sampling method provided in the embodiment of the present application. The implementation principle and technical effect of the sampling device provided in this embodiment are similar to those of the sampling method provided in the embodiment of the present application, and will not be repeated here.

[0201] Based on the above embodiment, a variant embodiment of the above embodiment is proposed. It should be noted that in order to make the description concise, only the differences from the above embodiment are described in the variant embodiment.

[0202] In one embodiment, the first sampling module 410 includes:

[0203] a determining unit, configured to determine a first sampling rate;

[0204] The sampling unit is configured to sample the data to be sampled at the first sampling rate.

[0205] In one embodiment, the determining unit is specifically configured as:

[0206] The first sampling rate is determined based on the size of the image block in the matrix block sampling process, the target sampling rate of the data to be sampled, and the target number of measurement values ​​of the data to be sampled.

[0207] In one embodiment, the first determining module 430 is specifically configured to:

[0208] Performing mean filtering on the initial image to obtain a filtered image;

[0209] Determining information difference data between the initial image and the filtered image;

[0210] For each image block, the information amount of the image block is determined based on the information difference data of the image block.

[0211] In one embodiment, the actual sampling parameters include one or more of an actual sampling rate and an actual number of measurement values.

[0212] In one embodiment, the sum of the first sampling rate and the second sampling rate is equal to the target sampling rate of the data to be sampled.

[0213] In one embodiment, the reconstruction module 420 is specifically configured as follows:

[0214] The measured values ​​after sampling are reconstructed through the sampling matrix to obtain the initial image.

[0215] In an exemplary embodiment, the present application provides a reconstruction device, Figure 5 A structural diagram of a reconstruction device provided in an embodiment of the present application; the reconstruction device can be integrated in a communication node, such as Figure 5 As shown, the reconstruction device provided by the present application includes:

[0216] An acquisition module 510 is configured to acquire an initial reconstruction result of a sampling result, where the sampling result is a result obtained by sampling using the sampling method provided in the present application;

[0217] The reconstruction module 520 is configured to reconstruct the initial reconstruction result through a U-shaped network that crosses the connection to obtain a reconstructed result.

[0218] The reconstruction device provided in this embodiment is used to implement the reconstruction method provided in the embodiment of the present application. The implementation principle and technical effect of the reconstruction device provided in this embodiment are similar to those of the reconstruction method provided in the embodiment of the present application, and will not be repeated here.

[0219] Based on the above embodiment, a variant embodiment of the above embodiment is proposed. It should be noted that in order to make the description concise, only the differences from the above embodiment are described in the variant embodiment.

[0220] In one embodiment, the reconstruction module 520 includes:

[0221] A processing module, configured to iteratively perform gradient mapping processing on the initial reconstruction result to obtain a reconstructed result;

[0222] The gradient mapping process includes: gradient descent processing and mapping processing through a U-type network that spans the connection.

[0223] In one embodiment, the processing module performs mapping processing by spanning the connected U-shaped network, including:

[0224] The data processed by gradient descent and the initial reconstruction result are mapped by a U-shaped network based on residual blocks and cross connections.

[0225] In one embodiment, the processing module performs mapping processing by spanning the connected U-shaped network, including:

[0226] The data processed by gradient descent and the initial reconstruction result are mapped by a residual block-based, moving window-based hierarchical visual self-attention model block and a U-shaped network with cross-connections.

[0227] In one embodiment, the reconstruction module 520 is specifically configured as follows:

[0228] The initial reconstruction result is mapped through a moving window-based hierarchical visual self-attention model block and a U-shaped network with cross-connections to obtain a reconstructed result.

[0229] In one embodiment, the processing module performs gradient descent processing including:

[0230] Vectorize the initial reconstruction results;

[0231] The initial reconstruction result in vector form is subjected to gradient descent based on the sampling matrix and the set step size.

[0232] In an exemplary embodiment, the present application also provides a communication node, Figure 6 A schematic diagram of the structure of a communication node provided in an embodiment of the present application. Figure 6 As shown, the communication node provided by the present application includes one or more processors 61 and a storage device 62; the processor 61 in the communication node can be one or more, Figure 6 A processor 61 is taken as an example; the storage device 62 is used to store one or more programs; the one or more programs are executed by the one or more processors 61, so that the one or more processors 61 implement the method provided in the embodiment of the present application. The communication node provided in the embodiment of the present application can execute the sampling method and / or reconstruction method provided in the embodiment of the present application.

[0233] In one embodiment, the present application may provide a first communication node for executing the sampling method and a second communication node for executing the reconstruction method. The first communication node and the second communication node may be the same communication node or different communication nodes. The communication node provided in this embodiment may be the first communication node and / or the second communication node.

[0234] The communication node further comprises: a communication device 63 , an input device 64 and an output device 65 .

[0235] The processor 61, storage device 62, communication device 63, input device 64 and output device 65 in the communication node may be connected via a bus or other means. Figure 6The example of connecting through bus is taken in the following.

[0236] The input device 64 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the communication node. The output device 65 may include a display device such as a display screen.

[0237] The communication device 63 may include a receiver and a transmitter. The communication device 63 is configured to perform information transmission and reception communication according to the control of the processor 61. The information includes but is not limited to the reconstruction result.

[0238] The storage device 62, as a computer-readable storage medium, can be configured to store software programs, computer executable programs, and modules, such as program instructions / modules corresponding to the sampling method described in the embodiment of the present application (for example, the first sampling module 410, the reconstruction module 420, the first determination module 430, the second determination module 440, and the second sampling module 450 in the device). For another example, program instructions / modules corresponding to the reconstruction method provided in the embodiment of the present application (for example, the acquisition module 510 and the reconstruction module 520 in the device).

[0239] The storage device 62 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the communication node, etc. In addition, the storage device 62 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the storage device 62 may further include a memory remotely arranged relative to the processor 61, and these remote memories may be connected to the communication node via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0240] In an exemplary embodiment, the present application also provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, any method described in the present application is implemented. The storage medium stores a computer program, and when the computer program is executed by a processor, the sampling method and the reconstruction method provided in the present application embodiment are implemented, wherein the sampling method includes: sampling the data to be sampled at a first sampling rate;

[0241] Reconstruct the measured values ​​obtained after sampling to obtain an initial image;

[0242] Determining the information amount of multiple image blocks in the initial image;

[0243] Allocating the second sampling rate based on the amount of information of each of the image blocks, and determining actual sampling parameters corresponding to each of the image blocks;

[0244] For each image block, a region corresponding to the image block in the data to be sampled is sampled based on actual sampling parameters of the image block to obtain a sampling result.

[0245] The reconstruction method includes: obtaining an initial reconstruction result of a sampling result, wherein the sampling result is a result obtained by sampling using the sampling method provided in an embodiment of the present application;

[0246] The initial reconstruction result is reconstructed through a U-shaped network that crosses the connection to obtain a reconstructed result.

[0247] The computer storage medium of the embodiment of the present application can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (Random Access Memory, RAM), a read-only memory (Read Only Memory, ROM), an erasable programmable read-only memory (ErasableProgrammable Read Only Memory, EPROM), a flash memory, an optical fiber, a portable CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination of the above. The computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0248] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to: electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0249] The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the above.

[0250] The computer program code for performing the operation of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and also conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0251] The above description is merely an exemplary embodiment of the present application and is not intended to limit the protection scope of the present application.

[0252] It will be appreciated by those skilled in the art that the term communication node may encompass any suitable type of wireless user equipment, such as a mobile phone, a portable data processing device, a portable web browser or a vehicle-mounted mobile station.

[0253] In general, various embodiments of the present application can be implemented in hardware or dedicated circuits, software, logic or any combination thereof. For example, some aspects can be implemented in hardware, while other aspects can be implemented in firmware or software that can be executed by a controller, microprocessor or other computing device, although the present application is not limited thereto.

[0254] Embodiments of the present application may be implemented by executing computer program instructions by a data processor of a mobile device, for example in a processor entity, or by hardware, or by a combination of software and hardware. The computer program instructions may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages.

[0255] The block diagram of any logic flow in the drawings of the present application may represent program steps, or may represent interconnected logic circuits, modules and functions, or may represent a combination of program steps and logic circuits, modules and functions. A computer program may be stored on a memory. The memory may have any type suitable for the local technical environment and may be implemented using any suitable data storage technology, such as but not limited to read-only memory (ROM), random access memory (RAM), optical memory devices and systems (digital versatile discs (DVD) or compact disks (CD)), etc. Computer-readable media may include non-transient storage media. A data processor may be any type suitable for the local technical environment, such as but not limited to a general-purpose computer, a special-purpose computer, a microprocessor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FGPAs) and a processor based on a multi-core processor architecture.

[0256] By way of exemplary and non-limiting examples, a detailed description of exemplary embodiments of the present application has been provided above. However, various modifications and adjustments to the above embodiments will be apparent to those skilled in the art when considered in conjunction with the accompanying drawings and claims, but will not depart from the scope of the present application. Therefore, the proper scope of the present application will be determined according to the claims.

Claims

1. A sampling method, characterized in that: include: Sampling the data to be sampled at a first sampling rate; Reconstruct the measured values ​​obtained after sampling to obtain an initial image; Determining the information amount of multiple image blocks in the initial image; Allocating the second sampling rate based on the amount of information of each of the image blocks, and determining actual sampling parameters corresponding to each of the image blocks; For each image block, a region corresponding to the image block in the data to be sampled is sampled based on actual sampling parameters of the image block to obtain a sampling result.

2. The method according to claim 1, characterized in that The step of sampling the data to be sampled at a first sampling rate comprises: determining a first sampling rate; The data to be sampled is sampled at the first sampling rate.

3. The method according to claim 2, characterized in that The determining of the first sampling rate comprises: The first sampling rate is determined based on the size of the image block in the matrix block sampling process, the target sampling rate of the data to be sampled, and the target number of measurement values ​​of the data to be sampled.

4. The method according to claim 1, characterized in that: The determining the information amount of the plurality of image blocks in the initial image comprises: Performing mean filtering on the initial image to obtain a filtered image; Determining information difference data between the initial image and the filtered image; For each image block, the information amount of the image block is determined based on the information difference data of the image block.

5. The method according to any one of claims 1 to 4, characterized in that: The actual sampling parameters include one or more of an actual sampling rate and an actual number of measurement values.

6. The method according to claim 1, characterized in that The sum of the first sampling rate and the second sampling rate is equal to the target sampling rate of the data to be sampled.

7. The method according to claim 1, characterized in that The step of reconstructing the measured values ​​obtained after sampling to obtain an initial image includes: The measured values ​​after sampling are reconstructed through the sampling matrix to obtain the initial image.

8. A reconstruction method, characterized in that: include: Obtaining an initial reconstruction result of a sampling result, wherein the sampling result is a result obtained by sampling according to any one of the methods of claims 1 to 7; The initial reconstruction result is reconstructed through a U-shaped network that crosses the connection to obtain a reconstructed result.

9. The method according to claim 8, characterized in that The reconstructing the initial reconstruction result by using the U-shaped network across the connection to obtain the reconstructed result includes: Iteratively performing gradient mapping processing on the initial reconstruction result to obtain a reconstructed result; The gradient mapping process includes: gradient descent processing and mapping processing through a U-type network that spans the connection.

10. The method according to claim 9, characterized in that Mapping is done through a U-shaped network across connections, including: The data processed by gradient descent and the initial reconstruction result are mapped by a U-shaped network based on residual blocks and cross connections.

11. The method according to claim 9, characterized in that Mapping is done through a U-shaped network across connections, including: The data processed by gradient descent and the initial reconstruction result are mapped by a residual block-based, moving window-based hierarchical visual self-attention model block and a U-shaped network with cross-connections.

12. The method according to claim 9, characterized in that The gradient descent process includes: Vectorize the initial reconstruction results; The initial reconstruction result in vector form is subjected to gradient descent based on the sampling matrix and the set step size.

13. The method according to claim 8, characterized in that The reconstructing the initial reconstruction result by using the U-shaped network across the connection to obtain the reconstructed result includes: The initial reconstruction result is mapped through a moving window-based hierarchical visual self-attention model block and a U-shaped network with cross-connections to obtain a reconstructed result.

14. A communication node, characterized in that: include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 13.

15. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.

Images