MRF image denoising method and system based on memory computing integrated device

Abstract

The application relates to the technical field of image processing, and provides an MRF image denoising method and system based on a memory and computing integrated device, which comprises the following steps: constructing a binary factor graph based on a four-neighborhood topology of a binary image, and obtaining a connection matrix based on multiple factor nodes and variable nodes; constructing a calculation matrix according to the connection matrix and configuring the calculation matrix to a memristor cross array, loading image observation values of the binary image to an input end of the memristor cross array, and obtaining a to-be-operated memristor cross array; performing voltage-current calculation and conversion through the to-be-operated memristor cross array, obtaining variable node messages and a log-likelihood ratio; and performing iterative judgment based on the variable node messages and the log-likelihood ratio, and outputting a denoised image. The application improves the calculation speed of a high-resolution image denoising application scene that needs real-time processing.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an MRF image denoising method and system based on a memory computing unit. Background Technology

[0002] Markov Random Field (MRF) image denoising is a classic probabilistic graphical model method widely used in computer vision and image processing. Existing MRF image denoising techniques are mainly based on message-passing frameworks such as Belief Propagation (BP) or variational inference, achieving effective noise removal through iterative optimization. In terms of hardware implementation, traditional MRF denoising processors primarily employ pure digital circuit architectures, deployed on FPGA, GPU, or CMOS-based Application-Specific Integrated Circuit (ASIC) platforms. These digital implementations typically discretize the MRF model into a finite state space, using fixed-point arithmetic to complete probability updates, message passing, and energy minimization calculations during the belief propagation process. At the algorithm optimization level, existing techniques widely employ logarithmic domain operations to avoid probability underflow, while utilizing simplification strategies such as minimum sum approximation and maximum product approximation to reduce computational complexity. Furthermore, to adapt to different noise types and image features, existing MRF denoising methods also integrate enhancement techniques such as multi-scale decomposition, adaptive neighborhood selection, and parameter learning, achieving significant progress in denoising effectiveness and algorithm robustness.

[0003] However, existing pure digital circuit MRF denoising schemes face severe power consumption and energy efficiency bottlenecks in practical applications. The belief propagation iterative process requires large-scale vector-matrix multiplication, logarithmic-summation, and message accumulation operations among all pixel nodes and factor nodes of the image. These intensive numerical calculations require frequent SRAM / DRAM random accesses and a large number of digital multipliers operating in parallel. The traditional von Neumann architecture's "storage-computation" separation mode causes data to be repeatedly moved between the processor and memory, resulting in huge memory access power consumption overhead. At the same time, the high power consumption characteristics of digital multipliers and floating-point units further exacerbate the overall energy consumption problem. In battery-powered scenarios such as edge computing and mobile vision, the power consumption level of existing digital MRF processors often exceeds the device's capacity, making it difficult to meet the low-power requirements of real-time image denoising. In addition, as image resolution continues to increase, the computational complexity and storage requirements of traditional digital solutions increase quadratically, severely limiting scalability. A breakthrough low-power hardware architecture is urgently needed to solve this fundamental technical bottleneck. Summary of the Invention

[0004] This invention provides an MRF image denoising method and system based on a memory computing unit to overcome the shortcomings of existing technologies.

[0005] This invention provides an MRF image denoising method based on a memory computing unit, comprising:

[0006] S1: Construct a bipartite factor graph based on the four-neighborhood topology of the binary image to obtain a connection matrix based on multiple factor nodes and variable nodes;

[0007] S2: Construct a calculation matrix based on the connection matrix and configure it to the memristor cross array. Load the image observation values ​​of the binary image into the input terminal of the memristor cross array to obtain the memristor cross array to be run.

[0008] S3: Calculate and convert voltage and current through the memristor cross array to be run, and obtain variable node messages and log-likelihood ratio;

[0009] S4: Based on the variable node message and the log-likelihood ratio, perform iterative judgment and output the denoised image.

[0010] According to the MRF image denoising method based on in-memory computing provided by the present invention, step S1 further includes:

[0011] S11: Binarize the input image to be denoised to obtain a binary image;

[0012] S12: For each pixel of the binary image, a variable node is created to obtain a variable node set; for each pixel of the binary image, a data factor is introduced to obtain a data factor set; for the connection relationship between every two adjacent pixel pairs of the binary image, a smoothing factor is created to obtain a smoothing factor set.

[0013] S13: Arrange the set of variable nodes, the set of data factors, and the set of smoothing factors into a sparse matrix to obtain the initial connection matrix;

[0014] S14: Reduce the factor nodes of the data factor set in the initial connection matrix, and use the observed values ​​of the pixels as the initial priors of the corresponding variable nodes to obtain the connection matrix.

[0015] According to the MRF image denoising method based on in-memory computing provided by the present invention, step S13 further includes:

[0016] S131: Number the factor nodes by row index to obtain the matrix row coordinates; number the variable nodes by column index to obtain the matrix column coordinates;

[0017] S132: Assign values ​​to the row coordinates and column coordinates of the matrix based on the adjacency relationship. If the factor node and the variable node are connected, the element is set to 1; if the factor node and the variable node are not connected, the element is set to 0, thus obtaining the initial connection matrix.

[0018] According to the MRF image denoising method based on in-memory computing provided by the present invention, step S2 further includes:

[0019] S21: Based on the connection matrix, the message transmission from variable node to factor node and the message transmission from factor node to variable node are arranged into vectors to obtain a message vector;

[0020] S22: Based on the connection matrix, extract the factor node numbers and variable node numbers to obtain the index vector;

[0021] S23: Based on the message vector and the index vector, construct a first calculation matrix and a second calculation matrix for calculating the message transmission from the variable node to the factor node;

[0022] S24: Based on the message vector, construct a third calculation matrix and a fourth calculation matrix for calculating the log-likelihood ratio of variable nodes;

[0023] S25: Concatenate the first calculation matrix, the second calculation matrix, the third calculation matrix, and the fourth calculation matrix to obtain the complete calculation matrix;

[0024] S26: Take the reciprocal of the non-zero element values ​​of the complete calculation matrix to obtain the resistance state configuration matrix; wherein, the zero value of the resistance state configuration matrix corresponds to a high resistance state, and the non-zero value of the resistance state configuration matrix corresponds to the corresponding resistance state;

[0025] S27: Configure the resistance state values ​​of the memristor cross array based on the resistance state configuration matrix to obtain the configured memristor cross array;

[0026] S28: Convert the image observations of the binary image into voltage signals and load them into the input of the configured memristor cross array to obtain the memristor cross array to be run.

[0027] According to the present invention, an MRF image denoising method based on a memory-based compute unit is provided, wherein the first calculation matrix is ​​used to perform a weighted summation of the transmission messages from factor nodes to variable nodes; the second calculation matrix is ​​used to load the initial prior log-likelihood ratio corresponding to the pixel observation value into the message calculation from variable nodes to factor nodes; the third calculation matrix is ​​used to perform a weighted summation of the transmission messages from factor nodes to variable nodes; and the fourth calculation matrix is ​​used to load the initial prior log-likelihood ratio corresponding to the pixel observation value into the calculation of the log-likelihood ratio of the variable nodes.

[0028] According to the MRF image denoising method based on in-memory computing provided by the present invention, step S28 further includes:

[0029] S281: Arrange the image observations of the binary image in a preset order to obtain the observation vector;

[0030] S282: Convert the observed value vector into an input voltage signal;

[0031] S283: The input voltage signal is applied to a specified row of the input terminal of the configured memristor cross array, and the initial state of the remaining input rows of the configured memristor cross array is set to 0 to obtain the memristor cross array to be run.

[0032] According to the MRF image denoising method based on in-memory computing provided by the present invention, step S3 further includes:

[0033] S31: Calculate the current based on the resistance values ​​and input voltages of multiple memristor cells in the memristor cross array to be operated, and obtain the current of multiple memristor cells.

[0034] S32: Based on Kirchhoff's current law, the currents of multiple memristor units on the same column are superimposed to obtain the output current;

[0035] S33: The output current is converted into an output voltage signal through a current-to-voltage conversion circuit;

[0036] S34: Extract the first part of the voltage corresponding to the variable node message and the second part of the voltage corresponding to the log-likelihood ratio from the output voltage signal to obtain the variable node message and the log-likelihood ratio.

[0037] According to the MRF image denoising method based on in-memory computing provided by the present invention, step S4 further includes:

[0038] S41: Perform polarity determination on the log-likelihood ratio. When the log-likelihood ratio is less than 0, it is determined to be the first pixel value. When the log-likelihood ratio is greater than 0, it is determined to be the second pixel value. Obtain the determination result of the current iteration.

[0039] S42: Calculate the number of flips between the current iteration result and the previous iteration result;

[0040] S43: Determine whether the number of flips is less than a preset threshold. If the number of flips is less than the preset threshold, output the current iteration determination result as a denoised image. If the number of flips is not less than the preset threshold, perform adjacent bit swapping on the variable node message to obtain the updated message, and reload the updated message to the input of the memristor cross array for iterative calculation.

[0041] According to the MRF image denoising method based on in-memory computing provided by the present invention, in step S41, the first pixel value is -1, corresponding to a black pixel; the second pixel value is +1, corresponding to a white pixel.

[0042] The present invention also provides an MRF image denoising system based on a memory computing unit, comprising:

[0043] The factor graph construction module is used to construct a bipartite factor graph based on the four-neighborhood topology of a binary image, and obtain a connection matrix based on multiple factor nodes and variable nodes.

[0044] The resistance configuration module is used to construct a calculation matrix based on the connection matrix and configure the resistance values ​​of multiple memristor cells in the memristor cross array according to the calculation matrix.

[0045] The memristor cross array is composed of multiple memristor units arranged in a row-column cross manner. After receiving the resistance state value configured by the resistance state configuration module, the memristor cross array is used to receive the input voltage and perform voltage and current calculations based on the configured resistance state value.

[0046] A current-to-voltage converter, connected to the column line output of the memristor cross array, is used to convert the output current into a voltage signal;

[0047] An iterative control module is used to perform iterative judgment based on the variable node message and the log-likelihood ratio. If the termination condition is met, a denoised image is output; otherwise, the updated message is reloaded into the memristor cross array. The iterative control module is implemented through a mixed-signal circuit or an analog circuit. The mixed-signal circuit is implemented through a digital circuit combined with an analog-to-digital converter and a digital-to-analog converter.

[0048] This invention provides an MRF image denoising method and system based on in-memory computing. By constructing a bipartite factor graph based on the four-neighbor topology of a binary image and obtaining a connection matrix, the image denoising problem can be mapped to a message passing problem of the factor graph, thus laying a solid theoretical foundation for subsequent hardware implementation. Secondly, by converting the connection matrix into a computation matrix and directly configuring it to the resistance state values ​​of the memristor cross array, this invention achieves a direct mapping from algorithm to hardware, avoiding the complex hardware-software interface conversions of traditional solutions and significantly reducing system complexity and power consumption. Subsequently, utilizing the physical characteristics of the memristor cross array for voltage and current calculations, this invention enables large-scale parallel matrix-vector multiplication operations in the analog domain, compared to traditional methods. The digital processor's step-by-step calculation method improves computation speed by several orders of magnitude while reducing power consumption by more than two orders of magnitude, making it particularly suitable for high-resolution image denoising applications requiring real-time processing. This invention also converts the output current of the memristor array into a processable voltage signal through current-to-voltage conversion and extracts variable node messages and log-likelihood ratios, achieving seamless integration between analog calculation results and subsequent processing circuits, ensuring the accuracy and stability of signal transmission. Furthermore, iterative judgment based on variable node messages and log-likelihood ratios, and determining output or continued iteration based on termination conditions, allows this invention to maintain denoising effectiveness while avoiding unnecessary computational overhead through adaptive iterative control, further improving overall energy efficiency. In particular, this invention provides both digital and analog circuit implementations. It can achieve flexible control and high decision accuracy through mixed-signal circuits, or extremely low power consumption and extremely fast response speed through pure analog circuits, providing selectable technical paths for different application needs and expanding the application scope of this invention. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0050] Figure 1 A schematic diagram of the MRF image denoising method based on a memory computing unit provided in this embodiment of the invention;

[0051] Figure 2 This is a schematic diagram of an MRF image denoising system based on a memory computing unit, provided as an embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, embodiments of this invention, and should not be construed as limiting the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. In the description of this invention, it should be understood that the terminology used is for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0053] The embodiments of the present invention are described below with reference to the figures.

[0054] like Figure 1 As shown, the present invention provides an MRF image denoising method based on a memory computing unit, comprising:

[0055] S1: Construct a bipartite factor graph based on the four-neighborhood topology of the binary image to obtain a connection matrix based on multiple factor nodes and variable nodes.

[0056] Step S1 further includes:

[0057] S11: Binarize the input image to be denoised to obtain a binary image.

[0058] Furthermore, the present invention performs binarization processing on the input image in step S11. Specifically, the binarization processing is to convert the input image into a black and white binary image, where black pixels correspond to grayscale value 0 and white pixels correspond to grayscale value 255. In the output binary image, each pixel takes the value of -1 or 1, where -1 corresponds to a black pixel and 1 corresponds to a white pixel.

[0059] S12: For each pixel of the binary image, a variable node is created to obtain a variable node set; for each pixel of the binary image, a data factor is introduced to obtain a data factor set; for the connection relationship between every two adjacent pixel pairs of the binary image, a smoothing factor is created to obtain a smoothing factor set.

[0060] Furthermore, in step S12, the present invention traverses the binary image, assigns a variable node to each pixel, and numbers them in order from left to right and from top to bottom to form a set of variable nodes. At the same time, the present invention introduces a data factor of degree 1 for each pixel. The data factor is used to characterize the consistency weight between the pixel and the observed value. Each data factor is connected to a variable node to form a set of data factors. Here, X is the input image, and the observed value Y is the original observed image obtained from transmission and other processes, which contains noise. Y has a certain offset from X, and Y is not necessarily binary. In addition, for the connection relationship between every two adjacent pixel pairs, the present invention creates a binary smoothing factor. The adjacent pixel pairs are determined based on the four-neighbor relationship to obtain a set of smoothing factors.

[0061] S13: Arrange the set of variable nodes, the set of data factors, and the set of smoothing factors into a sparse matrix to obtain the initial connection matrix.

[0062] Step S13 further includes:

[0063] S131: Number the factor nodes by row index to obtain the matrix row coordinates; number the variable nodes by column index to obtain the matrix column coordinates.

[0064] In step S131, the present invention numbers the factor nodes by row index. The factor nodes include a set of data factors and a set of smoothing factors. The present invention numbers them in the order from left to right and from top to bottom in the original image to obtain the matrix row coordinates. In addition, the variable nodes are numbered by column index in the same order to obtain the matrix column coordinates.

[0065] S132: Assign values ​​to the row coordinates and column coordinates of the matrix based on the adjacency relationship. If the factor node and the variable node are connected, the element is set to 1; if the factor node and the variable node are not connected, the element is set to 0, thus obtaining the initial connection matrix.

[0066] In step S132, the present invention assigns values ​​to the row coordinates and column coordinates of the matrix based on the adjacency relationship. Specifically, the present invention creates a sparse 0 / 1 matrix. If the factor nodes and variable nodes corresponding to the rows and columns are connected, the elements are set to 1; otherwise, the elements are set to 0, thus obtaining the initial connection matrix.

[0067] S14: Reduce the factor nodes of the data factor set in the initial connection matrix, and use the observed values ​​of the pixels as the initial priors of the corresponding variable nodes to obtain the connection matrix.

[0068] Furthermore, in step S14, the present invention first identifies the rows corresponding to the data factors in the initial connection matrix. Since the data factors are factor nodes of degree 1, each data factor is connected to only one variable node. The present invention removes these factor nodes of degree 1 and merges them into the variable nodes connected to them. The observed values ​​are used as the initial priors of the variable nodes. After the removal, the total number of factor nodes is reduced, and the remaining factor nodes are smoothing factors, thus obtaining the connection matrix.

[0069] The steps of S1 of the present invention will be explained below with reference to specific expressions and embodiments.

[0070] In step S1, the input image is first binarized. That is, if the input is a multi-valued image, binarization is required. After binarization, this invention creates a variable node for each pixel, resulting in a set of variable nodes. Furthermore, a data factor of degree 1 is introduced for each pixel. Characterization and Observation Values Consistency weight The aforementioned It is a binary value of -1 (corresponding to a black pixel, grayscale value 0) and 1 (corresponding to a white pixel, grayscale value 255). The original observation images obtained through a series of processes, including transmission, contain noise and may have some offset from the original image. It is not necessarily binary; for every two adjacent pixel pairs The connection between them creates a binary smoothing factor. The potential function is At this point, the total number of variable nodes is [number missing]. Total number of factor nodes ,in The pixel coordinates are For row coordinates, For column coordinates, For pixels The coordinates of adjacent pixels, For pixels The true value, For pixels The observed values, Let be the potential function of the data factor. These are the weighting coefficients of the data factors. Let be the potential function of the smoothing factor. These are the weighting coefficients of the smoothing factor. The number of rows in the image. This represents the number of columns in the image.

[0071] Furthermore, this invention organizes all connection relationships into a sparse 0 / 1 matrix. The matrix row index corresponds to the factor node, and the column index corresponds to the variable node. The indices are numbered sequentially from left to right and top to bottom in the original image. If a factor node and a variable node are connected in the same row and column, the element is set to 1. Because the image has a large number of pixels, this invention reduces factor nodes with a degree of 1 and merges them into the variable nodes they are connected to. This means that the observed value is used as the initial prior for the variable node, and subsequently used as input for iterative propagation, thus reducing the total number of factor nodes to [number missing]. .

[0072] S2: Construct a computation matrix based on the connection matrix and configure it to the memristor cross array. Load the image observation values ​​of the binary image to the input of the memristor cross array to obtain the memristor cross array to be run.

[0073] Step S2 further includes:

[0074] S21: Based on the connection matrix, the messages passed from variable nodes to factor nodes and from factor nodes to variable nodes are arranged into vectors to obtain message vectors.

[0075] In step S21, the present invention traverses each row of the connection matrix, with each row corresponding to a factor node. It finds the column positions in that row where the element is 1, and these column positions correspond to the variable nodes connected to that factor node. Since the smoothing factor is a factor node of degree 2, and each factor node connects to two variable nodes, each row has two elements of 1. Subsequently, the present invention extracts the transmission messages from two variable nodes to the factor node in each row of the connection matrix, arranging them in row order to form a transmission message vector from variable nodes to factor nodes. Simultaneously, the present invention extracts the transmission messages from the factor node to two variable nodes in each row, arranging them in the same order to form a transmission message vector from factor nodes to variable nodes. The two transmission message vectors have the same dimension, and elements at the same position correspond to the same factor node and variable node connection relationship, thus obtaining the message vector.

[0076] S22: Based on the connection matrix, extract the factor node numbers and variable node numbers to obtain the index vector.

[0077] In step S22, similarly, the present invention traverses the connection matrix and records the row index of each row as the factor node number. Since each factor node has a degree of 2, each factor node number appears twice in the vector and is adjacent to each other. They are arranged in row order to form a factor index vector. At the same time, the present invention finds the column index position with element 1 in each row. These column indices correspond to variable node numbers. The corresponding variable node numbers are extracted according to the factor node traversal order and arranged to form a variable index vector. Then, the factor index vector and the variable index vector are combined to obtain the index vector.

[0078] S23: Based on the message vector and the index vector, construct a first calculation matrix and a second calculation matrix for calculating the message transmission from the variable node to the factor node; S24: Based on the message vector, construct a third calculation matrix and a fourth calculation matrix for calculating the log-likelihood ratio of the variable node.

[0079] The first calculation matrix is ​​used to perform a weighted summation of the messages passed from factor nodes to variable nodes; the second calculation matrix is ​​used to load the initial prior log-likelihood ratio corresponding to the pixel observation value into the message calculation from variable nodes to factor nodes; the third calculation matrix is ​​used to perform a weighted summation of the messages passed from factor nodes to variable nodes; and the fourth calculation matrix is ​​used to load the initial prior log-likelihood ratio corresponding to the pixel observation value into the calculation of the log-likelihood ratio of the variable nodes.

[0080] In step S23, the present invention constructs a first calculation matrix and a second calculation matrix based on the message vector and the index vector.

[0081] The first calculation matrix is ​​used to perform a weighted summation of the messages passed from factor nodes to variable nodes, thus realizing the calculation of messages from variable nodes to factor nodes. For the first calculation matrix, this invention determines the set of factor nodes connected to each variable node based on the index vector. For a variable node v, this invention finds all positions in the variable index vector equal to v, and the factor index vector elements corresponding to these positions represent the factor nodes connected to variable node v. In the first calculation matrix, for the i-th position of the output vector, this invention needs to extract the messages of other factor nodes connected to the same variable node (excluding the i-th position) from the message vector passed from factor nodes to variable nodes, and perform a weighted summation. The matrix elements are weighted according to the connection relationships.

[0082] The second calculation matrix is ​​used to load the initial prior log-likelihood ratio corresponding to the pixel observation value into the message calculation from variable node to factor node. The present invention constructs the second calculation matrix based on the variable index vector, and the matrix elements represent the contribution weight of the initial prior log-likelihood ratio to each variable node to factor node message.

[0083] Step S24 constructs the third and fourth computation matrices based on the message vectors.

[0084] The third calculation matrix is ​​used to perform a weighted summation of the messages passed from factor nodes to variable nodes, thereby calculating the log-likelihood ratio of the variable nodes. For the third calculation matrix, this invention determines all factor nodes connected to each variable node based on the connection matrix. For each variable node, this invention sets a corresponding row in the third calculation matrix, where the elements of that row correspond to the message positions of the factor nodes connected to that variable node in the message vector passed from the factor node to the variable node. The matrix elements are assigned weight values ​​for weighted summation.

[0085] The fourth calculation matrix is ​​used to load the initial prior log-likelihood ratio corresponding to the pixel observation value into the calculation of the log-likelihood ratio of the variable node. In the fourth calculation matrix constructed in this invention, the rows of the matrix correspond to the variable nodes, the columns correspond to the initial prior log-likelihood ratio vector, and the matrix elements represent the contribution weight of the initial prior to the log-likelihood ratio of each variable node.

[0086] S25: Concatenate the first calculation matrix, the second calculation matrix, the third calculation matrix, and the fourth calculation matrix to obtain the complete calculation matrix.

[0087] In step S25, the present invention performs matrix concatenation in the row direction, concatenating the first and second calculation matrices by columns to form the upper half matrix, concatenating the third and fourth calculation matrices by columns to form the lower half matrix, and then concatenating the upper half matrix and the lower half matrix by rows to obtain the complete calculation matrix.

[0088] S26: Take the reciprocal of the non-zero element values ​​of the complete calculation matrix to obtain the resistance state configuration matrix; wherein, the zero value of the resistance state configuration matrix corresponds to a high resistance state, and the non-zero value of the resistance state configuration matrix corresponds to the corresponding resistance state.

[0089] In step S26, the present invention iterates through each element of the complete calculation matrix, calculates the reciprocal value of non-zero elements, and keeps zero elements as 0, to obtain the resistance state configuration matrix. In the resistance state configuration matrix, zero value corresponds to high resistance state, indicating that the memristor is not conducting, and non-zero value corresponds to the corresponding resistance state, indicating the conductance value of the memristor.

[0090] S27: Configure the resistance values ​​of the memristor cross array based on the resistance configuration matrix to obtain the configured memristor cross array.

[0091] The memristor cross array used in this invention is composed of multiple memristor cells arranged in a row-column cross pattern. In step S27, this invention maps each element value of the resistance state configuration matrix to the memristor cell at the corresponding position in the memristor cross array, and sets the resistance state value of the memristor cell to be equal to the element value at the corresponding position in the resistance state configuration matrix, thereby completing the resistance state configuration of the memristor cell and obtaining the configured memristor cross array.

[0092] S28: Convert the image observation values ​​of the binary image into voltage signals and load them into the input terminal of the configured memristor cross array to obtain the memristor cross array to be run.

[0093] Step S28 further includes:

[0094] S281: Arrange the image observations of the binary image in a preset order to obtain the observation vector.

[0095] In step S281, the present invention aims to arrange the image observations of the binary image in a preset order, which is from left to right and from top to bottom. The present invention extracts the observation value Y of each pixel of the binary image in this order. The observation value Y contains noise. The extracted observation values ​​are arranged in order to form a one-dimensional vector to obtain the observation value vector.

[0096] S282: Convert the observed value vector into an input voltage signal.

[0097] In step S282, the present invention converts each value in the observation vector into a corresponding voltage value, and the observation value and the voltage value are converted according to a linear mapping relationship to obtain the input voltage signal vector.

[0098] S283: The input voltage signal is applied to a specified row of the input terminal of the configured memristor cross array, and the initial state of the remaining input rows of the configured memristor cross array is set to 0 to obtain the memristor cross array to be run.

[0099] In step S283, the present invention aims to load the input voltage signal onto a designated row of the input terminal of the configured memristor cross array. The input terminal of the memristor cross array includes multiple row lines. The present invention loads each voltage value of the input voltage signal vector onto the designated row line respectively. The designated row corresponds to the input position of the observation vector in the complete calculation matrix. Subsequently, the present invention sets the voltage of the remaining input rows of the configured memristor cross array to 0, indicating that no voltage is applied to these row lines. After loading is completed, the designated row of the memristor cross array retains the input voltage signal until the iteration ends, thus obtaining the initialized memristor cross array.

[0100] The following explanation of step S2 of the present invention is based on specific expressions and embodiments.

[0101] In step S2, starting from the connection matrix, the messages passed from all variable nodes to factor nodes are arranged. In the connection matrix, each value 1 represents a pair of bidirectional messages that need to be passed. The messages from the two variables to the factors in each row of the connection matrix are arranged as follows: , dimension ,in This indicates that the element at the corresponding position in the connection matrix is ​​1, and the message arrangement from the corresponding factor node to the variable node is as follows. , dimension in ,in Represents variable nodes. Factor nodes with a representativeness of 2, Factor nodes with a degree of 1 (which are removed and therefore do not participate in message iteration, but serve as input sources). This represents the message vector passed from the variable node to the factor node. This is the message vector passed from the factor node to the variable node.

[0102] In the two vectors above, the factor nodes and variable nodes corresponding to each element at the same position have the same index. Arranging these factor node indexes into a vector yields... (Since each factor node has a degree of 2, every two adjacent nodes are the same.) Similarly, arrange the corresponding variable node numbers into a vector to obtain... ,in A vector for numbering factor nodes. A vector for numbering variable nodes.

[0103] To calculate the message from a variable node to a factor node based on the message from the factor node to the variable node, it is necessary to perform a weighted sum of the messages from all factor nodes connected to the variable node, excluding the message outflow direction. ,and .in, Variable Node To factor node The transmission of messages For factor nodes To variable node The message transmission also maintains the initial prior input at all times. This indicates the direction of message flow, excluding the outflow direction. Other connected factor nodes, This represents the set of all factor nodes connected to the variable node, i.e., the set of all factor nodes connected to the variable node. The set of all unpruned factor nodes of degree 2. This indicates the factor node that excludes the outflow direction of messages. For factor nodes To variable node The transmission of messages For factor nodes To variable node The transmission of messages Variable nodes that exclude the outflow direction of messages Other variable nodes, Represents the relationship between factor nodes A set of connected variable nodes. This indicates a variable node that excludes the outflow direction of messages.

[0104] Subsequently, the iterative calculation is rewritten into a vector-matrix multiplication form, which can be implemented using a memristor array. The expression implemented using memristors here is:

[0105] ;

[0106] in, For variable nodes The observed value of the corresponding pixel.

[0107] Rewriting the above expression in vector form, we get:

[0108] ;

[0109] Where vector Let be the vector formed by arranging the original prior inputs of the pixels, with dimension . ,matrix Let be the first computation matrix, where the elements are... , dimension ,in For the first The variable node number corresponding to each position For the first The variable node number corresponding to each position For the first The factor node number corresponding to each position. For the first The factor node numbers corresponding to each position, matrix The second computational matrix contains the following elements: , dimension .

[0110] Similarly, to calculate the log-likelihood ratio of the variable nodes, i.e. It is also implemented using a memristor, and the specific vector-matrix multiplication implementation is as follows:

[0111] ;

[0112] in, Let be a vector composed of the log-likelihood ratios of the variable nodes, with dimension . ,matrix The third computational matrix contains the elements. Size is ,matrix The fourth computational matrix contains the elements. Size is .

[0113] By combining the two equations above, we obtain:

[0114] ;

[0115] in, This is the matrix required to configure the memristor. When configuring it, the reciprocal of the matrix value needs to be taken as the resistance state value of the memristor. In particular, 0 corresponds to a high resistance state where it is not conducting, and 1 corresponds to a low resistance state where it is fully conducting.

[0116] S3: Calculate and convert voltage and current using the initialized memristor cross array to obtain variable node messages and log-likelihood ratio.

[0117] Step S3 further includes:

[0118] S31: Calculate the current based on the resistance values ​​and input voltages of multiple memristor cells in the memristor cross array to be operated, and obtain the current of multiple memristor cells.

[0119] Furthermore, the initialization memristor cross array comprises multiple memristor units, each located at the intersection of a row and a column. In the current calculation, this invention first obtains the input voltage value of the row line where each memristor unit is located and the resistance value of that memristor unit. Then, it calculates the current of that memristor unit according to Ohm's law. The current equals the voltage divided by the resistance value. After this invention traverses all memristor units in the memristor cross array and performs the above calculation for each memristor unit, multiple memristor unit currents are obtained, and their meanings are as follows: .

[0120] S32: Based on Kirchhoff's current law, the current of multiple memristor units on the same column line is superimposed to obtain the output current.

[0121] Furthermore, Kirchhoff's current law states that the sum of the currents flowing into a node is equal to the sum of the currents flowing out of the node. In a memristor cross array, multiple memristor cells are connected in the same column. In step S32, the present invention extracts the current values ​​of all memristor cells in each column and adds these current values ​​together to obtain the total current of that column. After performing the above superposition operation on all columns of the memristor cross array, each column obtains a total current value. The total current values ​​of multiple columns constitute the output current vector, thus obtaining the output current.

[0122] S33: The output current is converted into an output voltage signal through a current-to-voltage conversion circuit.

[0123] Furthermore, in step S33, the present invention inputs each current value of the output current vector into the current-voltage conversion circuit, the conversion circuit outputs the corresponding voltage value, and the multiple voltage values ​​constitute the output voltage signal vector to obtain the output voltage signal.

[0124] S34: Extract the first part of the voltage corresponding to the variable node message and the second part of the voltage corresponding to the log-likelihood ratio from the output voltage signal to obtain the variable node message and the log-likelihood ratio.

[0125] Furthermore, the output voltage signal vector is arranged according to the row structure of the complete calculation matrix. The first part corresponds to the calculation result of the message transmission from the variable node to the factor node, and the second part corresponds to the calculation result of the log-likelihood ratio of the variable node. In step S34 of this invention, the output voltage signal vector is divided into two parts according to the row division position of the complete calculation matrix. The first part is extracted as the first part voltage, corresponding to the variable node message, i.e. The currently calculated update message value represents the confidence level for the variable node to take the -1 or +1 state. The latter part is extracted as the second part voltage. Corresponding to the log-likelihood ratio, the variable node message and the log-likelihood ratio are obtained.

[0126] S4: Based on the variable node message and the log-likelihood ratio, perform iterative judgment and output the denoised image.

[0127] Step S4 further includes:

[0128] S41: Perform polarity determination on the log-likelihood ratio. When the log-likelihood ratio is less than 0, it is determined as the first pixel value. When the log-likelihood ratio is greater than 0, it is determined as the second pixel value. Obtain the determination result of the current iteration.

[0129] In step S41, the present invention obtains each value in the log-likelihood ratio vector, determines the sign of the value, and when the value is less than 0, the present invention determines the corresponding pixel as the first pixel value -1, and the first pixel value -1 corresponds to a black pixel. When the value is greater than 0, the present invention determines the corresponding pixel as the second pixel value +1, and the second pixel value +1 corresponds to a white pixel. After performing the above determination operation on all values ​​in the log-likelihood ratio vector, each value obtains a pixel value. Multiple pixel values ​​are arranged according to the spatial position of the original image to form a determination result image, thus obtaining the determination result of the current iteration.

[0130] In step S41, the first pixel value is -1, corresponding to a black pixel; the second pixel value is +1, corresponding to a white pixel.

[0131] S42: Calculate the number of flips between the current iteration result and the previous iteration result.

[0132] In step S42, the present invention obtains the pixel value vector of the current iteration judgment result and the pixel value vector of the previous iteration judgment result. Subsequently, the corresponding position elements of the two vectors are compared. When the pixel values ​​at the corresponding positions are not equal, it is recorded as a flip. After traversing all positions of the two vectors, the total number of flips is counted to obtain the number of flips.

[0133] S43: Determine whether the number of flips is less than a preset threshold. If the number of flips is less than the preset threshold, output the current iteration determination result as a denoised image. If the number of flips is not less than the preset threshold, perform adjacent bit swapping on the variable node message to obtain the updated message, and reload the updated message to the input of the memristor cross array for iterative calculation.

[0134] In step S43, the preset threshold is a pre-set upper limit for the number of flips. The present invention compares the calculated number of flips with the preset threshold. If the number of flips is less than the preset threshold, the present invention considers the iteration to be converged and outputs the result of the current iteration as the final denoised image. The denoised image contains the judgment pixel values ​​of all pixels and is arranged according to the spatial position of the original image to form a binary image.

[0135] If the number of flips is not less than a preset threshold, this invention considers the iteration to have not converged and performs adjacent bit swapping on the variable node message. Adjacent bit swapping refers to this invention obtaining the variable node message vector and swapping the positions of two adjacent elements in the vector: the first element swaps with the second, the third with the fourth, and so on, completing the swapping of all adjacent bits to obtain the updated message. When convergence fails, this invention converts the updated message into a voltage signal, loads it onto the corresponding row line of the memristor cross array input, and returns to step S31 for the next round of iteration calculation.

[0136] like Figure 2 As shown, the present invention also provides an MRF image denoising system based on a memory computing unit, comprising:

[0137] The factor graph construction module 100 is used to construct a bipartite factor graph based on the four-neighborhood topology of a binary image, and obtain a connection matrix based on multiple factor nodes and variable nodes.

[0138] The resistance configuration module 200 is used to construct a calculation matrix based on the connection matrix and configure the resistance values ​​of multiple memristor cells in the memristor cross array 300 according to the calculation matrix.

[0139] The memristor cross array 300 is composed of multiple memristor units arranged in a row-column cross manner. After receiving the resistance state value configured by the resistance state configuration module 200, the memristor cross array 300 is used to receive the input voltage and perform voltage and current calculations based on the configured resistance state value.

[0140] A current-to-voltage converter 400 is connected to the column line output terminal of the memristor cross array 300 and is used to convert the output current into a voltage signal.

[0141] The iterative control module 500 is used to perform iterative judgment based on the variable node message and the log-likelihood ratio. If the termination condition is met, the denoised image is output; otherwise, the updated message is reloaded into the memristor cross array. The iterative control module 500 is implemented by a mixed-signal circuit or an analog circuit. The mixed-signal circuit is implemented by a digital circuit in conjunction with an analog-to-digital converter and a digital-to-analog converter.

[0142] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0143] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for MRF image denoising based on a memory computing device, characterized in that, The method comprises the following steps: S1: constructing a binary factor graph based on the four-neighborhood topology of a binary image, and obtaining a connection matrix based on a plurality of factor nodes and variable nodes; S2: constructing a calculation matrix based on the connection matrix and configuring the calculation matrix to a memristor cross array, loading an image observation value of the binary image to an input end of the memristor cross array, and obtaining a to-be-run memristor cross array; Step S2 further comprises: S21: based on the connection matrix, vector arranging a message passed from a variable node to a factor node and a message passed from a factor node to a variable node, and obtaining a message vector; S22: based on the connection matrix, extracting a factor node number and a variable node number, and obtaining an index vector; S23: based on the message vector and the index vector, constructing a first calculation matrix and a second calculation matrix for calculating the message passed from the variable node to the factor node; S24: based on the message vector, constructing a third calculation matrix and a fourth calculation matrix for calculating a log-likelihood ratio of the variable node; S25: splicing the first calculation matrix, the second calculation matrix, the third calculation matrix, and the fourth calculation matrix to obtain a complete calculation matrix; S26: taking an inverse of a non-zero element value of the complete calculation matrix to obtain a resistance state configuration matrix; wherein a zero value of the resistance state configuration matrix corresponds to a high resistance state, and a non-zero value of the resistance state configuration matrix corresponds to a corresponding resistance state; S27: based on the resistance state configuration matrix, configuring a resistance state value of the memristor cross array to obtain a configured memristor cross array; S28: converting an image observation value of the binary image into a voltage signal and loading the voltage signal to an input end of the configured memristor cross array to obtain a to-be-run memristor cross array; S3: performing voltage-current calculation and conversion through the to-be-run memristor cross array to obtain a variable node message and a log-likelihood ratio; S4: based on the variable node message and the log-likelihood ratio, performing iterative judgment to output a denoised image.

2. The MRF image denoising method based on the storage-computing integrated device according to claim 1, characterized in that, The step S1 further comprises: S11: performing binaryzation processing on an input to-be-denoised image to obtain a binary image; S12: creating a variable node for each pixel point of the binary image to obtain a variable node set; introducing a data factor for each pixel point of the binary image to obtain a data factor set; and creating a smoothing factor for a connection relationship between each two adjacent pixel pairs of the binary image to obtain a smoothing factor set; S13: arranging the variable node set, the data factor set, and the smoothing factor set into a sparse matrix to obtain an initial connection matrix; S14: deleting factor nodes of the data factor set in the initial connection matrix, taking an observation value of a pixel point as an initial prior of a corresponding variable node, and obtaining the connection matrix.

3. The MRF image denoising method based on the storage-computing integrated device according to claim 2, characterized in that, The step S13 further comprises: S131: numbering factor nodes through a row index to obtain a matrix row coordinate; and numbering variable nodes through a column index to obtain a matrix column coordinate; S132: value the matrix row coordinates and the matrix column coordinates based on the adjacency relationship, if the factor node and the variable node are connected, the element is 1, if the factor node and the variable node are not connected, the element is 0, and an initial connection matrix is obtained.

4. The MRF image denoising method based on the compute-in-memory device according to claim 1, wherein, The first calculation matrix is used for weighted summation of the passing message from the factor node to the variable node; and the second calculation matrix is used for loading the initial prior log-likelihood ratio corresponding to the pixel point observation value into the message calculation from the variable node to the factor node. The third calculation matrix is used for weighted summation of the passing message from the factor node to the variable node; and the fourth calculation matrix is used for loading the initial prior log-likelihood ratio corresponding to the pixel point observation value into the calculation of the log-likelihood ratio of the variable node.

5. The MRF image denoising method based on the compute-in-memory device according to claim 1, wherein, The step S28 further includes: S281: arranging the image observation values of the binary image in a preset order to obtain an observation value vector; S282: converting the observation value vector into an input voltage signal; S283: loading the input voltage signal to a specified row of the input end of the configured memristor cross array, and setting the initial state of the remaining input rows of the configured memristor cross array to 0, to obtain a to-be-run memristor cross array.

6. The MRF image denoising method based on the compute-in-memory device according to claim 1, wherein, The step S3 further includes: S31: performing current calculation based on the resistance state values of the plurality of memristor units in the to-be-run memristor cross array and the input voltage, to obtain a plurality of memristor unit currents; S32: superimposing the plurality of memristor unit currents on the same column line based on the Kirchhoff's current law, to obtain an output current; S33: converting the output current into an output voltage signal through a current-voltage conversion circuit; S34: extracting a first part voltage corresponding to the variable node message and a second part voltage corresponding to the log-likelihood ratio from the output voltage signal, to obtain the variable node message and the log-likelihood ratio.

7. The MRF image denoising method based on the compute-in-memory device according to claim 1, wherein, The step S4 further includes: S41: performing polarity decision on the log-likelihood ratio, determining the first pixel value when the log-likelihood ratio is less than 0, and determining the second pixel value when the log-likelihood ratio is greater than 0, to obtain a current iteration decision result; S42: calculating the number of flips between the current iteration decision result and the last iteration decision result; S43: determining whether the number of flips is less than a preset threshold, outputting the current iteration decision result as a denoised image if the number of flips is less than the preset threshold, and performing adjacent bit exchange processing on the variable node message if the number of flips is not less than the preset threshold, to obtain an updated message, and loading the updated message to the input end of the memristor cross array again for iteration calculation.

8. The MRF image denoising method based on the storage-computing integrated device according to claim 7, characterized in that, In the step S41, the first pixel value is -1, corresponding to a black pixel; and the second pixel value is +1, corresponding to a white pixel.

9. A system for MRF image denoising based on a memory compute unit, comprising: The factor graph construction module is configured to construct a binary factor graph based on the four-neighborhood topology of the binary image, and obtain a connection matrix based on a plurality of factor nodes and variable nodes; The resistance state configuration module is configured to construct a calculation matrix based on the connection matrix, and configure resistance state values of a plurality of memristor units in the memristor cross array according to the calculation matrix. ​ The blocking state configuration module is further used for: based on the connection matrix, vector arranging the passing messages from the variable nodes to the factor nodes and the passing messages from the factor nodes to the variable nodes to obtain a message vector; based on the connection matrix, extracting the factor node numbers and the variable node numbers to obtain an index vector; according to the message vector and the index vector, constructing a first calculation matrix and a second calculation matrix for calculating the passing messages from the variable nodes to the factor nodes; based on the message vector, constructing a third calculation matrix and a fourth calculation matrix for calculating the log-likelihood ratios of the variable nodes; The first calculation matrix, the second calculation matrix, the third calculation matrix and the fourth calculation matrix are spliced to obtain a complete calculation matrix; The reciprocal of the non-zero element values of the complete calculation matrix is taken to obtain a blocking state configuration matrix; wherein the zero value of the blocking state configuration matrix corresponds to a high resistance state, and the non-zero value of the blocking state configuration matrix corresponds to a corresponding resistance state; based on the blocking state configuration matrix, the resistance state values of the memristor cross array are configured, and the image observation values of the binary image are converted into voltage signals and loaded to the input end of the memristor cross array; The memristor cross array is arranged in a row-column cross manner by a plurality of memristor units, and after receiving the resistance state values configured by the blocking state configuration module, the memristor cross array is used for receiving an input voltage and performing voltage-current calculation according to the configured resistance state values; The current-voltage converter is connected to the column line output end of the memristor cross array, and is used for converting the output current into a voltage signal; The iterative control module is used for iterative judgment based on the variable node messages and the log-likelihood ratios output by the current-voltage converter, and if the termination condition is met, a denoised image is output, otherwise the updated messages are reloaded to the memristor cross array; wherein the iterative control module is realized by a digital-analog hybrid circuit or an analog circuit, and the digital-analog hybrid circuit is realized by a digital circuit cooperating with an analog-digital converter and a digital-analog converter.

Images