Image data denoising and completion method, device, equipment and medium

Abstract

The present application relates to a method, apparatus, device, and medium for denoising and completing image data. The method includes: capturing the low-rank attributes of the image data to be denoised and completed based on an indication set of observable items corresponding to the image data to be denoised and completed based on a unified tensor network model, and decomposing the image data to be denoised and completed into multiple ring factor tensors and a core factor tensor; constructing a minimization model in the unified tensor network model, and updating each ring factor tensor and core factor tensor in the minimization model based on a proximal alternating minimization algorithm to continuously iterate the unified tensor network model until the unified tensor network model reaches a converged state; inputting the image data to be denoised and completed into the unified tensor network model that has reached a converged state, and determining the denoised and completed image data corresponding to the image data to be denoised and completed, so as to complete the denoising and completion of the image data. The present application can better preserve the detailed information of the image and improve the quality of image restoration.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image data denoising and completion method, a corresponding device, an electronic device, and a computer-readable storage medium. Background Art

[0002] Image sensors have developed rapidly in recent years, and while sales continue to rise, so too has the demand for image quality. One of the most important metrics is the signal-to-noise ratio. Due to hardware, environmental, and human influences, images inevitably contain noise and other interference, which significantly affects image detail and ultimately image quality. Images, with their large information content, long transmission distances, and high transmission speeds, have become a vital source and means of information acquisition for humans. However, during the generation, transmission, and capture processes, images are inevitably contaminated by noise, which can easily result in incomplete, partially missing, or damaged images.

[0003] Image completion, one of the most important technologies for low-level image understanding, has garnered widespread attention from both academia and industry. It frequently features as a key topic at international image-related conferences and is a crucial research topic across the fields of computer vision and image processing. Image completion involves using technical means to restore a clear image from damaged image data, based on information about the image's internal structural associations. Specific challenges include image denoising, image super-resolution, and JPEG compression restoration. Image completion is not just a single low-level image restoration problem; it also serves as a breakthrough in many high-level image semantic research areas, making it an integral part of image understanding.

[0004] In order to adapt to the existing technology, image data is inevitably contaminated by noise during the generation, transmission and shooting process, which may easily lead to technical problems such as incomplete images, partially defective images or partially damaged images. The applicant has made corresponding explorations to solve this problem. Summary of the Invention

[0005] The purpose of this application is to solve the above problems and provide an image data denoising and completion method, a corresponding device, an electronic device and a computer-readable storage medium.

[0006] In order to meet the various objectives of this application, this application adopts the following technical solutions:

[0007] An image data denoising and completion method proposed to meet one of the purposes of this application includes the following steps:

[0008] Acquire the image data to be denoised and completed, and determine an indication set of corresponding observable items according to the image data to be denoised and completed;

[0009] capturing low-rank properties of the to-be-denoised completed image data according to an indication set of observable items corresponding to the to-be-denoised completed image data based on a unified tensor network model, and decomposing the to-be-denoised completed image data into a plurality of ring factor tensors and a core factor tensor;

[0010] Constructing a minimization model in the unified tensor network model, and updating each of the ring factor tensors and the core factor tensor in the minimization model based on a proximal alternating minimization algorithm to continuously iterate the unified tensor network model until the unified tensor network model reaches a convergence state;

[0011] The image data to be denoised and completed is input into the unified tensor network model that has reached a converged state, and the denoised and completed image data corresponding to the image data to be denoised and completed is determined to complete the denoising and completion of the image data.

[0012] Optionally, the step of updating each of the ring factor tensors and the core factor tensor in the minimization model based on the proximal alternating minimization algorithm comprises the following steps:

[0013] Responding to a ring factor tensor update instruction, obtaining each ring factor tensor in the unified tensor network model;

[0014] Iterating the subproblem corresponding to the ring factor tensor to determine a matrix of the subproblem corresponding to the ring factor tensor after iteration;

[0015] The closed-form solution corresponding to the subproblem is determined according to the matrix of the subproblem corresponding to the ring factor tensor after iteration, so as to complete the update of the ring factor tensor.

[0016] Optionally, the step of updating each of the ring factor tensors and the core factor tensor in the minimization model based on the proximal alternating minimization algorithm comprises the following steps:

[0017] In response to a core factor tensor update instruction, obtaining a core factor tensor in the unified tensor network model;

[0018] Iterating the subproblem corresponding to the core factor tensor to determine a vector representation of the subproblem corresponding to the core factor tensor after iteration;

[0019] The closed-form solution corresponding to the sub-problem is determined according to the vectorized representation of the sub-problem corresponding to the core factor tensor after iteration, so as to complete the update of the core factor tensor.

[0020] Optionally, the subproblem corresponding to the ring factor tensor is

[0021]

[0022] Among them, the It is the N+1 order ring factor tensor of the unified tensor network model, and ρ is a proximal parameter.

[0023] The matrix of the subproblem corresponding to the ring factor tensor is:

[0024]

[0025] Among them, ρ is a proximal parameter, is the ring tensor The modulo-n expansion matrix of . is a tensor The modulo-n expansion matrix of . It is the tensor obtained by shrinking all the factor tensors except the nth ring factor tensor. The modulo-n expansion matrix of .

[0026] The closed-form solution corresponding to the subproblem is:

[0027]

[0028] Where I represents the identity matrix.

[0029] Optionally, the sub-problem corresponding to the core factor tensor is

[0030]

[0031] Among them, ρ is a proximal parameter, n∈[1,N] is the N+1 order ring factor tensor of the unified tensor network model; is the N-order core factor tensor of the unified tensor network model; (R 1,2 ,…,R n-1,n ,L1,…,L N ) is the rank of the unified tensor network model;

[0032] The vectorized representation of the subproblem corresponding to the core factor tensor

[0033]

[0034] in, and Represents tensors respectively and the core factor tensor Vectorized operations, It is the tensor after all ring factor matrices are contracted except the core factor tensor. The modulo-1 expansion of

[0035] The vectorized representation of the subproblem corresponding to the core factor tensor is:

[0036]

[0037] Where I represents the identity matrix.

[0038] Optionally, the unified tensor network model is

[0039]

[0040] Among them, the is the low-rank tensor to be recovered; n∈[1,N] is the N+1 order ring factor tensor of the unified tensor network model; is the N-order core factor tensor of the unified tensor network model; (R 1,2 ,…,R n-1,n ,L1,…,L N ) is the rank of the unified tensor network model.

[0041] Optionally, the minimization model is

[0042]

[0043] in, is a projection function that preserves the indicated set of observable entries while forcing all other values ​​to zero.

[0044] An image data denoising and completion device provided for another purpose of the present application includes:

[0045] an indication set determination module, configured to obtain the to-be-denoised and completed image data, and determine an indication set of corresponding observable items according to the to-be-denoised and completed image data;

[0046] an image data decomposition module configured to capture the low-rank properties of the to-be-denoised and completed image data based on a unified tensor network model according to an indication set of observable items corresponding to the to-be-denoised and completed image data, and decompose the to-be-denoised and completed image data into a plurality of ring factor tensors and a core factor tensor;

[0047] a minimization model construction module, configured to construct a minimization model in the unified tensor network model, and update each of the ring factor tensors and the core factor tensor in the minimization model based on a proximal alternating minimization algorithm to continuously iterate the unified tensor network model until the unified tensor network model reaches a convergence state;

[0048] The image data denoising and completion module is configured to input the image data to be denoised and completed into the unified tensor network model that has reached a converged state, determine the denoised and completed image data corresponding to the image data to be denoised and completed, so as to complete the denoising and completion of the image data.

[0049] An electronic device provided to meet another purpose of the present application includes a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the image data denoising and completion method described in the present application.

[0050] A computer-readable storage medium is provided to meet another purpose of the present application, which stores a computer program implemented according to the image data denoising and completion method in the form of computer-readable instructions. When the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

[0051] Compared with the existing technology, this application addresses the technical problem that image data in the existing technology is inevitably contaminated by noise during the generation, transmission and shooting process, which easily leads to incomplete images, partial image damage or partial image damage. This application performs image denoising and completion based on a unified tensor network model, which has but not limited to the following beneficial effects:

[0052] First, the unified tensor network model (UTN model) of this application can be expressed as any other tensor network model, such as a tensor train, a tensor ring, a fully connected tensor network, a tensor wheel, etc., thus having better generalization and representation capabilities and permutation dimension invariance. No matter how the target tensor pattern is permuted, the UTN decomposition remains essentially unchanged.

[0053] Secondly, the present application provides an image denoising and completion method based on a unified tensor network model. The method can denoise and complete incomplete images. The connection method of the unified tensor network model is more likely to retain image data detail information, and can better retain image data detail information, making the underlying image data detail information easier to propagate in the network. The final restored image has clear image details, thereby improving the quality and effect of image restoration.

[0054] Third, in the process of denoising and completing image data, the computational complexity is small, which greatly reduces the computational complexity of image denoising and completion processing and is relatively simple to implement, thereby restoring noisy and incomplete image data to clean, noise-free complete image data, and has better image denoising and completion performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0056] Figure 1 Schematic diagram of the process of the image data denoising and completion method in an embodiment of the present application;

[0057] Figure 2 This is a structural diagram of a unified tensor network model and other tensor network models in an embodiment of the present application;

[0058] Figure 3 This is a schematic diagram of the process of updating each ring factor tensor in an embodiment of the present application;

[0059] Figure 4 This is a schematic diagram of the process of updating the core factor tensor in an embodiment of the present application;

[0060] Figure 5 This is a principle block diagram of the image data denoising and completion device in an embodiment of the present application;

[0061] Figure 6 Schematic diagram of the structure of the computer device in the embodiment of the present application. DETAILED DESCRIPTION

[0062] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limiting the present application.

[0063] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0064] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0065] It will be understood by those skilled in the art that the terms "client," "terminal," and "terminal device" as used herein include both devices that are wireless signal receivers, i.e., devices that only have wireless signal receivers without transmission capabilities, and devices that have receiving and transmitting hardware capable of two-way communication over a two-way communication link. Such devices may include: cellular or other communication devices such as personal computers and tablet computers, which have single-line displays, multi-line displays, or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service), which may combine voice, data processing, fax, and / or data communication capabilities; PDA (Personal Digital Assistant), which may include a radio frequency receiver, a pager, Internet / Intranet access, a web browser, a notepad, a calendar, and / or a GPS (Global Positioning System) receiver; and conventional laptop and / or palmtop computers or other devices, which have and / or include a radio frequency receiver. As used herein, the terms "client," "terminal," or "terminal device" may be portable, transportable, or installed in a vehicle (air, sea, and / or land), or may be adapted and / or configured to operate locally and / or in a distributed manner at any other location on Earth and / or in space. As used herein, the terms "client," "terminal," or "terminal device" may also refer to a communication terminal, an Internet terminal, or a music / video playback terminal, such as a PDA, an MID (Mobile Internet Device), and / or a mobile phone with music / video playback capabilities, or may include a smart TV, a set-top box, or other device.

[0066] The hardware referred to by names such as "server", "client", and "service node" in this application is essentially an electronic device with capabilities equivalent to those of a personal computer. It is a hardware device that has the necessary components revealed by the von Neumann principle, such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. Computer programs are stored in its memory, and the central processing unit loads the program stored in the external memory into the internal memory for execution, executes the instructions in the program, and interacts with the input and output devices to complete specific functions.

[0067] It should be noted that the concept of "server" referred to in this application can also be extended to server clusters. Based on the network deployment principles understood by those skilled in the art, the servers described should be logically divided. In physical space, these servers can be independent of each other but callable through interfaces, or integrated into a single physical computer or a computer cluster. Those skilled in the art should understand this flexibility and should not use it to constrain the implementation of the network deployment method of this application.

[0068] Unless expressly specified, one or more technical features of the present application can be deployed on a server for implementation and accessed by a client through a remote call to obtain an online service interface provided by the server, or can be directly deployed and run on a client for implementation.

[0069] Unless expressly specified otherwise, the neural network models referenced or may be referenced in this application may be deployed on a remote server and remotely called on the client, or may be deployed and directly called on a client with sufficient device capabilities. In some embodiments, when it runs on the client, its corresponding intelligence may be obtained through transfer learning to reduce the requirements for the client's hardware operating resources and avoid excessive occupation of the client's hardware operating resources.

[0070] Unless explicitly specified, the various data involved in this application can be stored remotely on a server or on a local terminal device, as long as they are suitable for being called by the technical solution of this application.

[0071] Those skilled in the art should be aware that although the various methods of this application are described based on the same concept and thus exhibit commonality, unless otherwise specified, these methods can be independently executed. Similarly, the various embodiments disclosed in this application are all based on the same inventive concept. Therefore, concepts with the same expression, as well as concepts that are appropriately transformed for convenience despite different expression, should be understood as equivalent.

[0072] Unless expressly stated to be mutually exclusive, the various embodiments disclosed in this application may be cross-combined with the relevant technical features of the various embodiments to flexibly construct new embodiments, as long as such combination does not deviate from the creative spirit of this application and can meet the needs of the prior art or resolve certain deficiencies in the prior art. Those skilled in the art should be aware of such flexibility.

[0073] Based on the above example scenarios, please refer to Figure 1 In one embodiment, the image data denoising and completion method of the present application includes the following steps:

[0074] Step S10: Acquire the image data to be denoised and completed, and determine an indication set of corresponding observable items according to the image data to be denoised and completed;

[0075] In real life, image data is easily affected by sensor failure and other reasons during the acquisition process, so the acquired image data is usually incomplete and easily interfered by noise. The terminal device can obtain the image data to be denoised and completed locally or remotely. The image data to be denoised and completed can be various types of image data, such as incomplete and noisy image data, etc., to obtain incomplete and noisy N-order image data. When , an indication set Ω of observable items corresponding to the to-be-denoised and complemented image data can be determined.

[0076] Step S20: capturing the low-rank attribute of the to-be-denoised and completed image data according to an indication set of observable items corresponding to the to-be-denoised and completed image data based on a unified tensor network model, and decomposing the to-be-denoised and completed image data into a plurality of ring factor tensors and a core factor tensor;

[0077] For further information, see Figure 2 After determining the indicator set Ω of the observable items corresponding to the to-be-denoised and completed image data, since the acquired to-be-denoised and completed image data is incomplete and noisy, it is necessary to capture the low-rank properties of the to-be-denoised and completed image data through limited observation points in the to-be-denoised and completed image data, so as to reconstruct the clean image data. This can be based on a unified tensor network model (UTN model). The unified tensor network model (UTN model) decomposes an original N-order tensor data into N third-order ring factor tensors and an N-order core factor tensor, wherein each ring factor tensor is connected in pairs, and a core factor tensor is used to capture the connection between all ring factor tensors, such as Figure 2 As shown, from Figure 2It can be seen that the unified tensor network model can be expressed as any other tensor network model, such as tensor train, tensor ring, fully connected tensor network, tensor wheel, etc., and thus has a better generalization and representation capability.

[0078] The mathematical expression of the unified tensor network model is:

[0079]

[0080] in, is the low-rank tensor to be recovered, n∈[1,N] is the N+1 order ring factor tensor of the unified tensor network model, is the N-order core factor tensor of the unified tensor network model, (R 1,2 ,…,R n-1,n ,L1,…,L N ) is the rank of the unified tensor network model. Similarly, we define is a tensor The modulo-n expansion matrix of , and the others are similar. For the convenience of representation, we use Represents the unified tensor network model.

[0081] Step S30: constructing a minimization model in the unified tensor network model, and updating each of the ring factor tensors and the core factor tensor in the minimization model based on the proximal alternating minimization algorithm to continuously iterate the unified tensor network model until the unified tensor network model reaches a convergence state;

[0082] Furthermore, after decomposing the image data to be denoised and completed into multiple ring factor tensors and a core factor tensor based on the unified tensor network model, a minimization model in the unified tensor network model is constructed. According to the unified tensor network model, the following minimization model can be established:

[0083]

[0084] in, is a projection function that keeps entries indicating that the set Ω is observable while forcing all other values ​​to zero.

[0085] Since the minimization problem is convex for each variable, each variable can be updated using the proximal alternating minimization algorithm as follows:

[0086] For each ring factor tensor n∈[1,N], at the t+1th iteration, the subproblem corresponding to the ring factor tensor can be expressed as follows:

[0087]

[0088] The subproblem corresponding to the ring factor tensor can be converted into a matrix form as follows:

[0089]

[0090] Among them, ρ is a proximal parameter. is the ring factor tensor The modulo-n expansion matrix of . is a tensor The modulo-n expansion matrix of . It is the tensor obtained by shrinking all the factor tensors except the nth ring factor tensor. The modulo-n expansion matrix of .

[0091] The sub-question corresponding to the ring factor tensor can obviously obtain a closed-form solution, which is expressed as follows:

[0092]

[0093] Here, I represents the identity matrix, which we refold into the ring factor tensor after updating the subproblem

[0094] For the core factor tensor At the t+1th iteration, the subproblem corresponding to the core factor tensor can be expressed as follows:

[0095]

[0096] The subproblem corresponding to the core factor tensor can be converted into a vectorized form as follows:

[0097]

[0098] in, and Represents tensors respectively and the core factor tensor Vectorized operations, It is the tensor after all ring factor matrices are contracted except the core factor tensor. The modulo-1 expansion of .

[0099] The subproblem corresponding to the core factor tensor can also be solved in a closed form as follows:

[0100]

[0101] After updating the subproblem, fold it back into the core factor tensor

[0102] After updating each of the ring factor tensors and the core factor tensor, the unified tensor network model is continuously iterated until the unified tensor network model reaches a convergence state, and the update The sub-problem of can be expressed as follows at the t+1th iteration:

[0103]

[0104] Wherein, the Ω c represents the complement of the indicator set Ω.

[0105] Step S40: input the image data to be denoised and completed into the unified tensor network model that has reached a converged state, determine the denoised and completed image data corresponding to the image data to be denoised and completed, and complete the denoising and completion of the image data.

[0106] At this point, the update process of all variables is completed. After multiple iterations, the algorithm can obtain an optimal solution, and the unified tensor network model reaches a convergence state, thereby obtaining clean and complete tensor data. The image data to be denoised and completed is input into the unified tensor network model that has reached a converged state, and the denoised and completed image data corresponding to the image data to be denoised and completed is determined, that is, the noise-free and complete image data is determined to complete the denoising and completion of the image data.

[0107] As can be seen from the above embodiments, compared with the prior art, the present application addresses the technical problem in the prior art that image data is inevitably contaminated by noise during the generation, transmission and shooting process, which easily leads to incomplete images, partial image damage or partial image damage. The present application performs image denoising and completion based on a unified tensor network model, which has but is not limited to the following beneficial effects:

[0108] First, the unified tensor network model (UTN model) of this application can be expressed as any other tensor network model, such as a tensor train, a tensor ring, a fully connected tensor network, a tensor wheel, etc., thus having better generalization and representation capabilities and permutation dimension invariance. No matter how the target tensor pattern is permuted, the UTN decomposition remains essentially unchanged.

[0109] Secondly, the present application provides an image denoising and completion method based on a unified tensor network model. The method can denoise and complete incomplete images. The connection method of the unified tensor network model is more likely to retain image data detail information, and can better retain image data detail information, making the underlying image data detail information easier to propagate in the network. The final restored image has clear image details, thereby improving the quality and effect of image restoration.

[0110] Third, in the process of denoising and completing image data, the computational complexity is small, which greatly reduces the computational complexity of image denoising and completion processing and is relatively simple to implement, thereby restoring noisy and incomplete image data to clean, noise-free complete image data, and has better image denoising and completion performance.

[0111] Based on any embodiment of this application, please refer to Figure 3 The step of updating each of the ring factor tensors and the core factor tensor in the minimization model based on the proximal alternating minimization algorithm comprises the following steps:

[0112] Step S301: Responding to a ring factor tensor update instruction, obtaining each ring factor tensor in the unified tensor network model;

[0113] Step S303: Iterate the subproblem corresponding to the ring factor tensor to determine the matrix of the subproblem corresponding to the ring factor tensor after the iteration;

[0114] Step S305: Determine a closed-form solution corresponding to the sub-problem according to the matrix of the sub-problem corresponding to the ring factor tensor after iteration, so as to complete the update of the ring factor tensor.

[0115] Specifically, the subproblem corresponding to the ring factor tensor is

[0116]

[0117] Among them, the It is the N+1 order ring factor tensor of the unified tensor network model, and ρ is a proximal parameter.

[0118] The matrix of the subproblem corresponding to the ring factor tensor is:

[0119]

[0120] Among them, ρ is a proximal parameter, is the ring tensor The modulo-n expansion matrix of . is a tensor The modulo-n expansion matrix of . It is the tensor obtained by shrinking all the factor tensors except the nth ring factor tensor. The modulo-n expansion matrix of .

[0121] The closed-form solution corresponding to the subproblem is:

[0122]

[0123] Where I represents the identity matrix.

[0124] Based on any embodiment of this application, please refer to Figure 4 The step of updating each of the ring factor tensors and the core factor tensor in the minimization model based on the proximal alternating minimization algorithm comprises the following steps:

[0125] Step S3001: Responding to a core factor tensor update instruction, obtaining a core factor tensor in the unified tensor network model;

[0126] Step S3003: iterate the subproblem corresponding to the core factor tensor to determine the vector representation of the subproblem corresponding to the core factor tensor after the iteration;

[0127] Step S3005: Determine the closed-form solution corresponding to the sub-problem according to the vectorized representation of the sub-problem corresponding to the iterated core factor tensor to complete the update of the core factor tensor.

[0128] Specifically, the sub-problem corresponding to the core factor tensor is

[0129]

[0130] Among them, ρ is a proximal parameter, is the N+1 order ring factor tensor of the unified tensor network model; is the N-order core factor tensor of the unified tensor network model; (R 1,2 ,…,R n-1,n ,L1,…,L N ) is the rank of the unified tensor network model;

[0131] The vectorized representation of the subproblem corresponding to the core factor tensor is:

[0132]

[0133] in, and Represents tensors respectively and the core factor tensor Vectorized operations, It is the tensor after all ring factor matrices are contracted except the core factor tensor. The modulo-1 expansion of

[0134] The vectorized representation of the subproblem corresponding to the core factor tensor is:

[0135]

[0136] Where I represents the identity matrix.

[0137] See also Figure 5 An image data denoising and completion device provided to meet one of the purposes of this application includes an indication set determination module 1100, an image data decomposition module 1200, a minimization model construction module 1300 and an image data denoising and completion module 1400. Among them, the indicator set determination module 1100 is configured to obtain the image data to be denoised and complete, and determine the indicator set of the observable items corresponding to the image data to be denoised based on the indicator set of the observable items; the image data decomposition module 1200 is configured to capture the low-rank properties of the image data to be denoised based on the indicator set of the observable items corresponding to the image data to be denoised, and decompose the image data to be denoised into multiple ring factor tensors and a core factor tensor based on the unified tensor network model; the minimization model construction module 1300 is configured to construct a minimization model in the unified tensor network model, and update each ring factor tensor and the core factor tensor in the minimization model based on the proximal alternating minimization algorithm to continuously iterate the unified tensor network model until the unified tensor network model reaches a converged state; the image data denoising and completion module 1400 is configured to input the image data to be denoised into the unified tensor network model that has reached a converged state, determine the denoised completed image data corresponding to the image data to be denoised, and complete the denoising and completion of the image data.

[0138] Based on any embodiment of this application, please refer to Figure 6 Another embodiment of the present application further provides an electronic device, which can be implemented by a computer device, such as Figure 6 As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions, and the database may store a control information sequence, and when the computer-readable instructions are executed by the processor, the processor may implement an image data denoising and completion method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor may execute the image data denoising and completion method of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art will understand that Figure 6The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0139] In this embodiment, the processor is used to execute Figure 5 The memory stores the program code and various data required to execute the specific functions of each module and its submodule in the image data denoising and completion device. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program code and data required to execute all modules / submodules in the image data denoising and completion device of the present application. The server can call the server's program code and data to execute the functions of all submodules.

[0140] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the image data denoising and completion method described in any embodiment of the present application.

[0141] The present application also provides a computer program product, including a computer program / instruction, which, when executed by one or more processors, implements the steps of the image data denoising and completion method described in any embodiment of the present application.

[0142] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments of the method. The aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0143] The above description is only part of the implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

[0144] In summary, the present application provides an image denoising and completion method based on a unified tensor network model, which can denoise and complete incomplete images. The connection method of the unified tensor network model is easier to retain the detailed information of the image data, and can better retain the detailed information of the image data, making the underlying image data detailed information easier to propagate in the network. The final restored image has clear image details, thereby improving the quality and effect of image restoration.

Claims

1. A method for denoising and completing image data, characterized in that: The steps include: Acquire the image data to be denoised and completed, and determine an indication set of corresponding observable items according to the image data to be denoised and completed; Capturing the low-rank attribute of the to-be-denoised and completed image data according to an indication set of observable items corresponding to the to-be-denoised and completed image data based on a unified tensor network model, decomposing the to-be-denoised and completed image data into a plurality of ring factor tensors and a core factor tensor, wherein the unified tensor network model is: Among them, the is the low-rank tensor to be recovered; n∈[1,N] is the N+1 order ring factor tensor of the unified tensor network model; is the N-order core factor tensor of the unified tensor network model; (R 1,2 ,…,R n-1,n ,L1,…,L N ) is the rank of the unified tensor network model; Constructing a minimization model in the unified tensor network model, and updating each of the ring factor tensors and the core factor tensor in the minimization model based on a proximal alternating minimization algorithm to continuously iterate the unified tensor network model until the unified tensor network model reaches a convergence state; wherein the minimization model is: in, is a projection function that preserves the indicated set of observable entries while forcing all other values ​​to zero; The image data to be denoised and completed is input into the unified tensor network model that has reached a converged state, and the denoised and completed image data corresponding to the image data to be denoised and completed is determined to complete the denoising and completion of the image data.

2. The image data denoising and completion method according to claim 1, characterized in that: The step of updating each of the ring factor tensors and the core factor tensor in the minimization model based on the proximal alternating minimization algorithm comprises the following steps: Responding to a ring factor tensor update instruction, obtaining each ring factor tensor in the unified tensor network model; Iterating the subproblem corresponding to the ring factor tensor to determine a matrix of the subproblem corresponding to the ring factor tensor after iteration; The closed-form solution corresponding to the subproblem is determined according to the matrix of the subproblem corresponding to the ring factor tensor after iteration, so as to complete the update of the ring factor tensor.

3. The image data denoising and completion method according to claim 1, wherein: The step of updating each of the ring factor tensors and the core factor tensor in the minimization model based on the proximal alternating minimization algorithm comprises the following steps: In response to a core factor tensor update instruction, obtaining a core factor tensor in the unified tensor network model; Iterating the subproblem corresponding to the core factor tensor to determine a vector representation of the subproblem corresponding to the core factor tensor after iteration; The closed-form solution corresponding to the sub-problem is determined according to the vectorized representation of the sub-problem corresponding to the core factor tensor after iteration, so as to complete the update of the core factor tensor.

4. The image data denoising and completion method according to claim 2, wherein: The subproblem corresponding to the ring factor tensor is Among them, the n∈[1,N] is the N+1 order ring factor tensor of the unified tensor network model, and ρ is a proximal parameter; The matrix of the subproblem corresponding to the ring factor tensor is: Among them, ρ is a proximal parameter, is the ring tensor The modulo-n expansion matrix of ; is a tensor The modulo-n expansion matrix of ; It is the tensor obtained by shrinking all the factor tensors except the nth ring factor tensor. The modulo-n expansion matrix of ; The closed-form solution corresponding to the subproblem is: Where I represents the identity matrix.

5. The image data denoising and completion method according to claim 3, wherein: The subproblem corresponding to the core factor tensor is Among them, ρ is a proximal parameter, n∈[1,N] is the N+1 order ring factor tensor of the unified tensor network model; is the N-order core factor tensor of the unified tensor network model; (R 1,2 ,…,R n-1,n ,L1,…,L N ) is the rank of the unified tensor network model; The vectorized representation of the subproblem corresponding to the core factor tensor is: Where I represents the identity matrix, in, and Represents tensors respectively and the core factor tensor Vectorized operations, It is the tensor after all ring factor matrices are contracted except the core factor tensor. The modulo-1 expansion of .

6. An image data denoising and completion device, characterized in that: include: an indication set determination module, configured to obtain the to-be-denoised and completed image data, and determine an indication set of corresponding observable items according to the to-be-denoised and completed image data; An image data decomposition module is configured to capture the low-rank properties of the image data to be denoised and completed according to an indication set of observable items corresponding to the image data to be denoised and completed based on a unified tensor network model, and decompose the image data to be denoised and completed into multiple ring factor tensors and a core factor tensor; wherein the unified tensor network model is: Among them, the is the low-rank tensor to be recovered; n∈[1,N] is the N+1 order ring factor tensor of the unified tensor network model; is the N-order core factor tensor of the unified tensor network model; (R 1,2 ,…,R n-1,n ,L1,…,L N ) is the rank of the unified tensor network model; A minimization model construction module is configured to construct a minimization model in the unified tensor network model, and update each of the ring factor tensors and the core factor tensor in the minimization model based on a proximal alternating minimization algorithm to continuously iterate the unified tensor network model until the unified tensor network model reaches a convergence state; wherein the minimization model is: in, is a projection function that preserves the indicated set of observable entries while forcing all other values ​​to zero; The image data denoising and completion module is configured to input the image data to be denoised and completed into the unified tensor network model that has reached a converged state, determine the denoised and completed image data corresponding to the image data to be denoised and completed, so as to complete the denoising and completion of the image data.

7. An electronic device comprising a central processing unit and a memory, characterized in that: The central processing unit is configured to call and run a computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that It stores a computer program implemented according to the method described in any one of claims 1 to 5 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images