Image noise detection method, neural network model training method and device

Abstract

The present application discloses an image noise detection method, a neural network model training method and a device, which are used to solve the problem that traditional image noise detection methods are unable to handle the complex noise existing in real images. The method provided by the present application includes: training a neural network model according to a training sample set to obtain a trained neural network model. Noise detection is performed on an image to be detected based on the trained network model, including: obtaining an image to be detected, wherein the image to be detected includes multiple sub-image areas; performing feature extraction on the multiple sub-image areas to obtain image feature vectors corresponding to the multiple sub-image areas respectively; and determining the degree of compression noise corresponding to the multiple sub-image areas respectively according to the image feature vectors corresponding to the multiple sub-image areas respectively.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image noise detection method, a neural network model training method and a device. Background Art

[0002] With the development of digital multimedia technology, image compression technology has been widely used in the storage, transmission, and display of digital images to improve data transmission rates and storage efficiency. Conventional methods for detecting compressed images often produce compression noise when displayed, significantly impacting the user experience of display devices. Traditional methods for detecting image compression noise are mostly based on mathematical modeling. However, this approach is limited by the complexity of the noise and can only detect simple noise. However, in real life, image compression noise is complex and often occurs in more than one type of noise. Therefore, mathematical modeling-based methods cannot achieve effective detection results in real-world applications. Furthermore, mathematical modeling can only detect one type of noise in an image and output the noise level for the entire image. However, real-life images contain multiple types of noise, and the resulting noise complexity is complex when multiple noises are superimposed. Mathematical modeling-based methods are therefore incapable of detecting noise in such images. Summary of the Invention

[0003] The embodiments of the present application provide an image noise detection method, a neural network model training method and a device to solve the problem that traditional image noise detection methods are unable to handle the complex noise existing in real images.

[0004] In a first aspect, the present application provides a method for training a neural network model, comprising:

[0005] Acquire a training sample set, wherein the training sample set includes a plurality of compressed noise images and compressed noise levels corresponding to a plurality of sub-image regions included in each compressed noise image;

[0006] Inputting a first compressed noise image included in the training sample set into the neural network model to obtain predicted compressed noise levels corresponding to a plurality of sub-image regions included in the first compressed image, wherein the first compressed noise image is any one of the plurality of compressed noise images;

[0007] Calculating a loss value according to the compression noise levels respectively corresponding to the plurality of sub-image regions included in the training sample set and the predicted compression noise level;

[0008] The network parameters of the neural network model are adjusted according to the loss value to obtain a trained neural network model.

[0009] Based on the above solution, the trained neural network model can be used to perform image noise detection on the image to be detected, and then the compression noise levels corresponding to the multiple sub-image areas included in the image to be detected are obtained.

[0010] In one possible implementation, the first sub-image area in the first compressed noise image is obtained by compressing the sub-image area corresponding to the first sub-image area in the first original noise-free image by using a first compression algorithm with set compression parameters, and the compression noise level of the first sub-image area is determined based on the set compression parameters; or, the second sub-image area in the second compressed noise image is obtained by compressing the sub-image area corresponding to the second sub-image area in the second original noise-free image by continuously using multiple compression algorithms, wherein the second compressed noise image is any one of the multiple compressed noise images, and the compression noise level of the second sub-image area is obtained by weighting the compression levels corresponding to the multiple compression algorithms.

[0011] Based on this approach, different compression noise levels are selected for different sub-image regions of the input image, significantly increasing data diversity. A single image can contain multiple levels of compression noise, effectively simulating the complex compression noise found in real-world images. Furthermore, the entire process does not require manual labeling, saving both manpower and time.

[0012] In one possible implementation, the first compressed image includes N sub-image areas, where N is a positive integer, and the neural network model includes an encoder network and a fully connected layer. The first compressed noise image included in the training sample set is input into the neural network model to obtain predicted compression noise levels corresponding to the multiple sub-image areas included in the first compressed image, including: performing feature extraction on the N sub-image areas through the encoder network to obtain image feature vectors corresponding to the N sub-image areas; determining the probability of each of the N sub-image areas corresponding to multiple compression noise levels based on the image feature vectors corresponding to the N sub-image areas through the fully connected layer, the compression noise level corresponding to each sub-image area being the compression noise level corresponding to the maximum generation probability among the generation probabilities of each sub-image area corresponding to the multiple compression noise levels.

[0013] In one possible implementation, calculating the loss value based on the compression noise levels corresponding to the plurality of sub-image regions included in the first compressed noise image included in the training sample set and the predicted compression noise level includes: comparing the compression noise levels corresponding to the N sub-image regions included in the first compressed noise image in the training sample set with the compression noise levels corresponding to the N sub-image regions included in the first compressed noise image determined according to the neural network model to obtain N comparison results;

[0014] Determining, according to the N comparison results, that N sub-image regions respectively correspond to N loss values;

[0015] The weighted result obtained by weighting the N loss values adjusts the network parameters of the neural network; the weight used for the kth loss value satisfies the condition shown in the following formula:

[0016] w k =1-(|P′ (k) | / max(|P′|);

[0017] Among them, w k Represents the weight used for the loss value corresponding to the kth sub-image area, P′ (k) represents the probability of generating the compression noise level corresponding to the k-th sub-image area determined according to the neural network model, and max(|P′|) represents the maximum value of the probability of generating the compression noise levels corresponding to the N sub-image areas.

[0018] Based on this approach, a cross-entropy loss function is calculated between the predicted compression noise level for each sub-image region and the actual compression noise level for that sub-image region. This transforms the detection problem into a classification problem and solves it using a classic classification loss function. The loss function for the entire image is the weighted sum of the loss functions for all sub-image regions, ensuring that compression noise detection is learned for the entire image. Furthermore, this method prevents the network from overly focusing on sub-image regions with higher prediction probabilities, promoting equal learning across all sub-image regions.

[0019] In a second aspect, an embodiment of the present application provides an image noise detection method, comprising:

[0020] Acquire an image to be detected, the image to be detected including a plurality of sub-image areas; perform feature extraction on each of the plurality of sub-image areas to obtain image feature vectors corresponding to each of the plurality of sub-image areas; and determine the compression noise levels corresponding to each of the plurality of sub-image areas based on the image feature vectors corresponding to each of the plurality of sub-image areas.

[0021] Based on the above scheme, complex noise in an image can be detected, and the detection result of local noise in the image can be given, thereby improving the detection accuracy of image compression noise.

[0022] In a possible implementation, performing feature extraction on each of the multiple sub-image regions includes:

[0023] Feature extraction is performed on each of the multiple sub-image areas through a neural network model; wherein the neural network model is obtained through the training method of the neural network model described in the first aspect.

[0024] Based on the above scheme, the image features can be extracted through the neural network model, which has a larger receptive field.

[0025] In a possible implementation, the method further includes: performing denoising processing on the image to be detected according to compression noise levels corresponding to the multiple sub-image regions.

[0026] Based on the above solution, denoising is performed on multiple sub-image regions according to the obtained local compression noise levels, which can improve the denoising effect.

[0027] In a third aspect, an embodiment of the present application provides an electronic device, comprising:

[0028] A communication interface for receiving a training sample set;

[0029] A processor is configured to obtain the training sample set from the communication interface to execute the method described in the first aspect and different implementations of the first aspect.

[0030] In a fourth aspect, an embodiment of the present application provides a chip system comprising a memory and a processor; the processor and the memory are coupled; wherein the memory comprises program instructions, and when the program instructions are executed by the processor, the chip system executes any one of the methods in the first and second aspects.

[0031] In a fifth aspect, an embodiment of the present application provides a display device, including:

[0032] A controller, configured to execute the method described in the second aspect and different implementations of the second aspect;

[0033] The display is used to display the image after the compression noise is removed.

[0034] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer executes the method described in the first aspect and any possible implementation method of the first aspect.

[0035] In addition, the technical effects brought about by any implementation method of the third to sixth aspects can be referred to the technical effects brought about by the first aspect, the second aspect, and different implementation methods of the first aspect and the second aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0037] Figure 1 A schematic diagram of a usage scenario of a display device provided in an embodiment of the present application;

[0038] Figure 2 A configuration block diagram of a control device 100 provided in an embodiment of the present application;

[0039] Figure 3 A hardware configuration block diagram of a display device 200 provided in an embodiment of the present application;

[0040] Figure 4 A schematic diagram of the software architecture of a terminal device provided in an embodiment of the present application;

[0041] Figure 5A A schematic diagram of the system architecture provided in an embodiment of the present application;

[0042] Figure 5B Another system architecture diagram provided for an embodiment of the present application;

[0043] Figure 6 A schematic diagram of the electronic device structure provided in an embodiment of the present application;

[0044] Figure 7 A schematic diagram of a flow chart of an image noise detection method provided in an embodiment of the present application;

[0045] Figure 8 A schematic diagram of a process for constructing a training sample set provided in an embodiment of the present application;

[0046] Figure 9 A schematic diagram of segmentation of an original uncompressed image provided in an embodiment of the present application;

[0047] Figure 10 Another schematic diagram of segmentation of an original uncompressed image provided in an embodiment of the present application;

[0048] Figure 11A schematic diagram of a compression noise level table provided in an embodiment of the present application;

[0049] Figure 12 A schematic diagram of another compression noise level table provided in an embodiment of the present application;

[0050] Figure 13 A schematic diagram of the compression noise levels corresponding to multiple sub-image areas of a compressed noise image provided in an embodiment of the present application;

[0051] Figure 14 A schematic diagram of a training process of a neural network model provided in an embodiment of the present application;

[0052] Figure 15 A flowchart of a neural network model training method provided in an embodiment of the present application;

[0053] Figure 16 A flowchart of another neural network model training provided in an embodiment of the present application;

[0054] Figure 17 A schematic diagram of the training process of another neural network model provided in an embodiment of the present application;

[0055] Figure 18 A schematic diagram of the structure of a neural network model provided in an embodiment of the present application;

[0056] Figure 19 This is a schematic diagram of the output of a neural network model provided in an embodiment of the present application;

[0057] Figure 20 A schematic diagram of the structure of another neural network model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0059] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present application.

[0060] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0061] The following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios described below are only used to illustrate the embodiments of the present application and are not limiting. In specific implementation, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0062] The image noise detection method and neural network model training method provided in the embodiments of the present application can be implemented by an execution device. In some embodiments, the execution device can be a terminal device. The terminal device can be a display device with a display function. Display devices can include: smart TVs, mobile phones, tablet computers, etc.

[0063] The following describes the structure and application scenarios of the execution device by taking the execution device as a display device as an example. Figure 1 Schematic diagram of the use scenario of the display device in the embodiment. Figure 1 As shown, the display device 200 can also communicate data with the server 400, and the user can operate the display device 200 through the smart device 300 or the control device 100. In one possible example, the server 400 can execute the training method of the neural network model and provide it to the display device 200, and the display device 200 can execute the image noise detection method. In another possible example, a server other than the server 400 can execute the training method of the neural network model and provide it to the display device 200. The image noise detection method can be executed by the server 400 or by the display device 200.

[0064] In some embodiments, the control device 100 may be a remote controller, and communication between the remote controller and the display device may include infrared protocol communication, Bluetooth protocol communication, or at least one other short-range communication method, to control the display device 200 wirelessly or wiredly. The user may control the display device 200 by inputting user commands through at least one of buttons on the remote controller, voice input, and control panel input.

[0065] In some embodiments, the smart device 300 may include any one of a mobile terminal, a tablet computer, a computer, a laptop computer, an AR / VR device, etc.

[0066] In some embodiments, the smart device 300 may also be used to control the display device 200. For example, the display device 200 may be controlled using an application running on the smart device.

[0067] In some embodiments, the smart device 300 and the display device 200 may also be used to communicate data.

[0068] In some embodiments, the display device 200 can also be controlled in a manner other than the control device 100 and the smart device 300. For example, the display device 200 can directly receive the user's voice command control through a module for obtaining voice commands configured inside the display device 200, or it can receive the user's voice command control through a voice control device set outside the display device 200.

[0069] In some embodiments, the display device 200 also communicates data with the server 400. The display device 200 may be connected to a local area network (LAN), a wireless local area network (WLAN), or other networks. The server 400 may provide various content and interactions to the display device 200. The server 400 may be a single cluster or multiple clusters, and may include one or more types of servers.

[0070] In some embodiments, a software step executed by a step execution entity can be transferred to another step execution entity in data communication with the step execution entity as needed. For example, a software step executed by a server can be transferred to a display device in data communication with the server as needed, and vice versa.

[0071] Figure 2 Schematically shows a block diagram of the configuration of the control device 100 according to an exemplary embodiment. Figure 2 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive user input operation commands and convert the operation commands into commands that the display device 200 can recognize and respond to, acting as an interaction intermediary between the user and the display device 200.

[0072] In some embodiments, the communication interface 130 is used for external communication and includes at least one of a WIFI chip, a Bluetooth module, NFC, or an alternative module.

[0073] In some embodiments, the user input / output interface 140 includes at least one of a microphone, a touch pad, a sensor, a button, or alternative modules.

[0074] The embodiment will be described in detail below using the display device 200 as an example. It should be understood that Figure 3 The display device 200 shown is only an example, and the display device 200 may have more Figure 3 The more or less components shown in the figure can be combined with two or more components, or can have different component configurations. The various components shown in the figure can be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and / or application specific integrated circuits.

[0075] Figure 3 FIG. 2 is a block diagram showing a hardware configuration of the display device 200 according to an exemplary embodiment.

[0076] In some embodiments, the display apparatus 200 includes at least one of a tuner 210 , a communicator 220 , a detector 230 , an external device interface 240 , a controller 250 , a display 260 , an audio output interface 270 , a memory, a power supply, and a user interface.

[0077] In some embodiments, the controller includes a central processing unit, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and first to nth interfaces for input / output.

[0078] In some embodiments, the display 260 includes a display screen component for presenting images, and a driving component for driving image display, a component for receiving image signals output from a controller, and a component for displaying video content, image content, and a menu control interface, as well as a user control UI interface, etc.

[0079] In some embodiments, the display 260 may be at least one of a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.

[0080] In some embodiments, the tuner-demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0081] In some embodiments, the communicator 220 is a component used to communicate with external devices or servers using various communication protocols. For example, the communicator may include at least one of a Wi-Fi module, a Bluetooth module, a wired Ethernet module, or other network communication protocol chip or a near-field communication protocol chip, as well as an infrared receiver. The display device 200 can use the communicator 220 to send and receive control signals and data signals with the control device 100 or the server 400.

[0082] In some embodiments, detector 230 is used to collect signals from the external environment or external interactions. For example, detector 230 may include a light receiver (not shown) for collecting ambient light intensity. Alternatively, detector 230 may include an image collector, such as a camera, for collecting external environmental scenes, user attributes, or user interaction gestures. Alternatively, detector 230 may include a sound collector, such as a microphone, for receiving external sounds.

[0083] In some embodiments, the external device interface 240 may include, but is not limited to, any one or more of the following: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by multiple of the above interfaces.

[0084] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0085] In some embodiments, controller 250 controls the operation of the display device and responds to user operations via various software control programs stored in memory. Controller 250 controls the overall operation of display device 200. For example, in response to receiving a user command to select a UI object for display on display 260, controller 250 may perform operations related to the object selected by the user command.

[0086] In some embodiments, the object may be any selectable object, such as a hyperlink, an icon, or other operable control. Operations associated with the selected object include: displaying a page, document, or image connected to the hyperlink, or executing a program corresponding to the icon.

[0087] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM Random Access Memory (RAM), ROM (Read-Only Memory, ROM), a first interface to an nth interface for input / output, a communication bus (Bus), etc.

[0088] The CPU processor is the control center of the display device 200, including the system-on-chip SOC, such as Figure 3 As shown, the CPU is used to execute operating system and application instructions stored in memory, as well as various interactive instructions received from external input, to execute various applications, data, and content, ultimately displaying and playing various audio and video content. The CPU processor can include multiple processors, such as a main processor and one or more sub-processors.

[0089] In some embodiments, a graphics processor is configured to generate various graphical objects, such as at least one of icons, operation menus, and graphics displayed in response to user input commands. The graphics processor includes an arithmetic unit that performs operations based on various user input interactive commands and displays various objects according to display attributes; and a renderer that renders the various objects generated by the arithmetic unit. The rendered objects are then displayed on a display.

[0090] In some embodiments, the video processor is used to receive an external video signal and perform at least one of the following video processing operations, such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis, according to the standard codec protocol of the input signal, to obtain a signal that can be directly displayed or played on the display device 200.

[0091] In some embodiments, the video processor includes at least one of a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, and a display formatting module. The demultiplexing module is used to demultiplex the input audio and video data stream. The video decoding module is used to process the demultiplexed video signal, including decoding and scaling. The image synthesis module, such as an image synthesizer, is used to superimpose and mix the GUI signal generated by the graphics generator according to user input or itself with the scaled video image to generate an image signal for display. The frame rate conversion module is used to convert the input video frame rate. The display formatting module is used to receive the frame rate converted video output signal and change the signal to conform to the display format signal, such as outputting an RGB data signal.

[0092] In some embodiments, the audio processor is used to receive an external audio signal, perform decompression and decoding according to the standard codec protocol of the input signal, and perform at least one of noise reduction, digital-to-analog conversion, and amplification processing to obtain a sound signal that can be played in a speaker.

[0093] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command through a specific sound or gesture, and the user input interface may recognize the sound or gesture through a sensor to receive the user input command.

[0094] In some embodiments, a "user interface" is a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. A common form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It can be an interface element such as an icon, window, or control displayed on the display screen of an electronic device, where a control can include at least one of a visual interface element such as an icon, button, menu, tab, text box, dialog box, status bar, navigation bar, or widget.

[0095] In some embodiments, the user interface 280 is an interface for receiving control input (eg, a physical button on the display device body, or other interfaces).

[0096] In some embodiments, the system of the display device may include a kernel, a command parser (shell), a file system, and applications. The kernel, shell, and file system together form the basic operating system structure, which allows users to manage files, run programs, and use the system. After power is turned on, the kernel starts, activates kernel space, abstracts hardware, initializes hardware parameters, runs and maintains virtual memory, schedulers, signals, and inter-process communication (IPC). After the kernel starts, the shell and user applications are loaded. After startup, the application is compiled into machine code to form a process.

[0097] See also Figure 4 In some embodiments, the system is divided into four layers, from top to bottom: the application layer (referred to as the "application layer"), the application framework layer (referred to as the "framework layer"), the Android runtime and system library layer (referred to as the "system runtime library layer"), and the kernel layer.

[0098] In some embodiments, at least one application runs in the application layer. These applications can be window programs, system settings programs, clock programs, etc. that come with the operating system, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0099] The framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes predefined functions. The application framework layer acts as a processing center, determining the actions taken by applications in the application layer. Through the API, applications can access system resources and services during execution.

[0100] like Figure 4 As shown, in the embodiment of the present application, the application framework layer includes managers, content providers, etc., wherein the manager includes at least one of the following modules: an activity manager (ActivityManager) is used to interact with all activities running in the system; a location manager (Location Manager) is used to provide system services or applications with access to system location services; a package manager (Package Manager) is used to retrieve various information related to the application packages currently installed on the device; a notification manager (NotificationManager) is used to control the display and clearing of notification messages; a window manager (Window Manager) is used to manage icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0101] In some embodiments, the activity manager is used to manage the lifecycle of each application and common navigation back functions, such as controlling the exit, opening, and backing of an application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, taking screenshots, and controlling display window changes (such as shrinking the display window, shaking the display, distorting the display, etc.).

[0102] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0103] In some embodiments, the kernel layer is a layer between hardware and software. Figure 4As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0104] The display device 200 in the embodiment of the present application is not limited to a smart TV, but may also be an electronic device such as a mobile phone or a tablet computer.

[0105] In other embodiments, the execution device may be an electronic device, which may be implemented by one or more servers, which may be local servers or cloud servers. Figure 5A As shown, the server 500 can be implemented as a physical server or a virtual server. The server can be implemented as a single server or a server cluster consisting of multiple servers. The image noise detection method or neural network model training method provided in this application can be implemented by a single server or a server cluster. Figure 5A In the example, the server 500 is connected to the terminal device 600 and the display device 200. The server 500 can execute the training method of the neural network model. In some scenarios, the server 500 can also receive the image compression noise detection task sent by the terminal device 600, or send the image noise detection result to the terminal device 600. In other scenarios, the server 500 can also receive the image noise detection task sent by the display device 200 and perform image noise detection and denoising, or display the image after the compression noise is removed through the display device 200. Figure 5B As shown, the server 500 is connected to the display device 200 as an example. The server 500 can execute the training method of the neural network model. In some scenarios, the server 500 can receive the image noise detection task sent by the display device, and perform noise detection according to the image noise task. In one example, the service device can send the noise detection result to the display device, and the display device performs denoising according to the received noise detection result, and displays the denoised image. In another example, the server can denoise the image according to the noise detection result, and send the image after removing the compressed noise to the display device, so that the display device 200 displays the received image.

[0106] The electronic device may also be a personal computer, a handheld or laptop device, a mobile device such as a mobile phone, a tablet computer, a personal digital assistant, etc.

[0107] As an example, see Figure 6 As shown, the electronic device may include a processor 510 and a communication interface 520. The electronic device may also include a memory 530. Of course, the electronic device may also include other components. Figure 6 Not shown in the figure.

[0108] The communication interface 520 is used to communicate with the display device, to receive an image compression noise detection task sent by the display device, or to send an image compression noise detection result to an electronic device.

[0109] In the embodiments of the present application, the processor 510 may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present application may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0110] The processor 510 is the control center of the electronic device. It connects the various parts of the electronic device using various interfaces and routes. It performs various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 530 and calling data stored in the memory 530. Optionally, the processor 510 may include one or more processing units. The processor 510 may be, for example, a processor, a microprocessor, a controller, or other control component. For example, it may be a general-purpose central processing unit (CPU), a general-purpose processor, a digital signal processing (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.

[0111] The memory 530 can be used to store software programs and modules. The processor 510 executes various functional applications and data processing by running the software programs and modules stored in the memory 530. The memory 530 may mainly include a program storage area and a data storage area. The program storage area may store an operating system, at least one application required for a function, etc.; the data storage area may store data created based on business processing, etc. The memory 530 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs, and modules. The memory 530 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (RAM), a static random access memory (SRAM), a programmable read-only memory (PROM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic storage device, a magnetic disk, an optical disk, and the like. The memory 530 is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. The memory 530 in the embodiment of the present application may also be a circuit or any other device capable of performing a storage function, for storing program instructions and / or data.

[0112] It should be noted that the above Figure 1-6 The structure shown is only an example and is not limited in the embodiment of the present invention.

[0113] The present invention provides a method for detecting image noise. Figure 7 The process of the image noise detection method is exemplarily shown, and the process can be executed by an execution device, which can be Figure 3 The display device 200 shown in FIG. 2 may specifically perform image noise detection via the controller 250 in the display device 200. Alternatively, the execution device may be Figure 6 The electronic device shown can specifically perform image noise detection through the processor 510 in the electronic device. The specific process is as follows:

[0114] 201, obtaining an image to be detected.

[0115] In some embodiments, the image to be detected may include multiple sub-image areas, and each sub-image area has the same size.

[0116] 202 : Perform feature extraction on each of the multiple sub-image regions to obtain image feature vectors corresponding to each of the multiple sub-image regions.

[0117] In some embodiments, multiple sub-image regions may be input into a neural network model, and features may be extracted from each of the multiple sub-image regions using the neural network model. The neural network model may include an encoder layer, which performs feature extraction on each of the multiple sub-image regions to obtain image feature vectors corresponding to each of the multiple sub-image regions in the image to be detected.

[0118] 203 : Determine compression noise levels corresponding to the multiple sub-image regions respectively according to the image feature vectors corresponding to the multiple sub-image regions respectively.

[0119] In some embodiments, the compression noise level corresponding to each sub-image region may be determined based on the image feature vector corresponding to the sub-image region, thereby obtaining the compression noise levels corresponding to multiple image regions in the image to be detected.

[0120] In some embodiments, before the compression noise level of the image to be detected is detected, a compression noise level range is pre-set. The compression noise level range can be manually set, and this application does not specifically limit this. For example, the compression noise level range can be set to 0-S, that is, the compression noise level range corresponds to S+1 compression noise levels. The neural network model can also include a fully connected layer, which can determine an array of generation probabilities for each sub-image region corresponding to multiple compression noise levels based on image feature vectors corresponding to multiple sub-image regions. The compression noise level of each sub-image region is the compression noise level corresponding to the maximum generation probability among the multiple compression noise level generation probabilities for each sub-image region. Continuing with the above example, when the image to be detected includes N sub-image regions, the fully connected layer will output N generation probability arrays, each generation probability array representing the generation probability of S+1 compression noise levels for the sub-image region corresponding to the array. The compression noise level corresponding to the maximum generation probability among the S+1 compression noise level generation probabilities for each sub-image region is the compression noise level of that sub-image region.

[0121] In some embodiments, after obtaining multiple compression noise levels corresponding to multiple sub-image regions of an image to be detected, denoising processing can be performed on the image to be detected based on the compression noise levels corresponding to the multiple sub-image regions. Specifically, the multiple sub-image regions of the image to be detected correspond to multiple compression noise levels, and different denoising processing methods can be selected based on the different compression noise levels. The multiple sub-image regions can be processed separately using the different denoising processing methods to achieve a precise denoising effect.

[0122] This neural network model can detect complex noise in images and provide local noise detection results, thereby improving the accuracy of image compression noise detection. Furthermore, the local noise detection results can provide accurate noise level information to the denoising algorithm, allowing denoising to be performed on the local noise, and then the image after the compression noise is removed can be displayed on the display device.

[0123] The following is an explanation of the training process of the neural network model mentioned above. The neural network model is trained through a training sample set. The training sample set includes multiple compressed noise images and the compression noise levels corresponding to the multiple sub-image areas included in each compressed noise image in the multiple compressed noise images, and then the neural network model is trained through the training sample set. When constructing the training sample set, the multiple original noise-free images can be compressed separately to obtain the compression noise levels corresponding to the multiple compressed noise images and the multiple sub-image areas included in the multiple compressed noise images. The compression noise levels corresponding to the multiple compressed noise images and the multiple sub-image areas included in the multiple compressed noise images constitute the training sample set. It can be understood that a compressed noise image and the corresponding compression noise level are a sample. The following describes the process of obtaining multiple first compressed noise images with different compression noise levels corresponding to the first original noise-free image by taking the first original noise-free image among the multiple original noise-free images as an example, see Figure 8 shown.

[0124] 301 : Acquire a first original noise-free image from a plurality of original noise-free images, and divide the first original noise-free image into a plurality of sub-image regions.

[0125] In some embodiments, after obtaining the first original noise-free image, the first original noise-free image may be equally divided into a plurality of sub-image regions. For example, the first original noise-free image may be equally divided into m×n sub-image regions I(i,j), i∈[0,m), j∈[0,n). As an example, Figure 9As shown, the resolution of the first original noise-free image is 512×256, and the first original noise-free image can be divided into 64×32 sub-image areas, and the size of each sub-image area is 8×8, 8=512 / 64=256 / 32, then each sub-image area corresponds to a compression noise level, that is, 8×8 sub-image areas correspond to the same compression noise level. The resolution of the sub-image area will affect the resolution of the final noise detection result. The embodiment of the present application does not specifically limit the number of sub-image areas into which the first original noise-free image is divided. For example, the resolution of the first original noise-free image is 512×256, and the first original noise-free image can be divided into 128×64 sub-image areas, then the size of each sub-image area is 4×4, and each 4×4 sub-image area corresponds to a compression noise level, as shown in FIG. Figure 10 It can be understood that the more sub-image regions the first original noise-free image is divided into, the more accurate the compression noise detection result is.

[0126] 302 : Determine compression noise levels corresponding to a plurality of sub-image regions in a first original noise-free image.

[0127] In some embodiments, corresponding random compression noise levels can be selected for each of the multiple sub-image areas. In some scenarios, after the compression noise levels are randomly selected for all sub-image areas, the compression noise levels corresponding to the multiple sub-image areas can be saved in the form of a compression noise level table. For example, the first original noise-free image includes m×n sub-image areas, and the compression noise level q can be randomly selected for the m×n sub-image areas, and the compression noise levels q corresponding to the m×n sub-image areas can be saved in a compression noise level table Q, where each element in Q represents the compression noise level of the sub-image area, such as Figure 11 In some scenarios, the compression noise level table can be saved in the form of a table, such as Figure 12 shown.

[0128] In other embodiments, when selecting the random compression noise level, the selection can be based on a preset compression noise level range. For example, the compression noise level range can be 0 to 10, where 0 represents no compression noise, 1 represents the weakest compression noise level, and 10 represents the strongest compression noise level. In this case, q can be selected from a compression noise level of 0-10. It is understood that the image compression noise level range can also be other ranges, for example, a larger compression noise level range can be selected, such as 0-100, 0-1000, etc., and this application does not specifically limit this.

[0129] In some embodiments, after randomly selecting compression noise levels for a plurality of sub-image regions included in the first original noise-free image, the first original noise-free image includes at least two different compression noise levels.

[0130] 303 : Perform image compression on the multiple sub-image regions according to compression noise levels corresponding to the multiple sub-image regions, to obtain a first compressed noise image.

[0131] In some embodiments, the first sub-image region in the first compressed noisy image is obtained by compressing a sub-image region corresponding to the first sub-image region in the first original noise-free image using a set compression algorithm. The set compression algorithm corresponding to the first sub-image region is determined based on the compression noise level corresponding to the first sub-image region. It is understood that when different sub-image regions have different compression noise levels, different compression algorithms are used when compressing the sub-image regions.

[0132] In other embodiments, the first sub-image region in the first compressed noise image is obtained by compressing the sub-image region corresponding to the first sub-image region in the first original noise-free image using a first compression algorithm with set compression parameters, and the set compression parameters corresponding to the first sub-image region are determined based on the compression noise level corresponding to the first sub-image region. For example, the first original noise-free image includes m×n sub-image regions, and the compression noise level value Q(i,j) corresponding to each sub-image region can be selected according to the compression noise level table Q, i∈[0,m),j∈[0,n), and then image compression is performed on each sub-image region separately. The compressed noise image satisfies the conditions shown in the following formula:

[0133] O(i,j)=JPEG(I(i,j),Q(i,j));

[0134] Wherein, I(i,j) represents the sub-image area at position (i,j) in the first original noise-free image, Q(i,j) represents the compressed noise degree at position (i,j) in the compressed noise degree table Q, JPEG represents the image compression algorithm, and O(i,j) represents the compressed noise image of the sub-image area at position (i,j) in the first compressed noise image.

[0135] In some other embodiments, when compressing the original uncompressed image, multiple compression algorithms may be used to obtain a compressed noise image. Taking the compressed second noise image corresponding to the second original uncompressed image as an example, the second sub-image region in the second compressed noise image is obtained by continuously compressing the corresponding sub-image region in the second original noise-free image using multiple compression algorithms. The degree of compressed noise in the second sub-image region is obtained by weighting the compression degrees corresponding to the multiple compression algorithms. The second compressed noise image satisfies the conditions shown in the following formula:

[0136] O(i,j)=α1*JPEG1(I(i,j),Q(i,j))+α2*JPEG2(I(i,j),Q(i,j))+…+α n *JPEGn(I(i,j),Q(i,j))

[0137] Where I(i,j) represents the sub-image region at position (i,j) in the second original noise-free image, Q(i,j) represents the compressed noise level at position (i,j) in the compressed noise level table Q, i∈[0,m), j∈[0,n), JPEG1…JPEGn represent different image compression algorithms, O(i,j) is the compressed noise image of the sub-image region at position (i,j) in the second compressed noise image, α1…α n Represents the weights corresponding to different image compression algorithms, and α1+α2+…+α n =1.

[0138] It should be noted that at least two sub-image regions use different compression algorithm sets.

[0139] In some embodiments, the compression noise levels corresponding to the multiple sub-image areas included in the original noise-free image are all 0. After the compression noise level corresponding to each sub-image area is determined, the multiple sub-image areas can be compressed separately according to the compression noise level to obtain a compressed noise image. For example, the first original noise-free image includes m×n sub-image areas, and the compression noise level corresponding to each sub-image area is 0. After the compression noise corresponding to each sub-image area is determined, the sub-image area is compressed to obtain a first compressed noise image. For example, if the compression noise level corresponding to the first sub-image area in the first original image is 2, then the first sub-image area is compressed with a compression noise level of 2; if the compression noise image corresponding to the sub-image area in the 1st row and 2nd column of the first original noise-free image is 7, then the sub-image area in the 1st row and 2nd column is compressed with a compression noise level of 7. It can be understood that the image is compressed according to the compression noise levels corresponding to the multiple sub-image areas, thereby obtaining the first compressed noise image. The compression noise level corresponding to each sub-image in the first compressed noise image is as follows: Figure 13 shown.

[0140] This approach allows for image compression of multiple sub-image regions within the original noise-free image to varying degrees of compression noise, enhancing data diversity. Furthermore, a single compressed noise image can contain multiple levels of compression noise, better reflecting the complex compression noise present in the displayed image. Furthermore, constructing the training sample set eliminates the need for manual labeling, saving both manpower and time.

[0141] In some embodiments, after obtaining the training sample set, the neural network model can be trained according to the training sample set. The training process is as follows: Figure 14 Shown, including:

[0142] 401, obtain a training sample set.

[0143] The training sample set includes multiple samples, each of which includes a compressed noise image and compression noise levels corresponding to multiple sub-image regions included in the compressed noise image. Taking the first sample in the training sample set as an example, the neural network model is trained using the first compressed noise image in the first sample and the compression noise levels corresponding to the multiple sub-image regions included in the first compressed noise image. The training process is shown in steps 402-404.

[0144] 402 : Input the first compressed noise image included in the training sample set into the neural network model to obtain predicted compression noise levels corresponding to the multiple sub-image areas included in the first compressed image.

[0145] In some embodiments, the first compressed noise image can be input into a neural network model for feature extraction, and the generation probability of multiple compressed noise levels corresponding to each of the multiple sub-image areas included in the first compressed noise image is output. The compressed noise level corresponding to each sub-image area is the compressed noise level corresponding to the maximum generation probability among the generation probabilities of the multiple compressed noise levels corresponding to each sub-image area. The compressed noise levels corresponding to the multiple sub-images included in the first compressed noise image determined by the neural network model can be represented by the compressed noise detection result, such as Figure 15 As shown. Taking the example of a first compressed image including N sub-image regions, the first compressed noise image can be input into the neural network model, so that the neural network model outputs the generation probabilities of the first sub-image region corresponding to multiple compression noise levels among the N sub-image regions included in the first compressed noise image, and the compression noise level corresponding to the first sub-image region is the compression noise level corresponding to the maximum generation probability among the multiple compression noise levels corresponding to the first sub-image region. The first sub-image region is any sub-image region among the N sub-image regions, and N is a positive integer.

[0146] 403 : Calculate a loss value according to the compression noise levels corresponding to the multiple sub-image regions included in the training sample set and the predicted compression noise levels.

[0147] In some embodiments, the loss value of the first compressed noise image can be determined based on the detection result of the first compressed noise image output by the neural network model and the compression noise levels corresponding to the multiple sub-image areas included in the first compressed noise image, such as Figure 15 As shown. Taking the example of a first compressed noise image comprising N sub-image regions, the compression noise levels corresponding to the N sub-image regions of the first compressed noise image can be compared with the compression noise levels corresponding to the N sub-image regions of the first compressed noise image determined according to the neural network model to obtain N comparison results. Subsequently, N loss values corresponding to the N sub-image regions can be determined based on the N comparison results. Furthermore, the loss value corresponding to the first compressed noise image can be determined based on the N loss values corresponding to the N sub-image regions.

[0148] 404 , adjusting the network parameters of the neural network model according to the loss value to obtain a trained neural network model.

[0149] In some embodiments, when the loss value is less than or equal to the set value, the network parameters are saved and the training ends, such as Figure 15 If the loss value is greater than the set value, then repeat steps 402-404 until the loss value is less than or equal to the set value, save the network parameters of the trained neural network model, and end the training. Figure 16 shown.

[0150] In some embodiments, before executing step 401, step 401A may be further included, such as Figure 17 As shown, the details are as follows:

[0151] 401A, constructing a neural network model and initializing network parameters of the neural network model.

[0152] In some embodiments, a neural network model is first constructed and network parameters of the neural network model are initialized. The neural network model can be used to extract image features.

[0153] The structure of the neural network model in the embodiment of the present application is described below. It should be understood that the structure of the following neural network model is only an example and does not specifically limit the architecture of the neural network model. In some embodiments, the network structure of the neural network model may include an encoder network and a fully connected layer, such as Figure 18As shown in Figure 2 . The encoder network extracts features from multiple sub-image regions in the image to be detected, obtaining image feature vectors corresponding to each of the sub-image regions. Subsequently, a fully connected layer can be used to determine the probability of each sub-image region corresponding to multiple compression noise levels based on the image feature vectors corresponding to the multiple sub-image regions. For example, if an image includes m×n sub-image regions, the fully connected layer will output an m×n generation probability array, each representing the probability of that sub-image region corresponding to multiple compression noise levels. For example, if the compression noise level ranges from 0 to 10, the generation probability array output by the fully connected layer for the sub-image region at position (i, j) corresponding to different compression noise levels can be expressed as (0.02, 0.05, 0.69, 0.06, 0.02, 0.04, 0.03, 0.02, 0.01, 0.04, 0.02), where i∈[0,m] and j∈[0,n]. The index value of each generation probability in the array is the compression noise level value corresponding to that generation probability. For example, if the index of 0.05 in the array is 1, then 1 is the compression noise level corresponding to the generation probability of 0.05 (i.e., 0.05 is the generation probability of a compression noise level of 1). In some scenarios, the compression noise level corresponding to each sub-image area can be determined based on the generation probability array output by the fully connected layer. Specifically, the compression noise level corresponding to the maximum generation probability in the generation probability array can be used as the compression noise level corresponding to the sub-image area. Continuing with the above example, the sub-image area at position (i, j) corresponds to a plurality of compression noise levels, and the maximum generation probability in the generation probability array is 0.69. Then the compression noise level corresponding to 0.69 is used as the compression noise level of the sub-image area at position (i, j). The index of the maximum generation probability of 0.69 in the generation probability array is 2. Then 2 is the compression noise level corresponding to the generation probability of 0.69, that is, the compression noise level of the sub-image area at position (i, j) is 2. Furthermore, the compression noise levels corresponding to the plurality of sub-image areas included in the image to be detected can be determined by a neural network model, and the compression noise levels can be output in the form of a compression noise level table, see. Figure 19 Among them, the form of the image compression noise level table can be found in Figure 12 .

[0154] In the embodiment of the present application, a loss function can be used to calculate the loss value.

[0155] In a possible example, when the compressed noise image includes m×n sub-image regions, when calculating the loss value of the compression noise degree, the loss value may be determined by a loss function, and the loss function satisfies the following conditions:

[0156]

[0157] Among them, Q′(i,j) Indicates the compression noise level of the sub-image region at position (i, j) predicted by the neural network model. The predicted compression noise level can be expressed in the range of 0 to 10, where 0 represents no compression noise, 1 represents the weakest compression noise level, and 10 represents the strongest compression noise level. (i,j) Indicates the degree of compression noise of the sub-image region at position (i, j). The degree of compression noise of the real image can also be expressed in the range of 0 to 10, where 0 represents no compression noise, 1 represents the weakest compression noise, and 10 represents the strongest compression noise. (i,j) *log(Q′ (i,j) ) represents the cross entropy loss function of the compression noise detection result of the sub-image area at position (i, j), Represents the sum of the cross entropy functions of multiple sub-image regions included in the compressed noisy image.

[0158] Based on this approach, we calculate the cross-entropy loss function by combining the predicted compression noise level for each sub-image region with the actual compression noise level for that sub-image region. This transforms the detection problem into a classification problem and solves it using a classic classification loss function. The loss function for the entire image is the sum of the loss functions for all sub-image regions, ensuring that compression noise detection learning is performed across the entire image.

[0159] In another possible example, the loss value of the compressed noisy image can be obtained by weighted summing the loss values of each sub-image region passing through the loss function. For example, the loss function satisfies the conditions shown in the following formula:

[0160]

[0161] Among them, Q′ (i,j) Indicates the degree of compression noise of the sub-image area at position (i, j) output by the neural network model, Q (i,j) Indicates the degree of real image compression noise in the sub-image area at position (i, j), w (i,j) Indicates the weight used for the loss value of the sub-image area at position (i, j), -Q (i,j) *log(Q′ (i,j) ) represents the cross entropy loss function of the noise compression degree of the sub-image area at position (i, j), It represents the weighted sum of the cross entropy functions of all sub-image regions. The compressed noise image includes m×n sub-image regions.

[0162] For example, the weight used for the loss value corresponding to the sub-image region at position (i, j) satisfies the following formula:

[0163] w (i,j) =1-(|P′ ((i,j))| / max(|P′|);

[0164] Among them, w (i,j) Represents the weight used for the loss value corresponding to the sub-image area at position (i, j), P′ (i,j) It represents the probability of generating the degree of compressed noise corresponding to the sub-image area at position (i, j) determined by the neural network model, and max(|P′|) represents the maximum value of the probability of generating the degree of compressed noise corresponding to the m×n sub-image areas included in the compressed noise image.

[0165] In some scenarios, when the compressed noise image includes N sub-image regions, the sub-image region at position (i, j) can be represented as the kth sub-image region, and the loss function can also be expressed as Among them, the weight used for the loss value corresponding to the kth sub-image area can be expressed as w k =1-(|P′ (k) | / max(|P′|).

[0166] Through the above formula, the neural network model will not focus too much on the sub-image areas with larger prediction probabilities, and can promote equal learning of all sub-image areas.

[0167] In some other embodiments, the network structure of the neural network model may include an encoder network and a decoder network, such as Figure 20 As shown. The encoder network is responsible for mapping the image to be detected to a high-dimensional feature space, and obtaining image feature vectors of multiple sub-image areas in the high-dimensional feature space. The decoder is responsible for mapping the high-dimensional space back to a low-dimensional space, that is, reducing the dimensionality of the high-dimensional image feature vector to obtain a low-dimensional feature image. As an example, when the image to be detected includes m×n sub-image areas, the last layer of the decoder can be set to output a two-dimensional feature map of size [m, n]. Afterwards, each pixel value in the two-dimensional feature map output by the decoder is normalized based on the compression noise level range to obtain the compression noise level corresponding to each sub-image area in the image to be detected. For example, the compression noise level range can be 0-10, then each pixel value in the two-dimensional feature map is normalized to 0-10, and the normalized result of each pixel value can be used to represent the compression noise level corresponding to the sub-image area.

[0168] Based on the above scheme, the structural information of the image features can be retained to a greater extent, and the compression noise level of the image to be detected can be directly obtained through the above neural network model.

[0169] Based on the same technical concept, an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are run on a computer, the computer executes each step of the above-mentioned image noise detection method and neural network model training method.

[0170] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0171] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0172] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0173] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0174] Obviously, those skilled in the art may make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include these modifications and variations.

Claims

1. A training method for a neural network model, characterized in that: include: Acquire a training sample set, wherein the training sample set includes a plurality of compressed noise images and compressed noise levels corresponding to a plurality of sub-image regions included in each compressed noise image; The first compressed noise image is any one of the multiple compressed noise images; the first compressed image includes N sub-image areas, where N is a positive integer; the neural network model includes an encoder network and a fully connected layer; and at least two sub-image areas are compressed using different compression algorithms; Performing feature extraction on each of the N sub-image regions through the encoder network to obtain image feature vectors corresponding to each of the N sub-image regions; Determining, by the fully connected layer, a generation probability of a plurality of compression noise levels corresponding to each of the N sub-image regions based on image feature vectors corresponding to the N sub-image regions, wherein the compression noise level corresponding to each sub-image region is a compression noise level corresponding to a maximum generation probability among the generation probabilities of the plurality of compression noise levels corresponding to each sub-image region; Calculating a loss value according to compression noise levels corresponding to the plurality of sub-image regions included in the training sample set and a predicted compression noise level; The network parameters of the neural network model are adjusted according to the loss value to obtain a trained neural network model.

2. The method according to claim 1, wherein The first sub-image region in the first compressed noisy image is obtained by compressing a sub-image region corresponding to the first sub-image region in the first original noise-free image using a first compression algorithm with set compression parameters, wherein the compression noise level of the first sub-image region is determined based on the set compression parameters; or, The second sub-image area in the second compressed noise image is obtained by continuously compressing the sub-image area corresponding to the second sub-image area in the second original noise-free image using multiple compression algorithms, wherein the second compressed noise image is any compressed noise image among the multiple compressed noise images, and the compression noise degree of the second sub-image area is obtained by weighting based on the compression degrees corresponding to the multiple compression algorithms.

3. The method according to claim 1 or 2, wherein: The calculating of the loss value based on the compression noise levels respectively corresponding to the plurality of sub-image regions included in the first compression noise image included in the training sample set and the predicted compression noise level includes: Comparing the compression noise levels corresponding to the N sub-image regions included in the first compressed noise image in the training sample set with the compression noise levels corresponding to the N sub-image regions included in the first compressed noise image determined according to the neural network model to obtain N comparison results; Determining, according to the N comparison results, that N sub-image regions respectively correspond to N loss values; The weighted result obtained by weighting the N loss values adjusts the network parameters of the neural network; the weight used for the kth loss value satisfies the condition shown in the following formula: w k =1-(|P′ (k) | / max(|P′|); Among them, w k Represents the weight used for the loss value corresponding to the kth sub-image area, P′ (k) represents the probability of generating compression noise corresponding to the k-th sub-image region determined according to the neural network model, max(|P ′ |) represents the maximum value among the generation probabilities of the compression noise levels corresponding to the N sub-image areas.

4. A method for detecting image noise, characterized in that: include: Acquire an image to be detected, where the image to be detected includes a plurality of sub-image areas; Feature extraction is performed on each of the multiple sub-image regions using a neural network model; wherein the neural network model is trained using the neural network model training method according to any one of claims 1 to 3 to obtain image feature vectors corresponding to each of the multiple sub-image regions; and at least two of the sub-image regions are compressed using different compression algorithms; The compression noise levels respectively corresponding to the multiple sub-image areas are determined according to the image feature vectors respectively corresponding to the multiple sub-image areas.

5. The method according to claim 4, wherein The method further includes: performing denoising processing on the image to be detected according to the compression noise levels respectively corresponding to the multiple sub-image areas.

6. An electronic device, characterized in that: The electronic device comprises: A communication interface for receiving a training sample set; A processor, configured to obtain the training sample set from the communication interface to execute the method according to any one of claims 1 to 3.

7. A chip system, characterized in that: It comprises a memory and a processor; the processor and the memory are coupled; wherein the memory includes program instructions, and when the program instructions are executed by the processor, the chip system executes the method according to any one of claims 1 to 5.

8. A display device, characterized in that: include: A controller configured to execute the method according to claim 4 or 5; The display is used to display the image after the compression noise is removed.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 5.

Images