An image enhancement method, device, equipment and computer readable storage medium

Abstract

The application provides an image enhancement method and device, equipment and a computer readable storage medium. The method comprises: obtaining an encoding information sequence of an image to be enhanced, wherein the encoding information sequence is a sequence of encoding information of each pixel point; traversing the encoding information sequence, obtaining enhancement information and filter information corresponding to the enhancement information for the encoding information of the traversed pixel point, and enhancing the encoding information of the traversed pixel point by combining the enhancement information and the filter information to obtain enhanced encoding information; when the encoding information sequence is traversed, an enhanced encoding information sequence corresponding to the encoding information sequence is obtained; and based on the enhanced encoding information sequence, an enhanced image corresponding to the image to be enhanced is decoded. Through the application, the calculation efficiency of the image enhancement process can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the image processing technology in the computer application field, and particularly relates to an image enhancement method and device, equipment and a computer readable storage medium. BACKGROUND

[0002] Dark scene enhancement refers to enhancing the brightness of a low-light image without affecting the chroma (hue and saturation); in the field of artificial intelligence, through dark scene enhancement, the visual effect under normal sufficient light can be achieved.

[0003] Generally, in order to perform dark scene enhancement on an image, the image is first traversed in units of pixel points to obtain an enhancement matrix of the image, then the enhancement matrix is traversed to obtain a filter matrix of the image, and the dark scene enhancement is performed in combination with the enhancement matrix and the filter matrix. Since the above process of performing dark scene enhancement on the image involves two traversals of all pixel points of the image, resource consumption is large, resulting in low efficiency of dark scene enhancement of the image. SUMMARY

[0004] The embodiments of the present application provide an image enhancement method, device, equipment and computer readable storage medium, which can improve the dark scene enhancement efficiency of an image.

[0005] The technical solutions of the embodiments of the present application are as follows:

[0006] The embodiments of the present application provide an image enhancement method, comprising:

[0007] obtaining an encoding information sequence of a to-be-enhanced image, wherein the encoding information sequence is a sequence composed of the encoding information of each pixel point;

[0008] traversing the encoding information sequence, obtaining enhancement information and filter information corresponding to the enhancement information for the encoding information of the traversed pixel point, and enhancing the encoding information of the traversed pixel point in combination with the enhancement information and the filter information to obtain enhanced encoding information;

[0009] obtaining an enhanced encoding information sequence corresponding to the encoding information sequence when the traversal of the encoding information sequence is completed;

[0010] decoding the enhanced image corresponding to the to-be-enhanced image based on the enhanced encoding information sequence.

[0011] The embodiments of the present application provide an image enhancement device, comprising:

[0012] an image encoding module configured to obtain an encoding information sequence of a to-be-enhanced image, wherein the encoding information sequence is a sequence composed of the encoding information of each pixel point;

[0013] an image enhancement module, configured to traverse the sequence of encoded information, acquire, for the encoded information of the traversed pixel point, enhancement information and filter information corresponding to the enhancement information, and enhance the encoded information of the traversed pixel point by combining the enhancement information and the filter information, to obtain enhanced encoded information;

[0014] the image enhancement module is further configured to obtain a sequence of enhanced encoded information corresponding to the sequence of encoded information when the traversal of the sequence of encoded information is completed.

[0015] an image decoding module, configured to decode, based on the sequence of enhanced encoded information, an enhanced image corresponding to the image to be enhanced.

[0016] In an embodiment of the present application, the image enhancement module is further configured to generate a periodic traversal instruction based on a quantity threshold, wherein the quantity threshold is determined based on a register specification and an occupied space of the encoded information of the pixel point; and in response to the periodic traversal instruction, the encoded information of the quantity threshold pixel points and the encoded information of the associated pixel points corresponding to each of the traversed pixel points are sequentially traversed from the sequence of encoded information; and the enhancement information and the filter information corresponding to the enhancement information are acquired based on the encoded information of the associated pixel points corresponding to each of the traversed pixel points.

[0017] In an embodiment of the present application, the image enhancement module is further configured to shift the encoded information of each of the traversed pixel points and the encoded information of the associated pixel points respectively; perform equalization processing on the shifted encoded information of each of the traversed pixel points and the shifted encoded information of the associated pixel points, and determine an equalization processing result as initial enhancement information; map the initial enhancement information based on a filter mapping relationship to map out initial filter information; perform reverse shifting on the initial enhancement information to obtain the enhancement information, and perform reverse shifting on the initial filter information to obtain the filter information.

[0018] In an embodiment of the present application, the image enhancement module is further configured to perform equalization processing on the encoded information of each of the traversed pixel points and the encoded information of the associated pixel points, and determine an equalization processing result as the enhancement information; and map the enhancement information based on a filter mapping relationship to determine a mapping result as the filter information.

[0019] In the embodiment of the present application, the image enhancement module is further configured to obtain a first weighted result of the enhancement information and the enhancement weight; obtain a second weighted result of the filter information and the filter weight; and determine a combination result of the first weighted result and the second weighted result as the enhancement encoding information, wherein the enhancement encoding information is a result of enhancing the encoding information of the traversed pixel point.

[0020] In the embodiment of the present application, the image enhancement module is further configured to store the enhancement weight; and obtain the first weighted result of the enhancement information and the stored enhancement weight.

[0021] In the embodiment of the present application, the image enhancement module is further configured to obtain a pixel point layout of the image to be enhanced; determine, based on the pixel point layout, storage location information corresponding to each group of pixel points in the image to be enhanced; store the storage location information corresponding to each group of pixel points in the image to be enhanced; obtain to-be-traversed row and column information in response to the periodic traversal instruction; determine a target group and bias position information based on the to-be-traversed row and column information; determine target position information corresponding to the target group based on the stored storage location information corresponding to each group of pixel points in the image to be enhanced; determine to-be-traversed position information based on the target position information and the bias position information; and traverse, from the to-be-traversed position information of the image encoding information, the encoding information of the number-threshold pixel points and the associated pixel point encoding information corresponding to the encoding information of each traversed pixel point in sequence.

[0022] In the embodiment of the present application, the image encoding module is configured to present a first enhancement control corresponding to the image to be enhanced; and obtain the encoding information sequence of the image to be enhanced in response to a first enhancement operation acting on the first enhancement control.

[0023] In the embodiment of the present application, the image enhancement apparatus further includes an image presentation module configured to present the enhanced image.

[0024] In the embodiment of the present application, the image enhancement apparatus further includes an image determination module configured to present a second enhancement control for played video information; and determine a frame image corresponding to the video information as the image to be enhanced in response to a second enhancement operation acting on the second enhancement control.

[0025] In the embodiment of the present application, the image presentation module is further configured to replace presentation of the frame image with the enhanced image.

[0026] In the embodiment of the present application, the image encoding module is further configured to acquire a color channel image of the image to be enhanced; acquire a minimum color channel value of each pixel point based on the color channel image, and construct a gray scale image corresponding to the image to be enhanced based on the minimum color channel value of each pixel point; acquire a minimum gray scale value of each pixel point in a sliding window based on the gray scale image, and construct the encoding information sequence corresponding to the image to be enhanced based on the minimum gray scale value of each pixel point in the sliding window.

[0027] In the embodiment of the present application, the image decoding module is further configured to decode an initial enhanced image based on the enhanced encoding information sequence; and perform post-processing on the initial enhanced image to obtain the enhanced image corresponding to the image to be enhanced, wherein the post-processing includes at least one of denoising processing, sharpening processing and correction processing.

[0028] The embodiment of the present application provides an image enhancement device, comprising:

[0029] a memory configured to store executable instructions;

[0030] a processor configured to execute the executable instructions stored in the memory to implement the image enhancement method provided by the embodiment of the present application.

[0031] The embodiment of the present application provides a computer readable storage medium storing executable instructions, which are used to cause the processor to execute the image enhancement method provided by the embodiment of the present application.

[0032] The embodiment of the present application has at least the following beneficial effects: in the process of enhancing the encoding information of each pixel point, the encoding information of the pixel point traversed from the encoding information sequence is calculated to calculate the enhancement information and the filter information corresponding to the enhancement information, and the encoding information of each pixel point is enhanced based on the enhancement information and the filter information; therefore, the enhancement of the encoding information of each pixel point can be completed by one traversal, so that the resource consumption is less and the efficiency of dark scene enhancement of the image can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is an exemplary flow diagram of dark scene enhancement;

[0034] Figure 2 is an exemplary architecture diagram of the image enhancement system provided by the embodiment of the present application;

[0035] Figure 3 is an exemplary architecture diagram of the image enhancement system provided by the embodiment of the present application; Figure 2 is an exemplary architecture diagram of the terminal in the image enhancement system provided by the embodiment of the present application;

[0036] Figure 4is a flowchart of an optional image enhancement method provided by an embodiment of the present application;

[0037] Figure 5 is another flowchart of an optional image enhancement method provided by an embodiment of the present application;

[0038] Figure 6 is still another flowchart of an optional image enhancement method provided by an embodiment of the present application;

[0039] Figure 7 is an exemplary scene diagram of triggering dark scene enhancement provided by an embodiment of the present application;

[0040] Figure 8 is an exemplary image enhancement flowchart provided by an embodiment of the present application;

[0041] Figure 9 is a processing flowchart of S802 in Figure 8

[0042] Figure 10 is a single instruction multiple data stream processing manner provided by an embodiment of the present application;

[0043] Figure 11 is a single instruction single data stream processing manner provided by an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings, and the described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by those skilled in the art without making creative efforts fall within the scope of protection of the present application.

[0045] In the following description, "some embodiments" are described, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subset of all possible embodiments, and can be combined with each other without conflict.

[0046] In the following description, the terms "first\second" are only to distinguish similar objects, and do not represent a specific order of the objects, and it can be understood that "first\second" can be interchanged in a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0047] ​Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for describing the embodiments of the application only and is not intended to be limiting of the application.

[0048] The relevant data collection processing in the embodiments of the application should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and the personal information subject.

[0049] Before the embodiments of the application are further described in detail, the terms and phrases involved in the embodiments of the application are explained, and the terms and phrases involved in the embodiments of the application are applicable to the following explanations.

[0050] 1) Cloud Technology, refers to a kind of hosting technology that unifies a series of resources such as hardware, software and network in a wide area network or local area network to realize data calculation, storage, processing and sharing.

[0051] 2) Low-light image, is an image with brightness lower than a brightness threshold value collected in a scene with relatively dark illumination; for example, an image obtained by taking a photo or making a video at night; in a low-light image, due to the relatively dark illumination, the visual quality of the image is greatly reduced, resulting in loss of details on the image and reduction of contrast, which affects the subjective visual perception of people on one hand and the performance of subsequent image processing on the other hand.

[0052] 3) YUV (luminance chrominance), is a color encoding method, in which "Y" represents luminance, that is, gray scale value, and "U" and "V" represent chrominance; in YUV, color is represented by luminance and chrominance. Among them, luminance, also known as lightness, represents the light and dark degree of color; chrominance is the property of color excluding luminance, representing the hue and saturation of color, and hue refers to the relative light and dark degree of an image, which is manifested as color on a color image, and saturation refers to the vividness of color. In addition, commonly used YUV formats are, for example, "NV12".

[0053] 4) Single Instruction Multiple Data (SIMD), is an extension of the basic instruction set of a processor, mainly used for parallel operation instructions of continuous data storage; for example, in the image processing involved in the embodiments of the application, data in the memory is represented by less than or equal to 8 bits (bits) per pixel component, and usually only the lower 8 bits of a 32-bit or 64-bit register can be used for calculation when using a processor, resulting in resource waste; using SIMD can split a 64-bit register into 8 8-bit registers for use, and one instruction can complete 8 data processing operations at the same time.

[0054] 5) Enhancement matrix, which represents a global dark scene value, is a global variable obtained by using dark channel in YUV coding to evaluate the image as a whole, and is used to control the brightness, chroma and intensity of the image as a whole, so as to ensure that the overall brightness is not abnormal and the color restoration degree is maintained; the enhancement information sequence involved in the embodiment of the application is an enhancement matrix. Wherein, the acquisition process of the dark channel is as follows: first, the minimum value in the three channels corresponding to each pixel point is calculated, and the minimum channel value is taken as the value of the pixel point to obtain a gray image consistent with the size of the original image; then, the minimum value filtering is used for smoothing processing, that is, in the obtained gray image, a rectangular window of a certain size is taken with each pixel point as the center, and the minimum value of the gray values in the rectangular window is taken to replace the value of the center pixel point, so as to obtain a dark channel image corresponding to the original image.

[0055] 6) Filter matrix, which represents a local variable obtained by using Gaussian filtering and the like to process the dark channel, is used to distinguish the dark region and the bright region in the image during the dark scene enhancement processing; the filter matrix and the enhancement matrix are combined to control the dark scene enhancement effect; the filter information sequence involved in the embodiment of the application is a filter matrix.

[0056] 7) Processor increment, which is used to measure the performance of the processor; taking CPU increment as an example for description, the CPU increment represents the absolute increment of the CPU occupancy rate of the host process on the computer after a certain function or a certain algorithm is started; for example, before the dark scene enhancement is started, the CPU occupancy rate of the video conference is 5%, and after the dark scene enhancement is started, the CPU occupancy rate of the video conference is 7%, at this time, the CPU increment is 2%. The CPU increment reflects the performance of a certain function or a certain algorithm on the CPU, and the lower the value, the better the performance. In addition, since the core numbers of different computer devices are different, the value needs to be divided by the core number.

[0057] 8) Register, which is a small storage area in the processor used to store data, is used to temporarily store the data participating in the operation and the operation result; in the computer field, the register is an internal element of the processor.

[0058] 9) Control, which is a triggerable information displayed in the form of buttons, icons, links, texts, selection boxes, input boxes, tabs and the like; wherein, the triggering mode can be contact type triggering, non-contact type triggering, receiving instruction type triggering and the like; in addition, the various controls in the embodiment of the application can be a single control, or a general term of multiple controls.

[0059] 10) operation, a way for triggering the device to perform a process, such as a click operation, a double-click operation, a long-press operation, a sliding operation, a gesture operation, a received trigger instruction, etc.; in addition, various operations in the embodiments of the present application can be a single operation, or a total of multiple operations; and various operations in the embodiments of the present application can be touch operations, or non-touch operations.

[0060] 11) response, used to represent a condition or state on which the performed process depends, when the dependent condition or state is met, the performed operation or operations can be real-time, or have a set delay; in the absence of a specific description, there is no restriction on the execution order of multiple operations performed.

[0061] It should be noted that cloud computing is a computing mode that distributes computing tasks on a resource pool composed of a large number of computing machines, so that various application systems can obtain computing power, storage space and information services according to needs. The network providing resources is called "cloud"; the resources in the "cloud" are infinitely expandable to users and can be obtained at any time, used on demand, expanded at any time, and paid according to use. As a basic capability provider of cloud computing, a cloud computing resource pool (referred to as a cloud platform, generally referred to as an IaaS (Infrastructure as a Service) platform) is established, and a plurality of types of virtual resources are deployed in the resource pool for external customers to select and use. The cloud computing resource pool mainly includes: computing devices (virtualized machines containing operating systems), storage devices, network devices. According to logical functions, the IaaS layer can be deployed on the PaaS (Platform as a Service) layer, and the SaaS (Software as a Service) layer can be deployed on the PaaS layer, or the SaaS can be directly deployed on the IaaS; here, the PaaS is a platform for software running, such as a database, a web container, etc. The SaaS is various business software, such as a web portal website, a short message massager, etc. Generally, SaaS and PaaS are upper layers relative to IaaS. The image enhancement method provided in the embodiments of the present application can be implemented by means of IaaS, PaaS and PaaS in cloud computing.

[0062] Generally, in order to perform dark scene enhancement processing on an image, the image is usually traversed in units of pixels, an enhancement matrix of the image is obtained, the enhancement matrix is traversed to obtain a filter matrix of the image, and the dark scene enhancement is performed in combination with the enhancement matrix and the filter matrix; see Figure 1 , Figure 1 is an exemplary flowchart of dark scene enhancement; as Figure 1As shown, the exemplary dark scene enhancement process includes:

[0063] S101, start.

[0064] S102, traverse the encoding matrix of the image to extract the enhancement matrix.

[0065] Here, by traversing the encoding matrix of the image, the global dark scene value is extracted, and the enhancement matrix of the image is obtained.

[0066] S103, traverse the enhancement matrix to obtain the filter matrix and combine the dark scene enhancement with the enhancement matrix.

[0067] Here, by traversing the enhancement matrix, the local dark scene value is mapped, and the filter matrix is obtained.

[0068] S104, end.

[0069] Since the process of dark scene enhancement of the image in S101 to S104 is traversed twice for the whole image pixel, the operation of reading the pixel is repeated, the computer memory jump access is frequently performed, and the CPU resources are occupied, the resource consumption is large, and thus the efficiency of the dark scene enhancement of the image is low.

[0070] Based on this, the embodiment of the present application provides an image enhancement method, device, equipment and computer readable storage medium, which can improve the dark scene enhancement efficiency of the image and reduce the load of the processor. The following describes an exemplary application of the image enhancement device provided by the embodiment of the present application. The image enhancement device provided by the embodiment of the present application can be implemented as a notebook computer, a tablet computer, a desktop computer, a set-top box, a mobile device, a vehicle-mounted terminal, a smart television (such as a mobile phone, a portable music player, a personal digital assistant, a dedicated message device, a portable game device) and various types of user terminals, and can also be implemented as a server. The following will describe an exemplary application of the image enhancement device implemented as a terminal.

[0071] Referring to Figure 2 , Figure 2 is an optional architecture schematic diagram of the image enhancement system provided by the embodiment of the present application; as Figure 2 shown, to support an image enhancement application, in the image enhancement system 100, the terminal 200 (image enhancement device, exemplary terminal 200-1 and terminal 200-2 are shown) is connected to the server 400 through the network 300, and the network 300 can be a wide area network or a local area network, or a combination of the two. In addition, the image enhancement system 100 also includes a database 500 for providing data support to the server 400.

[0072] Terminal 200 is used to obtain the encoded information sequence of the image to be enhanced after the dark scene enhancement option is selected in the settings options for the presented video information (see Terminal 200-1), or after the image to be enhanced is selected on the interface and the dark scene enhancement button is clicked (see Terminal 200-2). The encoded information sequence is a sequence composed of the encoded information of each pixel. The encoded information sequence is traversed, and for the encoded information of the traversed pixels, enhancement information and the corresponding filtering information are obtained. The enhanced information and the filtering information are combined to enhance the encoded information of the traversed pixels to obtain enhanced encoded information. When the encoded information sequence has been traversed, the enhanced encoded information sequence corresponding to the encoded information sequence is obtained. Based on the enhanced encoded information sequence, the enhanced image corresponding to the image to be enhanced is decoded.

[0073] Server 400 is used to provide computing services to terminal 200 via network 300.

[0074] In some embodiments, server 400 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Terminal 200 may be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, in-vehicle terminal, smart TV, etc., but is not limited to these. Terminals and servers can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0075] See Figure 3 , Figure 3 This is provided by the embodiments of this application. Figure 2 A schematic diagram of the structural composition of one type of terminal. Figure 3 The terminal 200 shown includes at least one processor 210, a memory 250, at least one network interface 220, and a user interface 230. The various components in the terminal 200 are coupled together via a bus system 240. It is understood that the bus system 240 is used to implement communication between these components. In addition to a data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 3 The general labeled all buses as Bus System 240.

[0076] The processor 210 can be an integrated circuit chip that has the processing capability of signals, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., wherein the general-purpose processor can be a microprocessor or any conventional processor.

[0077] The user interface 230 includes one or more output devices 231 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 230 also includes one or more input devices 232 that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0078] The memory 250 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 250 optionally includes one or more storage devices physically located in proximity to the processor 210.

[0079] The memory 250 includes volatile memory or non-volatile memory, and can also include both volatile and non-volatile memory. Non-volatile memory can be read only memory (ROM), and volatile memory can be random access memory (RAM). The memory 250 described in embodiments of the present application is intended to include any suitable type of memory.

[0080] In some embodiments, the memory 250 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or a subset or superset thereof, which are described below.

[0081] The operating system 251 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0082] The network communication module 252 is used to reach other computer devices via one or more (wired or wireless) network interfaces 220, examples of which include Bluetooth, wireless compatibility certification (Wi-Fi), and universal serial bus (USB), etc.

[0083] In some embodiments, the terminal 200 further comprises a rendering module 253 configured to enable rendering of information (e.g., a user interface for operating a peripheral device and displaying content and information) via one or more output devices 231 (e.g., a display screen, a speaker, etc.) associated with the user interface 230.

[0084] An input processing module 254 configured to detect and interpret one or more user inputs or interactions from one or more input devices 232.

[0085] In some embodiments, the image enhancement apparatus provided by the embodiments of the present application can be implemented in a software manner, Figure 3 An image enhancement apparatus 255 stored in the memory 250 is shown, which can be software in the form of programs and plug-ins, etc., including the following software modules: an image encoding module 2551, an image enhancement module 2552, an image decoding module 2553, an image rendering module 2554, and an image determination module 2555. These modules are logical, and thus can be combined or further split according to the implemented functions. The functions of each module will be described below.

[0086] In some other embodiments, the image enhancement apparatus provided by the embodiments of the present application can be implemented in a hardware manner. As an example, the image enhancement apparatus provided by the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the image enhancement method provided by the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can use one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), or other electronic elements.

[0087] In the following, the image enhancement method provided by the embodiments of the present application will be described in conjunction with exemplary applications and implementations of the server provided by the embodiments of the present application.

[0088] Referring to Figure 4 , Figure 4 is an optional flowchart of the image enhancement method provided by the embodiments of the present application, which will be described in conjunction with the steps shown. Figure 4

[0089] S401, obtaining an encoding information sequence of an image to be enhanced.​

[0090] In the embodiment of the present application, the image enhancement device encodes each pixel point of the to-be-enhanced image based on the brightness of the pixel point to obtain the encoding information of each pixel point, and a sequence formed by the encoding information of each pixel point is the encoding information sequence corresponding to the to-be-enhanced image. Here, the image enhancement device can be the encoding information sequence directly obtained, or the encoding information sequence obtained by encoding the to-be-enhanced image, wherein the encoding is the encoding based on brightness, such as YUV encoding and extracting Y channel values.

[0091] It should be noted that the encoding information sequence is the encoding image based on brightness corresponding to the to-be-enhanced image, the encoding information of the pixel point in the encoding information sequence is in one-to-one correspondence with the information of the pixel point in the to-be-enhanced image, and the encoding information of each pixel point in the encoding information sequence is obtained based on the brightness of each pixel point, such as the Y channel value of the pixel point in the YUV image corresponding to the to-be-enhanced image, or the dark channel value of the pixel point of the to-be-enhanced image.

[0092] S402, traversing the encoding information sequence, obtaining the enhancement information and the filter information corresponding to the enhancement information for the encoding information of the traversed pixel point, and combining the enhancement information and the filter information to enhance the encoding information of the traversed pixel point to obtain the enhanced encoding information.

[0093] In the embodiment of the present application, after the image enhancement device obtains the encoding information sequence, in order to perform dark scene enhancement processing in units of pixel points, the image enhancement device traverses the encoding information sequence in units of pixel points. When the image enhancement device traverses the encoding information sequence, for the encoding information of the traversed pixel point, the image enhancement device obtains the enhancement information corresponding to the encoding information of the pixel point, and based on the enhancement information corresponding to the encoding information of the pixel point, the image enhancement device obtains the filter information corresponding to the encoding information of the pixel point, and combines the enhancement information corresponding to the encoding information of the pixel point and the filter information corresponding to the encoding information of the pixel point, and determines the combination result as the enhancement result corresponding to the encoding information of the pixel point, that is, the enhanced encoding information.

[0094] It should be noted that the enhancement information is a global evaluation result of the brightness of the pixel points, and the enhancement information sequence corresponding to the image to be enhanced, which is constituted by the enhancement information of each pixel point, represents a global dark scene value, which is used to control the brightness, chroma and intensity of the whole image, so as to ensure that the overall brightness is within a preset brightness range and the color restoration degree is maintained. The filter information is a local evaluation result of the brightness of the pixel points, and the filter information sequence corresponding to the image to be enhanced, which is constituted by the filter information of each pixel point, represents a local dark scene value, which is used to control the brightness enhancement amplitude of the dark region and the bright region of the image, and the brightness enhancement amplitude is inversely proportional to the brightness of the image, for example, the brightness of the dark region of the image is improved, and the bright region of the image is fine-tuned; and the filter information is mapped based on the enhancement information. In addition, the enhancement coding information is the coding information of each pixel after enhancement, and here, the enhancement means dark scene enhancement.

[0095] S403, when the coding information sequence is traversed, obtaining an enhancement coding information sequence corresponding to the coding information sequence.

[0096] In the embodiment of the present application, the image enhancement device performs dark scene enhancement on the coding information of each traversed pixel point by using the processing procedure described in S402, so as to obtain the corresponding enhancement coding information; thus, when the image enhancement device traverses the coding information sequence and completes the enhancement of the coding information of each pixel point, the enhancement coding information of each pixel point is obtained; and the enhancement coding information of each pixel point constitutes the enhancement coding information sequence corresponding to the coding information sequence. The coding information in the coding information sequence and the enhancement coding information in the enhancement coding information sequence correspond one by one, and the enhancement coding information sequence and the image to be enhanced correspond one by one based on the pixel points.

[0097] S404, based on the enhancement coding information sequence, decoding an enhancement image corresponding to the image to be enhanced.

[0098] In the embodiment of the present application, the image enhancement device decodes the obtained enhancement coding information sequence, and thus obtains the enhancement image corresponding to the image to be enhanced, wherein the enhancement image is the image to be enhanced after enhancement.

[0099] It should be noted that if the coding information sequence is directly obtained by the image enhancement device without encoding, S404 will not be performed, and the enhancement coding information sequence is directly returned.

[0100] It can be understood that, in the process of enhancing the encoding information of each pixel point, the filter information corresponding to the enhancement information is calculated at the same time as the enhancement information is calculated for the encoding information of the pixel point traversed from the encoding information sequence, and the encoding information of each pixel point is enhanced based on the enhancement information and the filter information at the same time; therefore, the enhancement of the encoding information of each pixel point can be completed through one traversal, so that the resource consumption is less, and the efficiency of dark scene enhancement of the image can be improved.

[0101] In the embodiment of the present application, S402 can be implemented by S4021 to S4023; that is, the image enhancement device traverses the encoding information sequence, and for the encoding information of the pixel point traversed, the enhancement information and the filter information corresponding to the enhancement information are obtained, including S4021 to S4023, which will be described below.

[0102] S4021, generating a periodic traversal instruction based on a quantity threshold.

[0103] It should be noted that the encoding information sequence of the image to be enhanced is stored in the memory of the image enhancement device, and the image enhancement device traverses the encoding information sequence by traversing the encoding information sequence in the memory through the processor. Here, the image enhancement device obtains the register specification and the occupied space of the encoding information of the pixel point in the processor, and determines the reading quantity corresponding to one instruction period based on the register specification and the occupied space of the encoding information of the pixel point, that is, the quantity threshold, and finally generates a single traversal instruction for obtaining the encoding information of the quantity threshold number of pixel points, that is, the periodic traversal instruction.

[0104] Exemplarily, the process of obtaining the quantity threshold can be implemented by formula (1), and formula (1) is as follows:

[0105]

[0106] Wherein, c is the quantity threshold; v is the register specification, for example, 64 bits, 32 bits; o is the occupied space of the encoding information of the pixel point, for example, 8 bits.

[0107] S4022, in response to the periodic traversal instruction, sequentially traversing the encoding information of the quantity threshold number of pixel points and the associated pixel point encoding information corresponding to the encoding information of each traversed pixel point from the encoding information sequence.

[0108] In the embodiment of the present application, when the image enhancement device traverses the sequence of encoded information, in response to each periodic traversal instruction for obtaining data, the image enhancement device traverses the encoded information of a threshold number of pixel points from the sequence of encoded information at one time, and the encoded information of each traversed pixel point in the encoded information of the threshold number of pixel points corresponds to the associated pixel point encoded information.

[0109] It should be noted that the associated pixel point encoded information can be the encoded information of the adjacent pixel point of each traversed pixel point, or the encoded information of the pixel point in the preset sliding window, etc., and the embodiment of the present application does not make a specific limitation thereto.

[0110] S4023, based on the associated pixel point encoded information corresponding to the encoded information of each traversed pixel point, obtaining the enhancement information and the filter information corresponding to the enhancement information.

[0111] It should be noted that the image enhancement device can randomly select one pixel point encoded information from the encoded information of each traversed pixel point and the associated pixel point encoded information to determine the enhancement information corresponding to the encoded information of the traversed pixel point, or determine the enhancement information corresponding to the encoded information of the traversed pixel point based on the overall encoded information of each traversed pixel point and the associated pixel point information, etc., and the embodiment of the present application does not make a specific limitation thereto. In addition, the image enhancement device judges the brightness intensity of the enhancement information, and determines the filter information corresponding to the enhancement information based on the judgment result; here, when the image enhancement device determines the filter information based on the judgment result, if it is determined based on the enhancement information that the brightness of the pixel point is high, the filter information lower than the enhancement information is obtained, and if it is determined based on the enhancement information that the brightness of the pixel point is low, the filter information higher than the enhancement information is obtained.

[0112] It can be understood that when traversing the sequence of encoded information, the reading of data is performed at one time through the SIMD mode, which not only improves the storage efficiency of the register, but also reduces the resource consumption caused by the interaction in the traversal process, improves the parallel degree of the traversal, and thus improves the traversal efficiency.

[0113] In the embodiment of the present application, S4023 can be implemented by S40231 to S40234; that is, the image enhancement device obtains the enhancement information and the filter information corresponding to the enhancement information based on the associated pixel point encoded information corresponding to the encoded information of each traversed pixel point, including S40231 to S40234, and each step will be described below.

[0114] S40231, respectively shifting the encoded information of each traversed pixel point and the associated pixel point encoded information.

[0115] It should be noted that when the associated pixel point encoding information belongs to a data type with a decimal point (for example, a floating point type, a double floating point type, etc.), in order to improve the efficiency of the calculation of the encoding information of each traversed pixel point and the associated pixel point encoding information, the image enhancement device shifts the associated pixel point encoding information to convert the associated pixel point encoding information into integer type data, and shifts the encoding information of each traversed pixel point to convert the encoding information of each traversed pixel point into integer type data; thus, the shifted associated pixel point encoding information and the shifted encoding information of each traversed pixel point are both integer type data.

[0116] S40232, equalize the shifted encoding information of each traversed pixel point and the shifted associated pixel point encoding information, and determine the equalization result as initial enhancement information.

[0117] In the embodiment of the present application, the image enhancement device equalizes the shifted encoding information of each traversed pixel point and the shifted associated pixel point encoding information to obtain the image overall evaluation result. Here, since the image enhancement device equalizes the shifted encoding information of each traversed pixel point and the shifted associated pixel point encoding information, the "shifted encoding information of each traversed pixel point and the shifted associated pixel point encoding information" after equalization is only the initial enhancement information of the traversed pixel point.

[0118] S40233, mapping the initial enhancement information based on the filter mapping relationship to obtain initial filter information.

[0119] In the embodiment of the present application, the image enhancement device can obtain the filter mapping relationship, which is used to determine the bright and dark areas of the image; here, the image enhancement device maps the initial enhancement information to obtain the image local evaluation result. Here, since the image enhancement device maps the initial enhancement information, the mapped information is only the initial filter information of the pixel point.

[0120] S40234, reverse shifting the initial enhancement information to obtain enhancement information, and reverse shifting the initial filter information to obtain filter information.

[0121] In the embodiment of the present application, the image enhancement device reverse shifts the initial enhancement information and the initial filter information to obtain the enhancement information corresponding to the initial enhancement information and the filter information corresponding to the initial filter information. The reverse shifting is a corresponding reverse process of the shifting in S40231.

[0122] In the embodiment of the present application, S4023 can be implemented by S40235 and S40236; that is, the image enhancement device obtains the enhancement information and the filter information corresponding to the enhancement information based on the associated pixel point coding information corresponding to the coding information of each traversed pixel point, including S40235 and S40236, which are described below.

[0123] S40235, equalizes the coding information of each traversed pixel point and the associated pixel point coding information, and determines the equalization result as the enhancement information.

[0124] S40236, maps the enhancement information based on the filter mapping relationship, and determines the mapping result as the filter information.

[0125] It should be noted that the image enhancement device can also directly calculate the coding information of each traversed pixel point and the associated pixel point coding information to obtain the enhancement information and the filter information corresponding to the coding information of the traversed pixel point.

[0126] In the embodiment of the present application, S402 can be implemented by S4021 to S4023; that is, the image enhancement device enhances the coding information of the traversed pixel point in combination with the enhancement information and the filter information to obtain the enhanced coding information, including S4021 to S4023, which are described below.

[0127] S4021, obtaining the first weighted result of the enhancement information and the enhancement weight.

[0128] It should be noted that the image enhancement device can obtain the weight corresponding to the enhancement information, which is the enhancement weight; and the image enhancement device obtains the combination result of the enhancement information and the enhancement weight, which is the first weighted result.

[0129] S4022, obtaining the second weighted result of the filter information and the filter weight.

[0130] It should be noted that the image enhancement device can obtain the weight corresponding to the filter information, which is the filter weight; and the image enhancement device obtains the combination result of the filter information and the filter weight, which is the second weighted result.

[0131] S4023, determining the combination result of the first weighted result and the second weighted result as the enhanced coding information.

[0132] It should be noted that the process of obtaining the enhanced coding information is the process of enhancing the coding information of the traversed pixel point, and the enhanced coding information is the result of enhancing the coding information of the traversed pixel point.

[0133] In the embodiment of the present application, S4021 is further preceded by S4024; that is, before the image enhancement device obtains the first weighting result of the enhancement information and the enhancement weight, the image enhancement method further comprises S4024, which is described as follows.

[0134] S4024, store the enhancement weight.

[0135] In the embodiment of the present application, the image enhancement device stores the experience value or experience parameter to realize the solidification of the experience value or experience parameter; for example, for the enhancement weight, the solidification of the enhancement weight is realized by storing the enhancement weight.

[0136] Correspondingly, in the embodiment of the present application, S4021 can be realized by S40211; that is, the sample acquisition device obtains the first weighting result of the enhancement information and the enhancement weight, which comprises S40211, which is described as follows.

[0137] S40211, obtain the first weighting result of the enhancement information and the stored enhancement weight.

[0138] It should be noted that, since the image enhancement device extracts the stored enhancement weight, here, the stored enhancement weight is directly obtained, and the stored enhancement weight is combined with the enhancement information to obtain the first weighting result.

[0139] In the embodiment of the present application, the experience value or experience parameter such as the filter mapping relationship and the filter weight can be solidified before participating in the calculation.

[0140] It can be understood that, by solidifying the experience value or experience parameter such as the filter mapping relationship, the filter weight and the enhancement weight, the calculation efficiency in the image enhancement process can be improved, so that the image enhancement efficiency can be improved.

[0141] In the embodiment of the present application, S402 can further comprise S405 to S407; that is, before the image enhancement device traverses the sequence of encoding information, the image enhancement method further comprises S405 to S407, which are described as follows.

[0142] S405, obtain the pixel point layout of the image to be enhanced.

[0143] It should be noted that the pixel point layout is the position information of each pixel point in the image to be enhanced in the image to be enhanced.

[0144] S406, determine the storage position information corresponding to each group of pixel points in the image to be enhanced based on the pixel point layout.

[0145] In the embodiment of the present application, the image enhancement device groups the pixels of the to-be-enhanced image based on the pixel layout (such as the process of determining the pixels of a row or the pixels of a column), determines each group (row or column) of pixels, and determines the storage location of the encoding information of each group of pixels in the memory, that is, the storage location information.

[0146] It should be noted that the image enhancement device can determine the relative storage location information of the last pixel in a group as the storage location information corresponding to the group of pixels.

[0147] For example, for a to-be-enhanced image with a pixel layout of 128*256, the storage location information of the first row of pixels (a group) is 256, the storage location information of the second row of pixels is 512, and so on, to obtain the storage location information of each group of pixels.

[0148] S407, store the storage location information corresponding to each group of pixels in the to-be-enhanced image.

[0149] In the embodiment of the present application, the image enhancement device stores the storage location information corresponding to each group of pixels obtained, to perform traversal processing according to the stored storage location information corresponding to each group of pixels.

[0150] Correspondingly, in the embodiment of the present application, S4022 can be implemented by S40221-S40223; that is, the image enhancement device responds to the periodic traversal instruction to sequentially traverse the encoding information of a number threshold of pixels and the associated pixel encoding information corresponding to the encoding information of each traversed pixel from the encoding information sequence, including S40221-S40223, which will be described below.

[0151] S40221, in response to the periodic traversal instruction, obtain the to-be-traversed row and column information, and determine the target group and the bias position information based on the to-be-traversed row and column information.

[0152] It should be noted that the to-be-traversed row and column information is the row and column position of the pixel in the pixel layout. Here, the image enhancement device can determine the row in the to-be-traversed row and column information as the target group, and the column in the to-be-traversed row and column information as the bias position information; or determine the column in the to-be-traversed row and column information as the target group, and the row in the to-be-traversed row and column information as the bias position information.

[0153] S40222, determine the target position information corresponding to the target group based on the stored storage location information corresponding to each group of pixels in the to-be-enhanced image, and determine the to-be-traversed position information based on the target position information and the bias position information.

[0154] It should be noted that the image enhancement device determines the storage location information corresponding to the target group from the storage location information corresponding to each group of pixel points in the image to be enhanced, and the storage location information corresponding to the target group is the target location information corresponding to the target group. In addition, the to-be-traversed location information is the superposition result of the target location information and the bias location information.

[0155] S40223, sequentially traverse the encoding information of the number threshold of pixel points from the to-be-traversed location information of the image encoding information, and the associated pixel point encoding information corresponding to the encoding information of each traversed pixel point.

[0156] It can be understood that the pixel points of the image to be enhanced are grouped by rows or columns, and then the traversal location of the pixel points to be traversed is determined according to the relative location information of the determined group, thereby reducing the calculation consumption of the location of each pixel point in the traversal process.

[0157] Referring to Figure 5 , Figure 5 is another optional flowchart of the image enhancement method provided by the embodiment of the present application; as shown in Figure 5 , in the embodiment of the present application, S401 can be implemented by S4011; that is, the image enhancement device acquires the encoding information sequence of the image to be enhanced, including S4011, which is described below.

[0158] S4011, presenting a first enhancement control corresponding to the image to be enhanced, and acquiring the encoding information sequence of the image to be enhanced in response to a first enhancement operation acting on the first enhancement control.

[0159] It should be noted that the first enhancement control is used to trigger the dark scene enhancement processing of the image to be enhanced, and the image enhancement device acquires the operation of the user on the dark scene of the image to be enhanced by presenting the first enhancement control corresponding to the image to be enhanced. When the user triggers the first enhancement control to perform the dark scene enhancement processing on the image to be enhanced, the image enhancement device also receives the first enhancement operation acting on the first enhancement control; at this time, the image enhancement device responds to the first enhancement operation to trigger the dark scene enhancement process of the image to be enhanced, thereby acquiring the encoding information sequence of the image to be enhanced and performing subsequent enhancement processing process on the encoding information sequence.

[0160] Continuing to refer to Figure 5 , in the embodiment of the present application, S404 can further include S408; that is, after the image enhancement device decodes the enhanced image corresponding to the image to be enhanced based on the enhanced encoding information sequence, the image enhancement method further includes S408, which is described below.

[0161] S408, presenting the enhanced image.

[0162] In the embodiment of the present application, the image enhancement device informs the user of the dark scene processing result of the to-be-processed image by presenting the enhanced image.

[0163] It should be noted that the first enhancement control can also be presented by another device, and then the image enhancement device, upon receiving a first enhancement request caused by another device in response to a first enhancement operation acting on the first enhancement control, acquires the encoding information sequence of the to-be-enhanced image in response to the first enhancement request. Thus, after the image enhancement device obtains the enhanced image, the image enhancement device sends the enhanced image to another device to present the enhanced image on another device.

[0164] Referring to Figure 6 , Figure 6 is another optional flowchart of the image enhancement method provided by the embodiment of the present application; as shown in Figure 6 , in the embodiment of the present application, S401 can further include S409; that is, before the image enhancement device acquires the encoding information sequence of the to-be-enhanced image, S409 can be further included, which is described as follows.

[0165] S409, for the played video information, presenting a second enhancement control, and determining a frame image corresponding to the video information as the to-be-enhanced image in response to a second enhancement operation acting on the second enhancement control.

[0166] It should be noted that the played video information can be real-time or historical, which is not limited in the embodiment of the present application. Here, the second enhancement control is used to trigger the dark scene enhancement processing of the video information, and the image enhancement device acquires the user's operation for the dark scene of the video information by presenting the second enhancement control corresponding to the video information. When the user triggers the second enhancement control to perform the dark scene enhancement processing on the video information, the image enhancement device also receives the second enhancement operation acting on the second enhancement control, and at this time, the image enhancement device performs the dark scene enhancement processing on the current video frame image of the video information in response to the second enhancement operation, so as to take the current frame image as the to-be-processed image; or since the image corresponding to the current video frame has been presented, at this time, the image enhancement device performs the dark scene enhancement processing from the next frame of the current video frame of the video information in response to the second enhancement operation, so as to take the next frame image as the to-be-processed image.

[0167] Continuing to refer to Figure 6 , in the embodiment of the present application, S404 can further include S410; that is, after the image enhancement device decodes the enhanced image corresponding to the to-be-enhanced image based on the enhanced encoding information sequence, the image enhancement method further includes S410, which is described as follows.

[0168] S410, replacing the presentation of the frame image with the enhanced image.

[0169] It should be noted that the second enhancement control can also be presented by other devices, and then the image enhancement device, upon receiving a second enhancement request of the other device in response to a second enhancement operation acting on the second enhancement control, obtains the encoding information sequence of the image to be enhanced in response to the second enhancement request. Thus, after the image enhancement device obtains the enhanced image, the image enhancement device sends the enhanced image to the other device to present the enhanced image on the other device.

[0170] In the embodiment of the present application, the image enhancement device obtains the encoding information sequence of the image to be enhanced, including: the image enhancement device obtains a color channel image of the image to be enhanced; and based on the color channel image, obtains a minimum color channel value of each pixel point, and constructs a grayscale image corresponding to the image to be enhanced based on the minimum color channel value of each pixel point; and based on the grayscale image, obtains a minimum grayscale value of each pixel point in a sliding window, and constructs an encoding information sequence corresponding to the image to be enhanced based on the minimum grayscale value of each pixel point in the sliding window.

[0171] In the embodiment of the present application, S404 can be implemented by S4041 and S4042; that is, the image enhancement device decodes the enhanced image corresponding to the image to be enhanced based on the enhanced encoding information sequence, including S4041 and S4042, which will be described below.

[0172] S4041, decoding an initial enhanced image based on the enhanced encoding information sequence.

[0173] It should be noted that the image enhancement device decodes the enhanced encoding information sequence to obtain a decoding result as an initial enhanced image.

[0174] S4042, post-processing the initial enhanced image to obtain an enhanced image corresponding to the image to be enhanced.

[0175] In the embodiment of the present application, the image enhancement device can directly determine the initial enhanced image as the image to be enhanced after dark scene enhancement, or can post-process the initial enhanced image and determine the post-processed initial enhanced image as the enhanced image corresponding to the image to be enhanced. The post-processing includes at least one of denoising processing, sharpening processing and correction processing.

[0176] It should be noted that the denoising processing is, for example, a bilateral filter denoising algorithm, an "NL-Mean" denoising algorithm, and a "BM3D" denoising algorithm; the correction processing is, for example, gamma correction to improve the brightness of the initial enhanced image; and the sharpening processing is to improve the loss of the initial enhanced image.

[0177] In the following, an exemplary application of the embodiment of the present application in an actual application scenario will be described.

[0178] Referring to Figure 7 , Figure 7 is a schematic diagram of a triggering dark scene enhancement provided by an embodiment of the present application; as Figure 7 shown, in the video interface 7-1 of the video conference, a setting box 7-2 is set for the conference video (video information); in response to clicking a video option 7-21 in the setting box 7-2, a video image 7-22 of the conference video and a dark scene enhancement option 7-23 (second enhancement control) corresponding to the video image 7-22 are presented in the setting box 7-2; when the user checks (second enhancement operation) the option 7-23, the frame image of the conference video presented in the video interface 7-1 is subjected to dark scene enhancement. Here, the dark scene enhancement is triggered by user operation, and compared with the processing of determining whether to perform dark scene enhancement based on the brightness or content of the frame image, the processing speed is faster.

[0179] It should be noted that when the frame image of the conference video is subjected to dark scene enhancement, referring to Figure 8 , Figure 8 is a flowchart of an image enhancement provided by an embodiment of the present application; as Figure 8 shown, it includes S801 to S803, wherein:

[0180] S801, a video image (to-be-enhanced image) is acquired.

[0181] S802, a one-time traversal is performed on the video image, and an enhancement value (enhancement information), a filter value (filter information), and an enhancement result (enhancement encoding information) of each pixel point are calculated.

[0182] It should be noted that by using the image enhancement method provided by the present application, S102 and S103 in Figure 1 are circularly merged, the number of memory access times is reduced, and the number of multiplication operations is reduced, so that the memory access overhead of the processor can be reduced, and the overhead of the multiplier can be reduced.

[0183] S803, end. The enhanced video image (enhanced image) is output, wherein the enhanced video image corresponds to an image composed of the enhancement results of all the pixel points.

[0184] Referring to Figure 9 , Figure 9 is a flowchart of the processing of S802 in Figure 8 provided by an embodiment of the present application; as Figure 9 shown, S802 can be implemented by S901 to S903, wherein:

[0185] S901, the video image is converted into a YUV format image.

[0186] S902, traverse the YUV format image once:

[0187] 1. Through SIMD, simultaneously obtain the dark channel value of 8 (quantity threshold) pixel points and the dark channel value of 4 pixel points around each pixel point (up, down, left and right) in one instruction cycle (cycle traversal instruction), and perform balanced weighting calculation (balanced processing) on the dark channel value of 5 pixel points (the dark channel value of each pixel point and the dark channel value of the surrounding 4 pixel points), to obtain an enhanced value;

[0188] 2. Perform mapping on the enhanced value to obtain a filter value;

[0189] 3. Perform weighted sum calculation on the filter value and the enhanced value to obtain the enhanced result of each pixel point.

[0190] When the traversal is completed, the YUV format enhanced image is obtained.

[0191] It should be noted that when traversing the YUV format image, the position of each row or column of pixel points is temporarily stored, so that when the information of the pixel points is taken, the temporarily stored result is directly added to the offset information, reducing the multiplication processing. In addition, the experience value is fixed during calculation, avoiding repeated calculation. In addition, during calculation, the processing mode of shifting first, then calculating and then shifting again is adopted, the data to be calculated is converted from floating point type to integer type, and the calculation efficiency is improved.

[0192] S903, format conversion is performed on the YUV format enhanced image to obtain an enhanced video image. Here, the YUV format enhanced image is converted into an image in the format of a video image.

[0193] It should be noted that when the image enhancement method provided in the embodiment of the present application is used for dark scene enhancement, if the obtained image is directly in YUV format, S902 is directly executed, and S903 is not executed. If the obtained image is not in YUV format, S901 to S903 are executed.

[0194] It should be further noted that, referring to Figure 10 , Figure 10 is a single instruction multiple data stream processing mode provided by the embodiment of the present application; as shown in Figure 10 , one instruction cycle 10-1 can obtain the dark channel value of 8 pixel points and the dark channel value of 4 pixel points around each pixel point 10-2; compared with Figure 11 the single instruction single data stream processing mode (one instruction cycle 11-1 only obtains the dark channel value of 1 pixel point and the dark channel value of the surrounding 4 pixel points 11-2), the parallelism is higher.

[0195] In the embodiments of this application, when the processor of the image enhancement device includes a CPU and a GPU (Graphics Processing Unit), the incremental CPU usage can be further reduced.

[0196] The image enhancement method provided in this application is cross-platform, for example, it can be applied to macOS desktop, Windows desktop, Android mobile, and iOS mobile. The application results (CPU increment) of the image enhancement method provided in this application on macOS desktop, Windows desktop, Android mobile, and iOS mobile are described below:

[0197] The iOS mobile app has a resolution of 640*360 and a frame rate of 30 FPS (Frames Per Second). Figure 7 When option 7-23 is selected to enable dark scene enhancement, the following is used: Figure 1 The described dark scene enhancement process has an average CPU increment of 1.99 per core; while the image enhancement method provided in this application embodiment has an average CPU increment of 0.48 per core.

[0198] iOS mobile device: resolution 1280*720, frame rate 30FPS. Figure 7 When option 7-23 is selected to enable dark scene enhancement, the following is used: Figure 1 The described dark scene enhancement process has an average CPU increment of 0.22 per core; while the image enhancement method provided in the embodiments of this application has an average CPU increment of 0.04 per core.

[0199] Android mobile device: resolution 640*360, frame rate 30FPS. Figure 7 When option 7-23 is selected to enable dark scene enhancement, the following is used: Figure 1 The described dark scene enhancement process has an average CPU increment of 8.18 per core; while the image enhancement method provided in this application embodiment has an average CPU increment of 0.4 per core.

[0200] Android mobile device: resolution 640*360, frame rate 30FPS. Figure 7 When option 7-23 is selected to enable dark scene enhancement, the following is used: Figure 1 The described dark scene enhancement process has an average CPU increment of 2.91 per core; while the image enhancement method provided in this application embodiment has an average CPU increment of 1.04 per core.

[0201] Windows desktop version: resolution 1280*720, frame rate 30FPS. Figure 7 When option 7-23 is selected to enable dark scene enhancement, the following is used: Figure 1 The described dark scene enhancement process has an average CPU increment of 2.13 per core; while the image enhancement method provided in this application embodiment has an average CPU increment of 0.18 per core.

[0202] Windows desktop version; resolution 1280*720, frame rate 30FPS. Figure 7 When option 7-23 is selected to enable dark scene enhancement, the following is used: Figure 1 The described dark scene enhancement process has an average CPU increment of 4.8 per core; while the image enhancement method provided in this application embodiment has an average CPU increment of 2.11 per core.

[0203] On the MacOSX desktop: resolution is 1280*720, frame rate is 30FPS. Figure 7 When option 7-23 is selected to enable dark scene enhancement, the following is used: Figure 1 The described dark scene enhancement process has an average CPU increment of 5.87 per core; while the image enhancement method provided in the embodiments of this application has an average CPU increment of 3.27 per core.

[0204] On the MacOSX desktop: resolution is 1280*720, frame rate is 30FPS. Figure 7 When option 7-23 is selected to enable dark scene enhancement, the following is used: Figure 1 The described dark scene enhancement process has an average CPU increment of 4.44 per core; while the image enhancement method provided in this application embodiment has an average CPU increment of 1.6 per core.

[0205] In summary, the image enhancement method provided in this application, when enabling dark scene enhancement, compared to... Figure 1 The described dark scene enhancement schemes have low CPU increments, some even approaching zero, thus consuming less power. Furthermore, when the image enhancement method provided in this application's embodiments is executed in a device including both a CPU and a GPU, the CPU increment is even lower.

[0206] The following description continues to illustrate the exemplary structure of the image enhancement device 255 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 3 As shown, the software module stored in the image enhancement device 255 of the memory 250 may include:

[0207] Image encoding module 2551 is used to acquire the encoding information sequence of the image to be enhanced, wherein the encoding information sequence is a sequence composed of the encoding information of each pixel;

[0208] The image enhancement module 2552 is configured to traverse the sequence of encoded information, obtain enhancement information and filter information corresponding to the enhancement information for the encoded information of the traversed pixel point, and enhance the encoded information of the traversed pixel point by combining the enhancement information and the filter information, to obtain enhanced encoded information.

[0209] The image enhancement module 2552 is further configured to obtain a sequence of enhanced encoded information corresponding to the sequence of encoded information when the traversal of the sequence of encoded information is completed.

[0210] The image decoding module 2553 is configured to decode the enhanced image corresponding to the to-be-enhanced image based on the sequence of enhanced encoded information.

[0211] In an embodiment of the present application, the image enhancement module 2552 is further configured to generate a periodic traversal instruction based on a quantity threshold, wherein the quantity threshold is determined based on a register specification and an occupied space of the encoded information of the pixel point; and in response to the periodic traversal instruction, the encoded information of the quantity threshold pixel points and the associated pixel point encoded information corresponding to the encoded information of each traversed pixel point are sequentially traversed from the sequence of encoded information; and the enhancement information and the filter information corresponding to the enhancement information are obtained based on the associated pixel point encoded information corresponding to the encoded information of each traversed pixel point.

[0212] In an embodiment of the present application, the image enhancement module 2552 is further configured to shift the encoded information of each traversed pixel point and the associated pixel point encoded information respectively; perform equalization processing on the shifted encoded information of each traversed pixel point and the shifted associated pixel point encoded information; determine an equalization processing result as initial enhancement information; map the initial enhancement information based on a filter mapping relationship to map out initial filter information; perform reverse shifting on the initial enhancement information to obtain the enhancement information, and perform reverse shifting on the initial filter information to obtain the filter information.

[0213] In an embodiment of the present application, the image enhancement module 2552 is further configured to perform equalization processing on the encoded information of each traversed pixel point and the associated pixel point encoded information, and determine an equalization processing result as the enhancement information; and map the enhancement information based on a filter mapping relationship to determine a mapping result as the filter information.

[0214] In the embodiment of the present application, the image enhancement module 2552 is further configured to obtain a first weighted result of the enhancement information and the enhancement weight; obtain a second weighted result of the filtering information and the filtering weight; and determine a combination result of the first weighted result and the second weighted result as the enhancement encoding information, wherein the enhancement encoding information is a result of enhancing the encoding information of the traversed pixel point.

[0215] In the embodiment of the present application, the image enhancement module 2552 is further configured to store the enhancement weight; and obtain the first weighted result of the enhancement information and the stored enhancement weight.

[0216] In the embodiment of the present application, the image enhancement module 2552 is further configured to obtain a pixel point layout of the image to be enhanced; determine, based on the pixel point layout, storage location information corresponding to each group of pixel points in the image to be enhanced; store the storage location information corresponding to each group of pixel points in the image to be enhanced; obtain to-be-traversed row and column information in response to the periodic traversal instruction; determine a target group and bias position information based on the to-be-traversed row and column information; determine target position information corresponding to the target group based on the stored storage location information corresponding to each group of pixel points in the image to be enhanced; determine to-be-traversed position information based on the target position information and the bias position information; and traverse, from the to-be-traversed position information of the image encoding information, the encoding information of the number-threshold pixel points and the associated pixel point encoding information corresponding to the encoding information of each traversed pixel point in sequence.

[0217] In the embodiment of the present application, the image encoding module 2551 is configured to present a first enhancement control corresponding to the image to be enhanced; and obtain the encoding information sequence of the image to be enhanced in response to a first enhancement operation acting on the first enhancement control.

[0218] In the embodiment of the present application, the image enhancement apparatus 255 further includes an image presentation module 2554 configured to present the enhanced image.

[0219] In the embodiment of the present application, the image enhancement apparatus 255 further includes an image determination module 2555 configured to present a second enhancement control for played video information; and determine a frame image corresponding to the video information as the image to be enhanced in response to a second enhancement operation acting on the second enhancement control.

[0220] In the embodiment of the present application, the image presentation module 2554 is further configured to replace presentation of the frame image with the enhanced image.

[0221] In the embodiment of the present application, the image encoding module 2551 is further configured to acquire a color channel image of the image to be enhanced; acquire a minimum color channel value of each pixel point based on the color channel image, and construct a grayscale image corresponding to the image to be enhanced based on the minimum color channel value of each pixel point; acquire a minimum grayscale value of each pixel point in a sliding window based on the grayscale image, and construct the encoding information sequence corresponding to the image to be enhanced based on the minimum grayscale value of each pixel point in the sliding window.

[0222] In the embodiment of the present application, the image decoding module 2553 is further configured to decode an initial enhanced image based on the enhanced encoding information sequence; and perform post-processing on the initial enhanced image to obtain the enhanced image corresponding to the image to be enhanced, wherein the post-processing includes at least one of denoising processing, sharpening processing and correction processing.

[0223] The embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the image enhancement method provided in the embodiment of the present application.

[0224] The embodiment of the present application provides a computer readable storage medium storing executable instructions, wherein the executable instructions are stored in the computer readable storage medium. When the executable instructions are executed by a processor, the processor will execute the image enhancement method provided in the embodiment of the present application, for example, as shown in the image enhancement method. Figure 4

[0225] In some embodiments, the computer readable storage medium can be FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disc, or CD-ROM, etc. memory; or various devices including one or any combination of the above memories.

[0226] In some embodiments, the executable instructions can be in the form of programs, software, software modules, scripts or codes, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and can be deployed in any form, including being deployed as independent programs or being deployed as modules, components, subroutines or other units suitable for use in a computing environment.

[0227] ​By way of example, executable instructions can correspond to a file in a file system, can be stored in a portion of a file that is used to store other programs or data, can be stored as one or more separate files in a file system, or can be stored and retrieved from another memory medium, such as a tape, a magnetic disk, or an optical disc.

[0228] By way of example, the executable instructions can be deployed to be executed on one computer device, or on multiple computer devices that are located at one site, or that are distributed across multiple sites and that are interconnected through a communication network.

[0229] In summary, according to the embodiments of the present application, in the process of enhancing the encoding information of each pixel point, the filter information corresponding to the enhancement information is calculated at the same time as the enhancement information is calculated for the encoding information of the pixel point traversed from the encoding information sequence, and the encoding information of each pixel point is enhanced based on the enhancement information and the filter information; therefore, the enhancement of the encoding information of each pixel point can be completed through one traversal, so that the resource consumption is less and the efficiency of dark scene enhancement of the image can be improved. In addition, the SIMD is used to improve the parallelism of the processor in the dark scene enhancement process; and the experience value or the experience parameter is solidified to avoid the processing of calculating the experience value or the experience parameter; and the calculation efficiency is improved through the floating-point type conversion reshaping and recalculation; in summary, the processor processing efficiency in the dark scene enhancement process can be improved and the processor increment can be improved.

[0230] The above merely provides an example of the embodiments of the present application, but is not used to limit the protection scope of the present application. Any modification, equivalent replacement, and improvement within the spirit and scope of the present application shall be included in the protection scope of the present application.

Claims

1. An image enhancement method, characterized in that, include: Obtain the encoded information sequence of the image to be enhanced, wherein the encoded information sequence is a sequence composed of the encoded information of each pixel; The cycle traversal instruction is generated based on a quantity threshold, wherein the quantity threshold is determined based on the register specification and the space occupied by the encoding information of the pixel. In response to the periodic traversal instruction, the encoded information of the number of pixels and the associated pixel encoded information corresponding to the encoded information of each traversed pixel are sequentially traversed from the encoded information sequence. Based on the encoding information of the associated pixel corresponding to the encoding information of each traversed pixel, enhancement information and filtering information corresponding to the enhancement information are obtained. The encoding information of the traversed pixel is enhanced by combining the enhancement information and the filtering information to obtain enhanced encoding information. When the encoded information sequence has been traversed, the enhanced encoded information sequence corresponding to the encoded information sequence is obtained; Based on the enhanced coding information sequence, the enhanced image corresponding to the image to be enhanced is decoded.

2. The method according to claim 1, characterized in that, The process of obtaining enhancement information and filtering information corresponding to the enhancement information based on the encoding information of the associated pixel points corresponding to the encoding information of each traversed pixel point includes: The encoding information of each traversed pixel and the encoding information of the associated pixel are shifted respectively; The encoding information of each traversed pixel after shifting and the encoding information of the associated pixel after shifting are subjected to equalization processing, and the equalization processing result is determined as the initial enhancement information; The initial enhancement information is mapped based on the filtering mapping relationship to obtain the initial filtering information; The initial enhancement information is reverse-shifted to obtain the enhancement information, and the initial filtering information is reverse-shifted to obtain the filtering information.

3. The method according to claim 1, characterized in that, The process of obtaining enhancement information and filtering information corresponding to the enhancement information based on the encoding information of the associated pixel points corresponding to the encoding information of each traversed pixel point includes: The encoding information of each traversed pixel and the encoding information of the associated pixel are subjected to equalization processing, and the equalization processing result is determined as the enhancement information; The enhanced information is mapped based on the filtering mapping relationship, and the mapping result is determined as the filtered information.

4. The method according to any one of claims 1 to 3, characterized in that, The enhancement information and the filtering information are combined to enhance the encoding information of the traversed pixels, resulting in enhanced encoding information, including: Obtain the first weighted result of the enhancement information and enhancement weights; Obtain the second weighted result of the filtering information and the filtering weights; The combination of the first weighted result and the second weighted result is determined as the enhanced coding information, wherein the enhanced coding information is the result of enhancing the coding information of the traversed pixels.

5. The method according to claim 4, characterized in that, Before obtaining the first weighted result of the enhancement information and enhancement weights, the method further includes: Store the enhanced weights; The first weighted result of obtaining the enhanced information and the enhanced weights includes: Obtain the first weighted result of the enhancement information and the stored enhancement weights.

6. The method according to claim 1, characterized in that, Before generating the periodic traversal instruction based on the quantity threshold, the method further includes: Obtain the pixel layout of the image to be enhanced; Based on the pixel layout, determine the storage location information corresponding to each group of pixels in the image to be enhanced; Store the storage location information corresponding to each group of pixels in the image to be enhanced; The step of responding to the periodic traversal instruction by sequentially traversing the encoded information of the number of pixels (a threshold number) from the encoded information sequence, and the associated pixel encoded information corresponding to the encoded information of each traversed pixel, includes: In response to the periodic traversal instruction, obtain the row and column information to be traversed; The target group and offset position information are determined based on the row and column information to be traversed; Based on the stored location information of each group of pixels in the image to be enhanced, the target location information corresponding to the target group is determined. Based on the target location information and the offset location information, determine the location information to be traversed; Starting from the position information to be traversed in the image encoding information, sequentially traverse the encoding information of the number of pixels that are the threshold number, and the encoding information of the associated pixels corresponding to the encoding information of each traversed pixel.

7. The method according to any one of claims 1 to 3 and 6, characterized in that, The step of obtaining the encoded information sequence of the image to be enhanced includes: Present the first enhancement control corresponding to the image to be enhanced; In response to a first enhancement operation applied to the first enhancement control, the sequence of encoded information of the image to be enhanced is obtained; After decoding the enhanced image corresponding to the image to be enhanced based on the enhanced coding information sequence, the method further includes: The enhanced image is then presented.

8. The method according to any one of claims 1 to 3, 6, characterized in that, Before obtaining the encoded information sequence of the image to be enhanced, the method further includes: A second enhanced control is presented based on the information of the video being played; In response to a second enhancement operation applied to the second enhancement control, the frame image corresponding to the video information is determined as the image to be enhanced; After decoding the enhanced image corresponding to the image to be enhanced based on the enhanced coding information sequence, the method further includes: The enhanced image is used to replace the rendering of the frame image.

9. The method according to any one of claims 1 to 3, 6, characterized in that, The step of obtaining the encoded information sequence of the image to be enhanced includes: Obtain the color channel images of the image to be enhanced; Based on the color channel image, the minimum color channel value of each pixel is obtained, and a grayscale image corresponding to the image to be enhanced is constructed based on the minimum color channel value of each pixel. Based on the grayscale image, the minimum grayscale value of each pixel in the sliding window is obtained, and the encoded information sequence corresponding to the image to be enhanced is constructed based on the minimum grayscale value of each pixel in the sliding window.

10. The method according to any one of claims 1 to 3, 6, characterized in that, Decoding the enhanced image corresponding to the image to be enhanced based on the enhanced coding information sequence includes: Based on the enhanced coding information sequence, the initial enhanced image is decoded; The initial enhanced image is post-processed to obtain the enhanced image corresponding to the image to be enhanced, wherein the post-processing includes at least one of denoising processing, sharpening processing, and correction processing.

11. An image enhancement device, characterized in that, include: An image encoding module is used to acquire an encoding information sequence of the image to be enhanced, wherein the encoding information sequence is a sequence composed of the encoding information of each pixel. An image enhancement module is used to generate periodic traversal instructions based on a quantity threshold, wherein the quantity threshold is determined based on register specifications and the space occupied by the encoding information of the pixels; in response to the periodic traversal instructions, it sequentially traverses the encoding information of the number of pixels up to the quantity threshold from the encoding information sequence, and the associated pixel encoding information corresponding to the encoding information of each traversed pixel; based on the associated pixel encoding information corresponding to the encoding information of each traversed pixel, it obtains enhancement information and filtering information corresponding to the enhancement information, and combines the enhancement information and the filtering information to enhance the encoding information of the traversed pixels to obtain enhanced encoding information; The image enhancement module is also used to obtain an enhanced encoded information sequence corresponding to the encoded information sequence after traversing the encoded information sequence; An image decoding module is used to decode the enhanced image corresponding to the image to be enhanced based on the enhanced encoding information sequence.

12. The apparatus as claimed in claim 11, characterized in that, The image enhancement module is also used for: The encoding information of each traversed pixel and the encoding information of the associated pixel are shifted respectively; the encoding information of each traversed pixel and the encoding information of the associated pixel after shifting are subjected to equalization processing, and the equalization processing result is determined as the initial enhancement information; The initial enhancement information is mapped based on the filtering mapping relationship to obtain the initial filtering information; The initial enhancement information is reverse-shifted to obtain the enhancement information, and the initial filtering information is reverse-shifted to obtain the filtering information.

13. The apparatus as claimed in claim 11, characterized in that, The image enhancement module is also used for: The encoding information of each traversed pixel and the encoding information of the associated pixel are subjected to equalization processing, and the equalization processing result is determined as the enhancement information; The enhanced information is mapped based on the filtering mapping relationship, and the mapping result is determined as the filtered information.

14. The apparatus according to any one of claims 11 to 13, characterized in that, The image enhancement module is also used for: Obtain the first weighted result of the enhancement information and enhancement weight; obtain the second weighted result of the filtering information and filtering weight; determine the combination result of the first weighted result and the second weighted result as the enhancement encoding information, wherein the enhancement encoding information is the result of enhancing the encoding information of the traversed pixels.

15. The apparatus as claimed in claim 14, characterized in that, The image enhancement module is also used for: Store the enhancement weights; obtain the first weighted result of the enhancement information and the stored enhancement weights.

16. The apparatus as claimed in claim 11, characterized in that, The image enhancement module is also used for: Obtain the pixel layout of the image to be enhanced; based on the pixel layout, determine the storage location information corresponding to each group of pixels in the image to be enhanced; store the storage location information corresponding to each group of pixels in the image to be enhanced. In response to the periodic traversal instruction, obtain the row and column information to be traversed; The target group and offset position information are determined based on the row and column information to be traversed; Based on the stored location information of each group of pixels in the image to be enhanced, the target location information corresponding to the target group is determined. Based on the target location information and the offset location information, determine the location information to be traversed; Starting from the position information to be traversed in the image encoding information, sequentially traverse the encoding information of the number of pixels that are the threshold number, and the encoding information of the associated pixels corresponding to the encoding information of each traversed pixel.

17. An image enhancement device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the image enhancement method according to any one of claims 1 to 10.

18. A computer-readable storage medium, characterized in that, It stores executable instructions for implementing the image enhancement method according to any one of claims 1 to 10 when executed by a processor.

19. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the image enhancement method according to any one of claims 1 to 10.

Images