Image processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an image processing method and device, electronic equipment and a storage medium, and is at least applied to the field of artificial intelligence and the field of image brightness adjusting.The method comprises the steps that brightness information of at least one pixel point in a to-be-processed image is extracted, and the brightness information of at least one pixel point in the to-be-processed image is obtained according to the brightness information of the at least one pixel point; determining an adjustment table suitable for the to-be-processed image from a preset image array; for each pixel point, querying a target brightness adjustment value corresponding to the pixel point from an adjustment table based on the brightness information of the pixel point; and carrying out image brightness processing on the to-be-processed image by adopting the target brightness adjustment value of each pixel point to obtain a processed image. According to the method and the device, the calculation complexity and cost can be greatly reduced, and calculation resources are saved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of the Internet, and relate to but are not limited to an image processing method, device, electronic device and storage medium. Background Art

[0002] Low-light image enhancement technology is a method of improving the brightness and clarity of low-light images by processing them. It has important applications in areas such as unmanned driving and medical imaging. As for video conferencing systems, since the system is used in places other than offices, homes, and conference rooms, the quality of images captured by the camera will also be affected by ambient light. In low-light environments, the image quality will be blurred and dim, affecting the user's visual experience and communication effect. Therefore, low-light image enhancement technology also plays an important role in video conferencing.

[0003] In related technologies, a deep learning-based image enhancement method is usually used to implement low-light image processing. During the implementation process, the three HSV channels of the image to be processed (i.e., the low-light image) are respectively input into three different image enhancement models. The image enhancement model is composed of some convolutional layers, pooling layers, deconvolution layers, batch normalization layers, and activation layers. The output of the image enhancement model is the HSV channel of the enhanced image.

[0004] However, in the related art, three different image enhancement models with complex structures are used to implement the low-light image processing process, which has high computational complexity and cost, and will greatly waste computing resources. Summary of the invention

[0005] The embodiments of the present application provide an image processing method, device, electronic device and storage medium, which can at least be applied to the fields of artificial intelligence and image brightness adjustment. By querying a preset image array, the target brightness adjustment value of each pixel in the image to be processed can be determined, which can greatly reduce the calculation complexity and cost and save computing resources.

[0006] The technical solution of the embodiment of the present application is implemented as follows:

[0007] An embodiment of the present application provides an image processing method, including: extracting brightness information of at least one pixel in an image to be processed; determining an adjustment table suitable for the image to be processed from a preset image array based on the brightness information of the at least one pixel; the adjustment table includes a brightness adjustment value of each pixel having brightness information; for each pixel, based on the brightness information of the pixel, querying a target brightness adjustment value corresponding to the pixel from the adjustment table; and performing image brightness processing on the image to be processed using the target brightness adjustment value of each pixel to obtain a processed image.

[0008] An embodiment of the present application provides an image processing device, which includes: an information extraction module, which is used to extract brightness information of at least one pixel in an image to be processed; a determination module, which is used to determine an adjustment table suitable for the image to be processed from a preset image array according to the brightness information of the at least one pixel; the adjustment table includes a brightness adjustment value of each pixel with brightness information; a query module, which is used to query a target brightness adjustment value corresponding to each pixel from the adjustment table based on the brightness information of the pixel; and an image brightness processing module, which is used to perform image brightness processing on the image to be processed using the target brightness adjustment value of each pixel to obtain a processed image.

[0009] In some embodiments, the determination module is also used to: determine the image brightness mean of the image to be processed based on the brightness information of the at least one pixel point; query the adjustment table corresponding to the image brightness mean in the preset image array; the preset image array is a preset image three-dimensional array, and the adjustment table is a two-dimensional enhancement table corresponding to the image brightness mean.

[0010] In some embodiments, the preset image three-dimensional array includes multiple two-dimensional enhancement tables, each of which corresponds to a brightness range; the determination module is also used to: determine the brightness range where the image brightness mean is located as the target brightness range; and determine the two-dimensional enhancement table corresponding to the target brightness range as the queried adjustment table corresponding to the image brightness mean.

[0011] In some embodiments, the query module is also used to: determine the image coordinates of each of the pixel points in the image to be processed; based on the image coordinates, determine the four pixel points located around each of the pixel points and adjacent to the pixel point as the adjacent pixel points; for each of the pixel points, based on the brightness information of the pixel point and the brightness information of the multiple adjacent pixel points, query the target brightness adjustment value corresponding to the pixel point from the adjustment table.

[0012] In some embodiments, the query module is also used to: determine the brightness value of an adjacent pixel corresponding to the pixel based on the brightness information of the pixel and the brightness information of the multiple adjacent pixels; and query the target brightness adjustment value corresponding to the pixel from the adjustment table using the brightness information of the pixel and the brightness value of the adjacent pixel as indexes.

[0013] In some embodiments, the query module is also used to: determine the pixel brightness mean between the pixel and each of the adjacent pixel points based on the brightness information of the pixel point and the brightness information of the multiple adjacent pixel points to obtain multiple pixel brightness means; calculate the average of the multiple pixel brightness means; round the average to obtain an adjacent pixel brightness value in integer form.

[0014] In some embodiments, the device also includes: a difference determination module, used to determine the difference between the brightness information of the pixel point and the brightness value of the adjacent pixel point; an index value determination module, used to determine the index value according to the difference when the absolute value of the difference is greater than the difference threshold; the query module is also used to query the target brightness adjustment value corresponding to the pixel point from the adjustment table based on the index value.

[0015] In some embodiments, the device also includes: an image format determination module, used to determine the image format of the image to be processed; an image format conversion module, used to convert the image format of the image to be processed if the image format is a color channel format, so as to convert the color channel format into a brightness channel format; the information extraction module is also used to: extract the information of the brightness channel of the image to be processed having the brightness channel format, and obtain the brightness information of at least one pixel in the image to be processed.

[0016] In some embodiments, the image brightness processing module is also used to: replace the brightness information of the corresponding pixel in the brightness channel format with the target brightness adjustment value of each pixel to obtain updated image data in the brightness channel format; perform image format conversion on the updated image data in the brightness channel format to obtain updated image data in the color channel format; perform image rendering based on the updated image data in the color channel format to obtain the processed image.

[0017] In some embodiments, the device further includes: a processing module, which is used to determine the preset image array by looping and iterating the following steps: obtaining sample data, wherein the sample data includes a sample image to be processed and a label brightness adjustment value; extracting brightness information of the sample image to be processed to obtain sample brightness information of each sample pixel in the sample image to be processed; determining a sample adjustment table applicable to the sample image to be processed from an image array according to the sample brightness information of each sample pixel; the sample adjustment table includes a sample brightness adjustment value of the sample pixel for each sample brightness information; for each sample pixel, based on the sample brightness information of the sample pixel, querying a target sample brightness adjustment value corresponding to the sample pixel from the sample adjustment table; performing loss calculation on the target sample brightness adjustment value and the label brightness adjustment value corresponding to the sample pixel to obtain a loss result; based on the loss result, updating the data in the image array to obtain an updated image array; when the loss result of the loop iteration process meets a preset convergence condition, stopping the loop iteration process, and determining the updated image array obtained when the loop iteration process is stopped as the preset image array.

[0018] In some embodiments, the loop iteration process includes N rounds of loop processing, where N is an integer greater than 1; the processing module is also used to: determine the image array during the first round of loop processing of the loop iteration process based on the sample brightness information of each sample pixel point in the sample image to be processed; wherein, after the first type of loop processing, the sample brightness adjustment value corresponding to each of the sample pixel points is the same as the sample brightness information of the corresponding sample pixel point; when performing the Nth round of loop processing of the loop iteration process, obtain the updated image array obtained after the N-1th round of loop processing; and determine the updated image array obtained after the N-1th round of loop processing as the image array during the Nth round of loop processing.

[0019] An embodiment of the present application provides an electronic device, comprising: a memory for storing executable instructions; and a processor for implementing the above-mentioned image processing method when executing the executable instructions stored in the memory.

[0020] An embodiment of the present application provides a computer program product, which includes executable instructions stored in a computer-readable storage medium; wherein a processor of an electronic device reads the executable instructions from the computer-readable storage medium and implements the above-mentioned image processing method when executing the executable instructions.

[0021] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute the executable instructions to implement the above-mentioned image processing method.

[0022] The embodiment of the present application has the following beneficial effects: by extracting the brightness information of at least one pixel in the image to be processed; then determining the adjustment table applicable to the image to be processed from the preset image array according to the brightness information of at least one pixel; and for each pixel, querying the target brightness adjustment value corresponding to the pixel from the adjustment table based on the brightness information of the pixel, so that after obtaining the target brightness adjustment value of each pixel, the target brightness adjustment value of each pixel is used to perform image brightness processing on the image to be processed to obtain the processed image. In this way, the target brightness adjustment value of each pixel in the image to be processed can be determined by querying the preset image array, which can greatly reduce the computational complexity and cost and save computational resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a schematic diagram of a low-light image enhancement method based on deep learning in the related art;

[0024] Figure 2 It is a structural schematic diagram of an image enhancement model in the related art;

[0025] Figure 3 is an optional architecture diagram of an image processing system provided in an embodiment of the present application;

[0026] Figure 4 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0027] Figure 5 It is an optional flowchart of the image processing method provided in the embodiment of the present application;

[0028] Figure 6 is another optional flowchart of the image processing method provided in the embodiment of the present application;

[0029] Figure 7 This is a schematic diagram of an implementation process for determining the brightness value of adjacent pixels provided in an embodiment of the present application;

[0030] Figure 8 It is a schematic diagram of the implementation flow of the method for obtaining a preset image array provided in an embodiment of the present application;

[0031] Fig. 9 This is a diagram showing the effect of applying the method of an embodiment of the present application to video conferencing software;

[0032] Fig.10 It is a schematic diagram of the implementation flow of the image processing method provided in the embodiment of the present application;

[0033] Fig.11 It is a schematic diagram of the inference phase flow of a three-dimensional array provided in an embodiment of the present application;

[0034] Fig.12 This is a diagram of the enhancement effect of using the method of the embodiment of the present application in a low-light scene;

[0035] Fig.13 This is a diagram of the enhancement effect of using the method of an embodiment of the present application in a scene with sufficient brightness. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0037] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meaning as those commonly understood by those skilled in the art of the technical field of the embodiments of the present application. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0038] Before explaining the image processing method of the embodiment of the present application, the method in the related art is first described.

[0039] Low-light image enhancement technology is a method of processing low-light images to improve their brightness and clarity. It has important applications in unmanned driving, medical imaging and other fields.

[0040] The low-light image enhancement method based on deep learning in related technologies is as follows: Figure 1 As shown, first, in step S1, the captured traffic image in RGB space is converted into HSV space; then, in step S2, the H, S, and V components are processed respectively using the image enhancement model to obtain enhanced H, S, and V components; finally, in step S3, the enhanced H, S, and V components are converted into RGB space to obtain a processed image.

[0041] Among them, the structure of the image enhancement model is as follows Figure 2As shown, it includes a coding and decoding network 201, an optimization model network 202 and a residual neural network 203. In step S2, when the image enhancement model is used to process the H, S, and V components respectively, the H, S, and V components can be input into the image enhancement model respectively, and the H, S, and V components are processed in sequence by the coding and decoding network 201, the optimization model network 202 and the residual neural network 203 of the image enhancement model, and finally the enhanced H, S, and V components are output.

[0042] However, the main disadvantage of the solution in the related art is that the computational cost is too high, and it is almost impossible to perform real-time calculations on the end side. Because the solution uses a convolutional neural network for parameter learning, the huge input and output sizes will generate massive calculations. And it is impossible to reduce the size by downsampling, because downsampling will cause blurring of the processed image. In addition, on ordinary computers without independent graphics cards, each frame of 1080p video may take seconds. The high computational cost makes it impossible for this type of algorithm to process the input video in real time on low-end and mid-range devices. Therefore, the solution in the related art cannot be used in video conferencing systems.

[0043] In addition, the deployment cost of the solutions in the related technologies is also relatively high, because the terminal devices usually include different platforms such as personal computers (PCs), Macs, iOS and Android, and each platform has different engines for neural network reasoning, such as TensorRT, NCNN and CoreML, etc. Therefore, the algorithm deployment process needs to be adapted to each platform, which has high labor costs.

[0044] In order to solve at least one of the above problems existing in the related art, an embodiment of the present application provides an image processing method. The advantage of this method over the solution in the related art is that it greatly reduces the computing pressure and deployment cost while taking into account the automatic learning ability. In the training stage, the embodiment of the present application also automatically learns the mapping relationship between input pixels and output pixels in a data-driven manner, thereby avoiding the inability of manually defined rules to achieve good scene adaptability. Unlike the related technical solutions, in the test reasoning stage, the embodiment of the present application does not need to load the neural network, but calculates through the table lookup method, which is extremely fast, so there is no need to use the inference engine, so that each end can use a set of codes, saving the adaptation cost.

[0045] The image processing method provided by the embodiment of the present application first extracts brightness information of the image to be processed to obtain brightness information of at least one pixel in the image to be processed; then, according to the brightness information of at least one pixel, determines an adjustment table suitable for the image to be processed from a preset image array; the adjustment table includes brightness adjustment values ​​of pixels of each brightness information; then, for each pixel, based on the brightness information of the pixel, queries the target brightness adjustment value corresponding to the pixel from the adjustment table; finally, uses the target brightness adjustment value of each pixel to perform image brightness processing on the image to be processed to obtain a processed image. In this way, the target brightness adjustment value of each pixel in the image to be processed can be determined by querying the preset image array, which can greatly reduce the computational complexity and cost, save computing resources, and realize real-time computing on the end side.

[0046] Here, firstly, an exemplary application of the image processing device of the embodiment of the present application is described, and the image processing device is an electronic device for implementing the image processing method. In one implementation, the image processing device (i.e., electronic device) provided in the embodiment of the present application can be implemented as a terminal or a server. In one implementation, the image processing device provided in the embodiment of the present application can be implemented as a laptop, a tablet computer, a desktop computer, a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable game device, an intelligent robot, an intelligent home appliance, and an intelligent vehicle-mounted device, etc., any terminal with image processing function and data query function; in another implementation, the image processing device provided in the embodiment of the present application can also be implemented as a server, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms. The terminal and the server can be directly or indirectly connected by wired or wireless communication, which is not limited in the embodiment of the present application. Next, an exemplary application when the image processing apparatus is implemented as a server will be described.

[0047] See also Figure 3 , Figure 3: is an optional architecture diagram of the image processing system provided by the embodiment of the present application. The embodiment of the present application is explained by applying the image processing method to any video conferencing application as an example. During the operation of the video conferencing application, due to the different environments of the participants, the ambient light of any participant may be dark, resulting in blurry and dim conference videos, affecting the user's visual experience and communication effect. Therefore, in order to improve the problems existing in the conference video of the participants, the image processing method provided by the embodiment of the present application can be used to adjust the brightness of the conference video, thereby improving the user's visual experience and the video conference participation effect.

[0048] In the embodiment of the present application, the image processing system 10 includes at least two terminals (such as Figure 3 The terminal 100-1 and the terminal 100-2 in the network 200, the network 200 and the server 300. The server 300 may be a server of a video conferencing application. The server 300 may constitute an image processing device of an embodiment of the present application. The terminal 100-1 and the terminal 100-2 are connected to the server 300 via the network 200, and the network 200 may be a wide area network or a local area network, or a combination of the two.

[0049] In the embodiment of the present application, a video conferencing application is installed on both the terminal 100-1 and the terminal 100-2, and the user can join the video conference by operating the client of the video conferencing application on the terminal 100-1 and the terminal 100-2. After joining the video conference, the terminal 100-1 and the terminal 100-2 can both turn on the image acquisition device of the local terminal to collect the video of the user in the current environment to realize the video conference. During the video conference, the terminal 100-1 and the terminal 100-2 will send each video frame of the collected video in the current environment as an image to be processed to the server 300 through the network 200. After receiving the image to be processed, the server 300 will use the image processing method provided in the embodiment of the present application to extract the brightness information of the image to be processed, and obtain the brightness information of at least one pixel in the image to be processed; and according to the brightness information of at least one pixel, determine the adjustment table suitable for the image to be processed from the preset image array; then, for each pixel, query the target brightness adjustment value corresponding to the pixel from the adjustment table based on the brightness information of the pixel; finally, use the target brightness adjustment value of each pixel to perform image brightness processing on the image to be processed to obtain the processed image. After obtaining the processed image, the server 300 can send the processed image to the terminal 100-1 and the terminal 100-2 respectively, so as to display the processed image on the terminal 100-1 and the terminal 100-2.

[0050] In some embodiments, the image processing method can also be implemented through terminals, that is, the terminal 100-1 and the terminal 100-2 are both installed with a video conferencing application, and the user can operate through the client of the video conferencing application on the terminal 100-1 and the terminal 100-2 to join the video conference. After joining the video conference, the terminal 100-1 and the terminal 100-2 can both turn on the image acquisition device of the local terminal to collect the video of each in the current environment to realize the video conference. During the video conference, each of the terminals 100-1 and 100-2 will use each video frame of the video collected in the current environment as an image to be processed, and thus use the image processing method provided by the embodiment of the present application to process each of the images to be processed, for example, the terminal extracts brightness information of the image to be processed to obtain brightness information of at least one pixel in the image to be processed; the terminal determines an adjustment table suitable for the image to be processed from a preset image array based on the brightness information of at least one pixel; the terminal queries the target brightness adjustment value corresponding to the pixel from the adjustment table based on the brightness information of the pixel for each pixel; the terminal uses the target brightness adjustment value of each pixel to perform image brightness processing on the image to be processed to obtain a processed image. After the terminals 100-1 and 100-2 obtain each frame of processed images, they can send the processed images to the server 300 through the network 200, so that the processed images of each terminal are integrated into the video conference through the server 300 to form a real-time video conference with a high visual experience.

[0051] The image processing method provided in the embodiment of the present application can also be implemented based on a cloud platform and through cloud technology. For example, the server 300 can be a cloud server. The cloud server extracts brightness information of the image to be processed, or determines an adjustment table suitable for the image to be processed from a preset image array, or queries a target brightness adjustment value corresponding to the pixel from the adjustment table based on the brightness information of the pixel, or uses the target brightness adjustment value of each pixel to perform image brightness processing on the image to be processed.

[0052] In some embodiments, a cloud storage may be provided, and a preset image array may be stored in the cloud storage, or a processed image may be stored in the cloud storage. Thus, when an image brightness processing request for a new image to be processed is received, the preset image array may be directly obtained from the cloud storage to query the target brightness adjustment value corresponding to each pixel in the new image to be processed, thereby realizing rapid image brightness processing of the new image to be processed.

[0053] It should be noted here that cloud technology refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to achieve data computing, storage, processing and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool that is used on demand and is flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, in the future, each item may have its own identification mark, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately. All kinds of industry data require strong system backing support, which can be achieved through cloud computing.

[0054] Figure 4 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, Figure 4 The electronic device shown may be an image processing device, which includes: at least one processor 310, a memory 350, at least one network interface 320, and a user interface 330. The various components in the image processing device are coupled together via a bus system 340. It is understood that the bus system 340 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 340 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 340 is not described in detail. Figure 4 Various buses are labeled as bus system 340 .

[0055] The processor 310 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0056] The user interface 330 includes one or more output devices 331 that enable presentation of media content, and one or more input devices 332 .

[0057] The memory 350 may be removable, non-removable or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drive, optical disk drive, etc. The memory 350 may optionally include one or more storage devices physically away from the processor 310. The memory 350 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 350 described in the embodiment of the present application is intended to include any suitable type of memory. In some embodiments, the memory 350 can store data to support various operations, and examples of these data include programs, modules, and data structures or subsets or supersets thereof, as exemplarily described below.

[0058] An operating system 351 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic businesses and processing hardware-based tasks; a network communication module 352 is used to reach other computing devices via one or more (wired or wireless) network interfaces 320. Exemplary network interfaces 320 include: Bluetooth, wireless compatibility certification (WiFi), and Universal Serial Bus (USB), etc.; an input processing module 353 is used to detect one or more user inputs or interactions from one of the one or more input devices 332 and translate the detected inputs or interactions.

[0059] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 4 An image processing device 354 stored in a memory 350 is shown. The image processing device 354 may be an image processing device in an electronic device. The image processing device 354 may be software in the form of a program or a plug-in, and includes the following software modules: an information extraction module 3541, a determination module 3542, a query module 3543, and an image brightness processing module 3544. These modules are logical, and therefore may be arbitrarily combined or further split according to the functions implemented. The functions of each module will be described below.

[0060] In some embodiments, the device provided in the embodiments of the present application can be implemented in hardware. As an example, the device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the image processing method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), or other electronic components.

[0061] The image processing methods provided in the embodiments of the present application may be executed by an electronic device, wherein the electronic device may be a server or a terminal, that is, the image processing methods in the embodiments of the present application may be executed by a server, or by a terminal, or may be executed through interaction between a server and a terminal.

[0062] Figure 5 This is an optional flow chart of the image processing method provided in the embodiment of the present application. Figure 5 The steps shown are explained as Figure 5 As shown, the execution subject of the image processing method is a server as an example for description, and the method includes the following steps S101 to S104:

[0063] Step S101, extracting brightness information of at least one pixel in the image to be processed.

[0064] Here, the image to be processed may be an image acquired by any image acquisition method, for example, an image captured in real time, a certain frame in a captured video, an image generated after image synthesis, a certain frame in an edited video generated after video editing, or an image downloaded from the network, etc. In the embodiment of the present application, the specific acquisition method of the image to be processed may be determined according to the application scenario of the image processing method.

[0065] After the server obtains the image to be processed, it extracts brightness information of the image to be processed to obtain the brightness information of each pixel in the image to be processed. Here, brightness information extraction refers to extracting information related to brightness from each pixel in the image to be processed. Brightness information refers to the brightness value used to characterize the brightness of the pixel.

[0066] For example, the image to be processed can be converted into YUV format, where YUV is a color encoding method commonly used in various video processing components. When encoding the image to be processed, YUV allows for reduced bandwidth for chrominance, taking into account human perception. YUV is a type of true-color color space. "Y" represents luminance (Luminance or Luma), which is the grayscale value, while "U" and "V" represent chrominance (Chrominance or Chrom a), which describes the color and saturation of the image and is used to specify the color of the pixel. In the YUV format, the value of the Y channel of each pixel in the image to be processed can be extracted to obtain the brightness information of each pixel in the image to be processed.

[0067] Step S102: determining an adjustment table suitable for the image to be processed from a preset image array according to brightness information of at least one pixel.

[0068] Here, the preset image array is a mapping relationship between the brightness information of the input pixel point and the brightness information of the output pixel point learned in advance through training. The preset image array includes multiple adjustment tables, and each adjustment table includes the brightness adjustment value of the pixel point of each brightness information. Among them, the brightness adjustment value of the pixel point can be positive or negative. When the brightness adjustment value is positive, it indicates that when the brightness adjustment value is used to adjust the brightness of the pixel point, a brightness enhancement adjustment is performed; when the brightness adjustment value is negative, it indicates that when the brightness adjustment value is used to adjust the brightness of the pixel point, a brightness reduction adjustment is performed. Of course, the brightness adjustment value of the pixel point can also be zero. When the brightness adjustment value of the pixel point is zero, it indicates that the brightness of the pixel point is not adjusted.

[0069] In some embodiments, the preset image array may be a preset image three-dimensional array or a preset image two-dimensional array. If the preset image array is a preset image three-dimensional array, the preset image three-dimensional array may include multiple two-dimensional enhancement tables and two-dimensional attenuation tables, that is, the adjustment table may be a two-dimensional enhancement table or a two-dimensional attenuation table. The two-dimensional enhancement table includes the enhancement value corresponding to the brightness information of each pixel point, and the enhancement values ​​in the two-dimensional enhancement table are all positive values, which are adjustment values ​​for brightening the brightness of the corresponding pixel point in the image to be processed; the two-dimensional attenuation table includes the attenuation value corresponding to the brightness information of each pixel point, and the attenuation values ​​in the two-dimensional attenuation table are all negative values, which are adjustment values ​​for dimming the brightness of the corresponding pixel point in the image to be processed.

[0070] Each two-dimensional enhancement table or two-dimensional attenuation table corresponds to a brightness interval. If the brightness information of the image to be processed falls within a certain target brightness interval, the two-dimensional enhancement table or two-dimensional attenuation table corresponding to the target brightness interval is the adjustment table suitable for the image to be processed. If the preset image array is a preset image two-dimensional array, the preset image three-dimensional array includes multiple one-dimensional enhancement tables and one-dimensional attenuation tables, that is, the adjustment table can be a one-dimensional enhancement table or a one-dimensional attenuation table, that is, the queried adjustment table is a one-dimensional table.

[0071] In the implementation process, the image brightness mean corresponding to the brightness information of all pixels in the image to be processed can be calculated, and the image brightness mean is the mean corresponding to the brightness values ​​of all pixels. Then, see in which brightness interval the image brightness mean falls, and by querying the preset image three-dimensional array, determine the two-dimensional enhancement table or two-dimensional attenuation table corresponding to the brightness interval in the preset image three-dimensional array as the two-dimensional enhancement table or two-dimensional attenuation table suitable for the image to be processed.

[0072] In the embodiment of the present application, the preset image array is learned in advance and stored, for example, the preset image array can be stored in a preset storage unit or a cloud storage. In this way, when performing image brightness processing on the image to be processed, the preset image array can be directly obtained from the preset storage unit or the cloud storage, and the steps in the image processing method of the embodiment of the present application can be implemented.

[0073] Step S103 : for each pixel, based on the brightness information of the pixel, query the target brightness adjustment value corresponding to the pixel from the adjustment table.

[0074] Here, the adjustment table can be queried with the brightness information of each pixel as an index, so as to query the target brightness adjustment value corresponding to the corresponding pixel from the adjustment table, and the target brightness adjustment value is used to enhance or weaken the brightness of the corresponding pixel. In the embodiment of the present application, a preset image array is queried for each pixel in the image to be processed to obtain the target brightness adjustment value.

[0075] In some embodiments, when the adjustment table is a two-dimensional enhancement table, since the two-dimensional enhancement table is an enhancement table with two dimensions, two index values ​​are required when querying in the two-dimensional enhancement table to be able to query a final target brightness adjustment value from the two-dimensional enhancement table. In the embodiment of the present application, the target brightness adjustment value of the pixel point, that is, the target brightness enhancement value, can be queried from the two-dimensional enhancement table in combination with the brightness value of the pixel point and the brightness values ​​of multiple adjacent pixels.

[0076] In some embodiments, when the adjustment table is a two-dimensional attenuation table, since the two-dimensional attenuation table is also an attenuation table with two dimensions, two index values ​​are required when searching in the two-dimensional attenuation table to be able to search for a final target brightness adjustment value from the two-dimensional attenuation table. In the embodiment of the present application, the target brightness adjustment value of the pixel point, i.e., the target brightness attenuation value, can be searched from the two-dimensional attenuation table in combination with the brightness value of the pixel point and the brightness values ​​of multiple adjacent pixels.

[0077] In some embodiments, the two-dimensional enhancement table and the two-dimensional attenuation table may be the same table, which may be called a two-dimensional adjustment table. In this table, if the target brightness adjustment value corresponding to the brightness information of a pixel point is an enhancement value, then the target brightness adjustment value is positive; if the target brightness adjustment value corresponding to the brightness information of a pixel point is a attenuation value, then the target brightness adjustment value is negative.

[0078] In some embodiments, a plurality of adjacent pixels corresponding to each pixel in the image to be processed may be determined first, and then, for each pixel, based on the brightness information of the pixel and the brightness information of the plurality of adjacent pixels, the target brightness adjustment value corresponding to the pixel may be queried from the adjustment table. Adjacent pixels refer to a plurality of pixels located around the pixel and adjacent to the pixel. Usually in an image, the absolute value of the difference between the brightness value of any pixel and the brightness value of the adjacent pixel is less than or equal to the difference threshold, in which case the pixel is considered to be a normal pixel. If the absolute value of the difference between the brightness value of any pixel and the brightness value of the adjacent pixel is greater than the difference threshold, it indicates that the pixel may be a noise point. For the noise point in the image to be processed, the brightness of the pixel may not be adjusted, or a suitable pixel value may be determined for the pixel as an index value, so that the two-dimensional adjustment table can be queried to further implement the brightness adjustment of the pixel.

[0079] In the embodiment of the present application, the target brightness adjustment value is used to adjust the brightness value of the corresponding pixel point. The target brightness adjustment value can be the brightness value of the corresponding pixel point after being enhanced or weakened, or it can be the brightness change value that needs to be enhanced or weakened. Here, when the target brightness adjustment value is the brightness value of the corresponding pixel point after being enhanced or weakened, the original brightness value of the pixel point can be modified to the queried target brightness adjustment value, thereby achieving the brightness adjustment of the pixel point; when the target brightness adjustment value is the brightness change value that needs to be enhanced or weakened, the sum of the original brightness value of the pixel point and the brightness change value can be calculated, and the calculation result is used as the final brightness value to be presented by the pixel point, thereby achieving the brightness adjustment of the pixel point.

[0080] Step S104 , using the target brightness adjustment value of each pixel point, performing image brightness processing on the image to be processed to obtain a processed image.

[0081] In an embodiment of the present application, after obtaining the target brightness adjustment value of each pixel, the target brightness adjustment value of each pixel is used to update the brightness value of the corresponding pixel, and then combined with other color information of each pixel, the rendering information of each pixel is obtained, and each pixel is rendered according to the rendering information to obtain a processed image.

[0082] Here, the brightness value of a pixel may be the Y channel information of the pixel, other color information may be the U channel and V channel information of the pixel, and the rendering information refers to the YUV value of the pixel.

[0083] The image processing method provided in the embodiment of the present application obtains the brightness information of at least one pixel in the image to be processed by extracting the brightness information of the image to be processed; then, according to the brightness information of at least one pixel, an adjustment table suitable for the image to be processed is determined from the preset image array; and for each pixel, the target brightness adjustment value corresponding to the pixel is queried from the adjustment table based on the brightness information of the pixel, so that after obtaining the target brightness adjustment value of each pixel, the target brightness adjustment value of each pixel is used to perform image brightness processing on the image to be processed to obtain the processed image. In this way, the target brightness adjustment value of each pixel in the image to be processed can be determined by querying the preset image array, which can greatly reduce the calculation complexity and cost, save computing resources, and because the terminal side can directly query the preset image array to complete the subsequent image brightness processing process, it can also realize real-time calculation on the terminal side. In some embodiments, if the image processing method of the embodiment of the present application is deployed on the server side, and the image brightness processing process is executed by the server, real-time calculation on the server side can be realized.

[0084] In some embodiments, the image processing system includes at least a terminal and a server. The embodiment of the present application is described by taking a video conferencing application installed on a terminal as an example. The server constitutes a background server of the video conferencing application, and the user can join the video conference by operating the client of the video conferencing application on the terminal. After joining the video conference, the terminal can turn on the image acquisition device of this terminal to collect the video of the user in the current environment to realize the video conference. During the video conference, the terminal will send each video frame of the video collected in the current environment to the server as an image to be processed. After receiving the image to be processed, the server will use the image processing method provided in the embodiment of the present application to perform image brightness processing on the image to be processed to obtain a processed image.

[0085] Since there may be multiple participants in a video conference, there may be multiple terminals in the image processing system. The embodiment of the present application is described by taking the first terminal and the second terminal as examples, wherein both the first terminal and the second terminal can turn on the image acquisition device to collect the video of each user in the current environment to realize the video conference. In the process of the video conference, the first terminal and the second terminal will send each video frame of the video collected in the current environment as an image to be processed to the server. After receiving the image to be processed, the server will use the image processing method provided in the embodiment of the present application to perform image brightness processing on the respective images to be processed to obtain the processed image. Finally, the processed images of the first terminal and the second terminal are fused to form a conference video presented on the current interface of the first terminal and the second terminal. It should be noted that for the first terminal and the second terminal, when performing image brightness processing, the entire implementation steps are the same.

[0086] In some embodiments, during a video conference, only one participant may turn on the image acquisition device. For example, if a user of the first terminal makes a report during the conference, the user of the first terminal may turn on the image acquisition device, and the users of other terminals may turn off the image acquisition device. At this time, during the video conference, only the video of the user of the first terminal in the current environment is presented. This application is explained by taking the example that the first terminal acquires the image to be processed and the second terminal only displays the conference video on the current interface.

[0087] Figure 6 is another optional flow chart of the image processing method provided in the embodiment of the present application, such as Figure 6 As shown, the method includes the following steps S201 to S222:

[0088] Step S201: The first terminal receives a trigger operation from a user.

[0089] Here, the trigger operation may be an operation performed by a user on a client of a video conferencing application, and the trigger operation may include an input operation of an image to be processed, a selection operation of an image to be processed, and an image brightness processing trigger operation, etc. The user may input an image to be processed on the client of a video conferencing application and click an image brightness processing button, thereby generating the trigger operation.

[0090] Step S202: The first terminal responds to the triggering operation and generates an image brightness processing request.

[0091] After receiving the trigger operation, the first terminal encapsulates the image to be processed into an image brightness processing request, where the image brightness processing request is used to request the server to perform image brightness processing on the image to be processed.

[0092] In some embodiments, the image brightness processing request may include the image to be processed, or may also include the image identifier of the image to be processed, where the image identifier is used to uniquely identify the corresponding image to be processed, and the image identifier may be the image name, address, image content, image type and other information of the image to be processed.

[0093] Step S203: The first terminal sends an image brightness processing request to the server.

[0094] Step S204: the server responds to the image brightness processing request, parses the image brightness processing request, and obtains the image to be processed.

[0095] In the embodiment of the present application, the image brightness processing request may include the image to be processed, or may also include the image identifier of the image to be processed. When the image to be processed includes the image to be processed, the server may parse the image brightness processing request to obtain the image to be processed; when the image to be processed includes the image identifier of the image to be processed, the server may parse the image identifier and obtain the image to be processed based on the image identifier. In the implementation process, the image identifier may be the name of the image, the storage address of the image, etc.

[0096] Step S205: the server determines the image format of the image to be processed.

[0097] Here, the image format includes but is not limited to: RGB format and YUV format. Among them, the RGB format is also called RGB color mode. The RGB color mode is a color standard in the industry. It obtains various colors by changing the three color channels of red (R), green (G), and blue (B) and superimposing them on each other. RGB represents the colors of the three channels of red, green, and blue. This standard almost includes all the colors that human vision can perceive and is one of the most widely used color systems. In computer graphics, RGB colors are often encountered. The red, green, and blue in RGB directly correspond to the visible light parts respectively. The RGB value can establish an accurate digital coordinate system. The digital coordinate system is called color space. The red part defines an axis in the coordinate system, followed by green and blue. The image format of the image to be processed is usually obtained in RGB format.

[0098] YUV format is a color encoding method. Usually digital video is compressed using YUV format. YUV is a general term for a series of color spaces that encode brightness information and color information separately. Similar to RGB, YUV also uses three values ​​to represent color, namely Y value, U value and V value.

[0099] Step S206: If the image format of the image to be processed is a color channel format, the server performs image format conversion on the image to be processed to convert the color channel format into a brightness channel format.

[0100] Here, the color channel format may be in RGB format, and the brightness channel format may be in YUV format. If the image format of the image to be processed is in RGB format, the image to be processed in RGB format may be converted into the image to be processed in YUV format. When performing image format conversion, conversion formulas of different color spaces may be used for format conversion.

[0101] For example, for RGB data and YUV data in decimal form, where U∈[-0.5,0.5]; R, G, B∈[0,1], the conversion from RGB format to YUV format can be achieved by the following formula (1):

[0102]

[0103] Among them, R, G, and B respectively represent the corresponding values ​​of each pixel of the image to be processed in the RGB format; Y, U, and V respectively represent the corresponding values ​​of each pixel of the image to be processed in the YUV format.

[0104] Alternatively, for RGB data and YUV data in decimal form, where U∈[-0.5,0.5]; R, G, B∈[0,1], the conversion from RGB format to YUV format can also be achieved by the following formula (2):

[0105]

[0106] For another example, for integer RGB data and YUV data, R, G, B [0, 255]; U, V ~ [-128, 128], the conversion from RGB format to YUV format can be achieved by the following formula (3):

[0107]

[0108] The above formulas (1) to (3) are merely exemplary. Of course, in other cases, other conversion formulas may be used to convert between RGB format and YUV format, which will not be described in detail in the embodiments of the present application.

[0109] Step S207: the server extracts information of the brightness channel of the image to be processed in the brightness channel format to obtain brightness information of each pixel in the image to be processed.

[0110] In an embodiment of the present application, the information of the brightness channel of the image to be processed in the YUV format is extracted, which can be to extract the Y value of each pixel in the image to be processed in the YUV format, and the Y value of each pixel constitutes the brightness information of the corresponding pixel.

[0111] Step S208: The server determines the image brightness mean of the image to be processed according to the brightness information of each pixel.

[0112] Here, the average of the Y values ​​of all pixels in the entire image to be processed can be calculated to obtain the average image brightness.

[0113] Step S209: the server searches the preset image array for an adjustment table corresponding to the image brightness mean value.

[0114] In some embodiments, the preset image array is a preset image three-dimensional array, and the adjustment table is a two-dimensional adjustment table corresponding to the image brightness mean. The preset image three-dimensional array includes multiple two-dimensional adjustment tables, each of which corresponds to a brightness range; please continue to refer to Figure 6 , Figure 6 It is also shown that the query adjustment table in step S209 can be implemented by the following steps S2091 to S2092:

[0115] Step S2091, determining the brightness interval where the image brightness mean value is located as the target brightness interval.

[0116] Step S2092: determine the two-dimensional adjustment table corresponding to the target brightness range as the queried adjustment table corresponding to the image brightness mean value.

[0117] Here, the target brightness interval may be used as an index to search for a two-dimensional adjustment table corresponding to the target brightness interval in the preset three-dimensional array of images, and the two-dimensional adjustment table is an adjustment table corresponding to the image brightness mean.

[0118] In some embodiments, the multiple brightness intervals in the preset image three-dimensional array do not overlap, and the lengths of the multiple brightness intervals may be the same or different. That is, when the brightness value is divided into multiple brightness intervals, the complete value interval of the brightness value may be divided into multiple sub-intervals, each of which constitutes a brightness interval. When dividing, the division may be performed in an equal division form or randomly, to obtain multiple brightness intervals with different interval lengths.

[0119] For example, the brightness value ranges from 0 to 255. The brightness values ​​from 0 to 60 (i.e., greater than or equal to 0 and less than 60) can be divided into the first interval, and the brightness values ​​from 150 to 255 (i.e., greater than or equal to 150 and less than or equal to 255) can be divided into the last interval. The brightness values ​​between 60 and 150 (i.e., greater than or equal to 60 and less than 150) can be divided into equal intervals. For example, an interval can be formed every 30 brightness values, forming three brightness intervals, namely [60, 90), [90, 120), and [120, 150]. In this way, the brightness values ​​from 0 to 255 are finally divided into the following five brightness intervals: [0, 60), [60, 90), [90, 120), [120, 150), and [150, 255].

[0120] Each brightness interval corresponds to a two-dimensional adjustment table. You can first determine in which brightness interval the image brightness mean of the image to be processed falls, and then determine the two-dimensional adjustment table corresponding to the brightness interval as the queried adjustment table. For example, if the brightness interval [0, 60) corresponds to the two-dimensional adjustment table T1, the brightness interval [60, 90) corresponds to the two-dimensional adjustment table T2, the brightness interval [90, 120) corresponds to the two-dimensional adjustment table T3, the brightness interval [120, 150) corresponds to the two-dimensional adjustment table T4, and the brightness interval [150, 255] corresponds to the two-dimensional adjustment table T5. If the image brightness mean of the image to be processed is calculated to be 125, it can be determined that the adjustment table queried from the preset image three-dimensional array is the two-dimensional adjustment table T4.

[0121] Step S210: the server determines the image coordinates of each pixel point in the image to be processed.

[0122] In the embodiment of the present application, an image coordinate can be set for each pixel in the image to be processed. For example, any vertex of the image or the center of the image can be used as the coordinate origin to set a two-dimensional image coordinate (x, y) for each pixel. The image coordinate (x, y) can constitute a position identifier of each pixel, and the position of the corresponding pixel in the image to be processed can be determined based on the position identifier.

[0123] In step S211, the server determines four pixels located around each pixel and adjacent to the pixel as adjacent pixels based on the image coordinates.

[0124] In the embodiment of the present application, there are four pixels located around the pixel and adjacent to the pixel. For example, for a pixel at position (x, y), the adjacent pixels of the pixel are respectively located at positions (x+1, y), (x-1, y), (x, y+1), and (x, y-1). The adjacent pixels can be determined based on the image coordinates of each pixel and other pixels.

[0125] Step S212: for each pixel, the server determines the brightness value of adjacent pixels corresponding to the pixel based on the brightness information of the pixel and the brightness information of multiple adjacent pixels.

[0126] In some embodiments, see Figure 7 , Figure 7 It is shown that determining the brightness value of the adjacent pixel corresponding to the pixel can be achieved by following steps S2121 to S2123:

[0127] Step S2121, based on the brightness information of the pixel point and the brightness information of multiple adjacent pixels, determine the average brightness of the pixel point between the pixel point and each adjacent pixel point, and obtain the average brightness of multiple pixels.

[0128] Here, for a pixel at any (x, y) position, the average brightness of the pixel and each adjacent pixel can be calculated, that is, the average brightness of the pixel and each adjacent pixel is calculated to obtain the average brightness of the four pixels.

[0129] For example, if the brightness value of the pixel at position (x, y) is Y x,y 、The brightness value of the pixel at position (x+1, y) is Y x+1,y 、The brightness value of the pixel at position (x-1, y) is Y x-1,y 、The brightness value of the pixel at position (x, y+1) is Y x,y+1 、The brightness value of the pixel at position (x, y-1) is Y x,y-1 , then we can calculate Get the average brightness of the four pixels.

[0130] Step S2122, calculating the average of the brightness values ​​of multiple pixels.

[0131] In the embodiment of the present application, after calculating the average brightness values ​​of four pixel points, an average value of the average brightness values ​​of the four pixel points is calculated.

[0132] Step S2123, rounding the average value to obtain the brightness value of the adjacent pixel in integer form.

[0133] Here, the rounding process may be rounding down.

[0134] Step S213: The server uses the brightness information of the pixel and the brightness values ​​of adjacent pixels as indexes to query the target brightness adjustment value corresponding to the pixel from the adjustment table.

[0135] In the embodiment of the present application, the adjustment table is a two-dimensional adjustment table with two dimensions. Therefore, when searching in the adjustment table, two index values ​​are required to be able to search for a final target brightness adjustment value from the adjustment table. In the embodiment of the present application, the brightness information of each pixel and the brightness value of the adjacent pixel corresponding to the pixel can be used as indexes to search for the target brightness adjustment value corresponding to the pixel from the adjustment table.

[0136] In some embodiments, the brightness information of each pixel can also be analyzed based on the brightness information of the pixel and the brightness values ​​of the adjacent pixels of the pixel to determine whether the brightness information of the pixel is accurate, that is, to determine whether the pixel is a noise point. This can be achieved by the following steps:

[0137] In step S214, the server determines the difference between the brightness information of the pixel and the brightness value of the adjacent pixel.

[0138] Step S215: the server determines whether the absolute value of the difference is greater than a difference threshold.

[0139] If the judgment result is yes, then execute step S216; if the judgment result is no, it indicates that the pixel point is not a noise point, then execute step S213.

[0140] Step S216: The server determines the index value according to the difference.

[0141] Here, if the difference between the brightness information of a pixel and the brightness value of an adjacent pixel is greater than a difference threshold, that is, the difference between the brightness value of a pixel and the brightness value of an adjacent pixel is greater than a difference threshold, it indicates that the pixel is a noise point, and therefore, the brightness value of the pixel can be re-determined.

[0142] When re-determining the brightness value of the pixel, the brightness value of the adjacent pixel can be determined as the brightness value of the pixel, or the average between the brightness value of the pixel and the brightness value of the adjacent pixel can be calculated, and the average can be determined as the brightness value of the pixel. Alternatively, an index value query table can be provided, and the index value query table is a pre-learned two-dimensional table, wherein the first dimension is the brightness value of the pixel, the second dimension is the brightness value of the adjacent pixel of the pixel, and the value in the two-dimensional table is the index value. In this way, when re-determining the brightness value of a certain pixel A, the brightness value used to update the pixel A can be queried from the index value query table based on the brightness value of the pixel A and the brightness value of the adjacent pixel of the pixel A as an index, and the brightness value of the pixel A is updated using the brightness value, that is, the index value is determined.

[0143] Step S217: the server searches the adjustment table for a target brightness adjustment value corresponding to the pixel point based on the index value.

[0144] Here, the brightness adjustment value corresponding to the pixel point can be queried from the adjustment table using the index value and the brightness value of the adjacent pixel point as indexes.

[0145] Step S218: The server uses the target brightness adjustment value of each pixel to replace the brightness information of the corresponding pixel in the brightness channel format to obtain image data in the updated brightness channel format.

[0146] Step S219: the server performs image format conversion on the updated brightness channel format image data to obtain updated color channel format image data.

[0147] Here, the color channel format may be RGB format, and the image data in the color channel format is the updated RGB data for each pixel of the image to be processed; the brightness channel format may be YUV format, and the image data in the brightness channel format is the updated YUV data for each pixel of the image to be processed.

[0148] The updated YUV data may be converted into updated RGB data. When performing image format conversion, a formula applicable to this image format conversion may be determined according to the data form of the updated YUV data to perform image format conversion.

[0149] Step S220: The server performs image rendering based on the image data in the updated color channel format to obtain a processed image.

[0150] Step S221: the server sends the processed image to the first terminal and the second terminal respectively.

[0151] Step S222: The first terminal and the second terminal display the processed image on the current interface.

[0152] The embodiments of the present application can also be applied to the following scenarios: the image processing method is applied to image shooting software. After a user uses the image shooting software on the terminal to take a picture, the picture can be used as an image to be processed to extract brightness information, and then the image processing method provided by the embodiments of the present application can be used to perform image brightness processing to obtain a processed image. The processed image is a picture with high authenticity and a certain visual experience.

[0153] In the implementation process, the image shooting software may also have an image brightness processing button. When the user triggers the image brightness processing button, the image shooting software turns on the image brightness processing function. In this way, when the user takes a picture, the image brightness processing is automatically performed. Alternatively, after the user takes a picture, the image brightness processing button may be displayed in the picture editing bar. When the user selects the image brightness processing button, the method provided in the embodiment of the present application is used to perform image brightness processing on the picture taken this time.

[0154] The embodiments of the present application can also be applied to the following scenarios: applying the image processing method to a video editing application or a short video application. In the video editing application, when a user edits a video, for an edited video or a video to be edited or some video clips in the edited video, the image brightness processing function can be triggered by selecting the image brightness processing button in the video editing application, thereby performing image brightness processing on each video frame in the edited video or the video to be edited or some video clips in the edited video, and finally obtaining an edited video after brightness processing, thereby improving the effect of video editing. When performing image brightness processing on each video frame in the edited video or the video to be edited or some video clips in the edited video, the following two situations are included:

[0155] In case one, it is necessary to perform image brightness enhancement processing on each video frame in an edited video or a video to be edited or some video clips in an edited video. At this time, the method provided in the embodiment of the present application can be used to query the target brightness adjustment value with a positive value for each pixel point of the video frame, and then use the positive target brightness adjustment value to adjust the brightness of each pixel point, thereby achieving image brightness enhancement of the video frame.

[0156] Case 2: It is necessary to perform image brightness reduction processing on each video frame in the edited video or the video to be edited or some video clips in the edited video. At this time, the method provided in the embodiment of the present application can be used to query each pixel point of the video frame to have a negative target brightness adjustment value, and then use the negative target brightness adjustment value to adjust the brightness of each pixel point, thereby achieving image brightness reduction of the video frame.

[0157] Case 3: It is necessary to enhance the image brightness of a part of the area and reduce the image brightness of another part of the area for each video frame in the edited video or the video to be edited or some video clips in the edited video. At this time, the method provided in the embodiment of the present application can be used to query the target brightness adjustment value of each pixel point of the video frame with a positive value or a negative value, so as to adjust the brightness of each pixel point with a positive value or a negative value, so as to reduce the image brightness of the video frame. It should be noted that if the brightness of any pixel point does not need to be adjusted, the target brightness adjustment value of the pixel point is zero.

[0158] In some embodiments, with respect to the above-mentioned preset image three-dimensional array, a method for obtaining the preset image array is further provided. Figure 8 It is a schematic diagram of the implementation flow of the method for obtaining a preset image array provided in an embodiment of the present application. The method for obtaining a preset image array can be executed by a loop iteration module, and the loop iteration model can be considered as a model training module. Among them, the loop iteration module can be a module in an image processing device (i.e., an electronic device), that is, the loop iteration module can be a server or a terminal; or, it can also be another device independent of the image processing device, that is, the loop iteration module is other electronic devices other than the server and terminal used to implement the image processing method.

[0159] like Figure 8 As shown, when training and learning the preset image array, the preset image array can be determined by looping and iterating the following steps S301 to S307 until the loss result of the loop iteration process meets the preset convergence condition, the loop iteration process is stopped, and the updated image array obtained when the loop iteration process is stopped is determined as the preset image array. In the embodiment of the present application, the preset image array is taken as a preset image three-dimensional array as an example, and can be Figure 8 The steps shown determine the preset image three-dimensional array. Of course, in other embodiments, the preset image array may also be a preset image two-dimensional array, and the method provided in the embodiment of the present application may also be used to determine the preset image two-dimensional array.

[0160] Step S301, obtaining sample data, wherein the sample data includes a sample image to be processed and a label brightness adjustment value.

[0161] Here, the sample image to be processed may be any image, and the label brightness adjustment value refers to the brightness adjustment value in the processed image obtained after artificial brightness processing is performed on the sample image to be processed.

[0162] Step S302 , extracting brightness information of the sample image to be processed, and obtaining sample brightness information of each sample pixel in the sample image to be processed.

[0163] In the embodiment of the present application, brightness information extraction refers to extracting information related to brightness from each sample pixel in the sample to be processed image. Sample brightness information refers to a brightness value used to characterize the brightness of the sample pixel. For example, the sample brightness information can be the value of the Y channel of each sample pixel in the sample to be processed image, that is, the Y value.

[0164] Step S303: determining a sample adjustment table applicable to the sample image to be processed from the image array according to the sample brightness information of each sample pixel.

[0165] In the embodiment of the present application, the sample adjustment table includes a sample brightness adjustment value of a sample pixel point of each sample brightness information.

[0166] The loop iteration process includes N rounds of loop processing, where N is an integer greater than 1; wherein, before the first round of loop processing, the image array during the first round of loop processing of the loop iteration process is determined based on the sample brightness information of each sample pixel point in the sample image to be processed. For example, the three-dimensional array of the image during the first round of loop processing of the loop iteration process can be determined; wherein, after the first type of loop processing, the sample brightness adjustment value corresponding to each sample pixel point is the same as the sample brightness information of the corresponding sample pixel point.

[0167] During the Nth round of loop processing in the loop iteration process, an updated image array obtained after the N-1th round of loop processing is obtained. For example, an updated three-dimensional image array obtained after the N-1th round of loop processing can be obtained; the updated three-dimensional image array obtained after the N-1th round of loop processing is determined as the three-dimensional image array during the Nth round of loop processing.

[0168] In the embodiment of the present application, the image brightness mean corresponding to the sample brightness information of all sample pixels in the sample to be processed image can be calculated, and the image brightness mean is the mean corresponding to the brightness values ​​of all sample pixels. Then, see which brightness interval the image brightness mean falls in, and by querying the current three-dimensional image array, determine the sample two-dimensional enhancement table corresponding to the brightness interval in the three-dimensional image array as the sample adjustment table applicable to the sample to be processed image in this round of training.

[0169] In some embodiments, after each round of training, the three-dimensional array of images obtained in the training round can be stored in the cache unit. In this way, in the next round of training, the three-dimensional image data obtained in the previous round of training can be obtained from the cache unit, and based on the three-dimensional image data obtained in the previous round of training, a sample adjustment table applicable to the sample to-be-processed images in the current round of training can be queried.

[0170] Step S304: determine a plurality of sample adjacent pixel points corresponding to each sample pixel point in the sample image to be processed.

[0171] Here, the sample adjacent pixel point refers to a plurality of sample pixel points that are located around the sample pixel point and are adjacent to the sample pixel point. Usually in an image, the absolute value of the difference between the brightness value of any sample pixel point and the brightness value of the sample adjacent pixel point is less than or equal to the difference threshold. At this time, the sample pixel point is considered to be a normal pixel point. If the absolute value of the difference between the brightness value of any sample pixel point and the brightness value of the sample adjacent pixel point is greater than the difference threshold, it indicates that the sample pixel point may be a noise point. For the noise point in the sample to be processed image, the brightness of the sample pixel point may not be adjusted, or a suitable pixel value may be determined for the sample pixel point as an index value, so that the sample two-dimensional enhancement table can be queried to further implement the brightness adjustment of the sample pixel point.

[0172] Step S305 , for each sample pixel, based on the sample brightness information of the sample pixel and the sample brightness information of a plurality of sample adjacent pixels, query the target sample brightness adjustment value corresponding to the sample pixel from the sample adjustment table.

[0173] Step S306, performing loss calculation on the target sample brightness adjustment value and the label brightness adjustment value corresponding to the sample pixel point to obtain a loss result.

[0174] Here, the L1 loss can be used to calculate the difference between the target sample brightness adjustment value and the label brightness adjustment value corresponding to the sample pixel point to obtain the loss result.

[0175] Step S307: based on the loss result, update the data in the image array to obtain an updated image array.

[0176] In the embodiment of the present application, the data in the three-dimensional array of images can be updated based on the loss result to obtain an updated three-dimensional array of images. In the implementation process, the learning of the preset three-dimensional array of images can be realized by looping and iterating the above steps S301 to S307 until the loss function converges at the loss result, and the learning of the preset three-dimensional array of images is stopped. At this time, the preset three-dimensional array of images is the preset image array finally determined.

[0177] In some embodiments, a certain amount of sample data can be provided, and in each loop iteration, a part of the sample data can be used for a loop iteration training. When all the sample data are used to learn the preset image three-dimensional array, the learning of the preset image three-dimensional array is stopped. At this time, the preset image three-dimensional array is the preset image three-dimensional array that is finally determined.

[0178] In other embodiments, a threshold of the number of loop iterations may be pre-set. When the number of loop iterations reaches the threshold value for learning the preset image three-dimensional array using sample data, the learning of the preset image three-dimensional array is stopped. The preset image three-dimensional array at this time is the preset image three-dimensional array that is finally determined.

[0179] In some other embodiments, a total duration threshold of the iterative training may be preset. In the process of learning the preset image three-dimensional array using sample data, the training duration of the iterative training is recorded. When the training duration reaches the total duration threshold, the learning of the preset image three-dimensional array is stopped. The preset image three-dimensional array at this time is the preset image three-dimensional array that is finally determined.

[0180] The following is an explanation of an exemplary application of an embodiment of the present application in a practical application scenario.

[0181] The embodiment of the present application proposes a low-light image and over-exposed image enhancement algorithm based on deep learning and 3D table lookup. The technical innovation of the embodiment of the present application is to use deep learning methods to automatically learn the enhancement strength from the data during the training phase and save it as three-dimensional table data (i.e., a three-dimensional array of preset images). In the use phase, there is no need to load the deep learning model, and the preset image three-dimensional array is directly queried through the table lookup method to obtain the enhanced value. The advantage of the embodiment of the present application is that it has a strong learning ability and avoids the high computing cost of deep learning when performing network forward computing, which is very suitable for real-time reasoning of end-side devices.

[0182] The solution of the embodiments of the present application can be used in a variety of scenarios, such as online video calls, live broadcast software, short video platforms, unmanned driving, medical imaging, etc. Fig. 9 is an application effect diagram of the method of the embodiment of the present application on the video conferencing software, wherein Fig. 9 Figure (a) is the original image. Fig. 9 Figure (b) is an enhanced image. It can be seen that through the method of the embodiment of the present application, users can see the other party's facial expressions, gestures and other details more clearly in the video conference, thereby improving the effect and quality of communication.

[0183] The image processing method of the embodiment of the present application is described below.

[0184] Fig.10 is a schematic diagram of the implementation flow of the image processing method provided in the embodiment of the present application, such as Fig.10As shown, firstly, a video frame in RGB format (i.e., the image to be processed) is received as input, and the video frame in RGB format is converted to YUV space, where the Y channel represents brightness information and the UV channel represents chrominance information. In order to reduce the processing complexity, the subsequent operations of the embodiment of the present application only process the Y channel, and the UV channel remains unchanged. Then, the mean value of all Y in the video frame is calculated to obtain the image brightness mean value Y mean ; According to the image brightness mean Y mean The two-dimensional array (i.e., the two-dimensional adjustment table, for example, can be represented as a two-dimensional enhancement table T) is obtained by indexing from the preset three-dimensional array of images. 2D , of course, in other embodiments it can also be a two-dimensional attenuation table). For example, the three-dimensional array T 3D The dimension of Y is N×255×255, where the last two dimensions are fixed in size, that is, the last two dimensions are fixed to 255, and the dimension N can be selected according to the memory usage requirements. mean The theoretical value range of Y is 0 to 255. mean Divided into N segments, for example, when Y mean If it falls in the first segment, use 1 as the index to get the two-dimensional enhanced table T 2D .

[0185] It should be noted that the three-dimensional array here has N two-dimensional enhancement tables, and N two-dimensional enhancement tables mean that there are N enhancement modes. The embodiment of the present application uses the average brightness of the picture as the basis for selecting the enhancement table. For example, when the average brightness of the picture is very low, the two-dimensional enhancement table obtained by the index should correspond to a relatively high enhancement intensity; when the average brightness of the picture is relatively high, it should correspond to a relatively low enhancement intensity. In this way, the embodiment of the present application can use different enhancement operations for different scenes, and has a certain degree of scene adaptability.

[0186] The image brightness mean value range can be segmented as follows, although Y mean The theoretical value range of is 0 to 255, but when the mean is less than 60, the whole picture will be very dark to the human eye; when the mean is higher than 150, the human eye will feel very bright. Therefore, those less than 60 can be considered to be in the same segment, and those greater than 150 can also be considered to be in the same segment. The values ​​between 60 and 150 can be divided into uniform segments, so that all tables can be used more efficiently. Note that 60 and 150 here are empirical values ​​and can be changed in practice.

[0187] According to the two-dimensional enhancement table T 2D The index is the enhanced E. The operation flow of this step is as follows: First, the dimension of the two-dimensional enhancement table is 255×255, the dimension of Y is H×W, and the brightness of the pixel at the coordinate (x, y) can be expressed as Y x,y We use Y x,yand the mean brightness M of its neighboring pixels x,y As an index, M x,y The calculation method of is shown in the following formula (4):

[0188]

[0189] Among them, Ceil represents round down.

[0190] You can see Y x,y and M x,y The value range of Y is from 0 to 255, so x,y and M x,y Can be used as an index to obtain the two-dimensional enhanced table T 2D The enhanced pixel value E is obtained by taking the value x,y By performing this operation on all pixels in Y, we can get the enhanced Y, which is named E.

[0191] It should be noted that, in fact, a one-dimensional enhancement table can also achieve the purpose of enhancement. Assume that when Y x,y When Y is 3, the value at index 3 can be found from the one-dimensional enhancement table as 9. Using a two-dimensional enhancement table, not only the own value can be used, but also nearby values ​​can be used as indexes. x,y When M is 3, x,y is 100. At this time, the value at (x, y) is too different from the nearby mean, indicating that this may be a noise point. The one-dimensional enhancement table cannot notice this information and still thinks that the index is 3, so the enhanced value is 9. The two-dimensional enhancement table can combine these two information and the difference between it and the surrounding values ​​to determine a more reasonable enhancement value. Therefore, the two-dimensional enhancement table has a certain denoising ability.

[0192] Fig.11 is a flow chart of the inference phase of the three-dimensional array provided in the embodiment of the present application. The following describes how to obtain the three-dimensional array (also called the three-dimensional enhanced table) through training. Fig.11 As shown in the figure, the training process of this algorithm is similar to the above Fig.10 The algorithm flow in is mostly the same, except that the loss calculation and parameter update process are added. The embodiment of the present application provides a batch of expert-adjusted image pairs, and manually adjusts various low-light images and overexposed images to obtain images with comfortable brightness. First, perform the RGB to YUV conversion operation to obtain the brightness channels Y and G of the low-light image (or overexposed image) and the expert-adjusted image, and then process Y according to the above process to obtain the enhanced E, and then use the L1 loss to calculate the difference between G and E. The L1 loss is shown in the following formula (5):

[0193] L1=|GE| (5).

[0194] The goal of learning in the training phase is to minimize the L1 loss so that the E obtained by the algorithm is close to the manually adjusted image G. The only learnable parameter in the entire processing flow is the three-dimensional array T 3D In order to reduce the learning difficulty, T 3D You can use the original value as initialization, so that E and Y will be the same at the beginning. As the training iterates, T 3D With continuous updates, E is getting closer and closer to G.

[0195] The image processing method provided in the embodiment of the present application can adjust the brightness of low-light images and overexposed images to obtain images that are more comfortable to the human eye, such as Fig.12 As shown, it is an enhancement effect diagram of the method of the embodiment of the present application in a low-light scene. It can be seen that compared with the original image ( Fig.12 (a) in the figure), the enhanced image ( Fig.12 (b) in the figure) can more easily distinguish the edge of the packaging bag and the texture of the carpet. At the same time, the embodiment of the present application uses a 3D table lookup method, which can index different two-dimensional enhancement tables according to the average brightness of the picture, has a certain scene adaptability, and can perform relatively strong enhancement in low illumination, such as Fig.12 As shown in the figure. For overexposed images with sufficient brightness, only slight enhancement or brightness reduction will be performed, so as not to cause overexposure of the image. Fig.13 As shown, Fig.13 Figure (a) is the result obtained by using software with fixed enhancement strength. Fig.13 Figure (b) shows the effect of the embodiment of the present application. Fig.13 Because the middle (a) image uses the same enhancement strength as in the low-light scene, it will cause overexposure and poor contrast when facing a scene with sufficient brightness. This problem does not exist in the embodiment of the present application.

[0196] In addition, the solution of the embodiment of the present application has a very low computational cost. Using mainstream configurations such as 720p and 1080p, the embodiment of the present application takes only a few milliseconds per frame on low-end and mid-end hardware, meeting the real-time processing requirements of the end side for the video.

[0197] It should be noted that, for the above schemes of the embodiments of the present application, the various specific implementation details mentioned in the embodiments of the present application, such as parameter settings, brightness mean segmentation, etc., can be freely changed, and changes to such content still do not exceed the protection scope of the embodiments of the present application. In addition to using L1 loss as a constraint, the embodiments of the present application can also use L2 loss, GAN loss, and perceptual loss, etc., which can all be used as loss functions, and changing the loss function does not exceed the protection scope of the embodiments of the present application. Although the embodiments of the present application describe the use of brightness mean as the index of the three-dimensional enhancement table, other parameters can of course be used, and changing the index method does not exceed the protection scope of the embodiments of the present application.

[0198] It can be understood that in the embodiments of the present application, the content involving user information, such as information such as images to be processed and three-dimensional arrays of preset images, if it involves data related to user information or corporate information, when the embodiments of the present application are applied to specific products or technologies, it is necessary to obtain user permission or consent, or to blur this information to eliminate the correspondence between this information and the user; and the relevant data collection and processing should be strictly in accordance with the requirements of relevant national laws and regulations when applied in examples, and the informed consent or separate consent of the subject of personal information should be obtained, and subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the subject of personal information.

[0199] The following is a description of an exemplary structure of the image processing device 354 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 4 As shown, the image processing device 354 includes: an information extraction module 3541, which is used to extract the brightness information of at least one pixel in the image to be processed; a determination module 3542, which is used to determine an adjustment table suitable for the image to be processed from a preset image array according to the brightness information of the at least one pixel; the adjustment table includes a brightness adjustment value of each pixel with brightness information; a query module 3543, which is used to query the target brightness adjustment value corresponding to the pixel from the adjustment table for each pixel based on the brightness information of the pixel; an image brightness processing module 3544, which is used to use the target brightness adjustment value of each pixel to perform image brightness processing on the image to be processed to obtain a processed image.

[0200] In some embodiments, the determination module is also used to: determine the image brightness mean of the image to be processed based on the brightness information of the at least one pixel point; query the adjustment table corresponding to the image brightness mean in the preset image array; the preset image array is a preset image three-dimensional array, and the adjustment table is a two-dimensional enhancement table corresponding to the image brightness mean.

[0201] In some embodiments, the preset image three-dimensional array includes multiple two-dimensional enhancement tables, each of which corresponds to a brightness range; the determination module is also used to: determine the brightness range where the image brightness mean is located as the target brightness range; and determine the two-dimensional enhancement table corresponding to the target brightness range as the queried adjustment table corresponding to the image brightness mean.

[0202] In some embodiments, the query module is also used to: determine the image coordinates of each of the pixel points in the image to be processed; based on the image coordinates, determine the four pixel points located around each of the pixel points and adjacent to the pixel point as the adjacent pixel points; for each of the pixel points, based on the brightness information of the pixel point and the brightness information of the multiple adjacent pixel points, query the target brightness adjustment value corresponding to the pixel point from the adjustment table.

[0203] In some embodiments, the query module is also used to: determine the brightness value of an adjacent pixel corresponding to the pixel based on the brightness information of the pixel and the brightness information of the multiple adjacent pixels; and query the target brightness adjustment value corresponding to the pixel from the adjustment table using the brightness information of the pixel and the brightness value of the adjacent pixel as indexes.

[0204] In some embodiments, the query module is also used to: determine the pixel brightness mean between the pixel and each of the adjacent pixel points based on the brightness information of the pixel point and the brightness information of the multiple adjacent pixel points to obtain multiple pixel brightness means; calculate the average of the multiple pixel brightness means; round the average to obtain an adjacent pixel brightness value in integer form.

[0205] In some embodiments, the device also includes: a difference determination module, used to determine the difference between the brightness information of the pixel point and the brightness value of the adjacent pixel point; an index value determination module, used to determine the index value according to the difference when the absolute value of the difference is greater than the difference threshold; the query module is also used to query the target brightness adjustment value corresponding to the pixel point from the adjustment table based on the index value.

[0206] In some embodiments, the device also includes: an image format determination module, used to determine the image format of the image to be processed; an image format conversion module, used to convert the image format of the image to be processed if the image format is a color channel format, so as to convert the color channel format into a brightness channel format; the information extraction module is also used to: extract the information of the brightness channel of the image to be processed having the brightness channel format, and obtain the brightness information of at least one pixel in the image to be processed.

[0207] In some embodiments, the image brightness processing module is also used to: replace the brightness information of the corresponding pixel in the brightness channel format with the target brightness adjustment value of each pixel to obtain updated image data in the brightness channel format; perform image format conversion on the updated image data in the brightness channel format to obtain updated image data in the color channel format; perform image rendering based on the updated image data in the color channel format to obtain the processed image.

[0208] In some embodiments, the device further includes: a processing module, which is used to determine the preset image array by looping and iterating the following steps: obtaining sample data, wherein the sample data includes a sample image to be processed and a label brightness adjustment value; extracting brightness information of the sample image to be processed to obtain sample brightness information of each sample pixel in the sample image to be processed; determining a sample adjustment table applicable to the sample image to be processed from an image array according to the sample brightness information of each sample pixel; the sample adjustment table includes a sample brightness adjustment value of the sample pixel for each sample brightness information; for each sample pixel, based on the sample brightness information of the sample pixel, querying a target sample brightness adjustment value corresponding to the sample pixel from the sample adjustment table; performing loss calculation on the target sample brightness adjustment value and the label brightness adjustment value corresponding to the sample pixel to obtain a loss result; based on the loss result, updating the data in the image array to obtain an updated image array; when the loss result of the loop iteration process meets a preset convergence condition, stopping the loop iteration process, and determining the updated image array obtained when the loop iteration process is stopped as the preset image array.

[0209] In some embodiments, the loop iteration process includes N rounds of loop processing, where N is an integer greater than 1; the processing module is also used to: determine the image array during the first round of loop processing of the loop iteration process based on the sample brightness information of each sample pixel point in the sample image to be processed; wherein, after the first type of loop processing, the sample brightness adjustment value corresponding to each of the sample pixel points is the same as the sample brightness information of the corresponding sample pixel point; when performing the Nth round of loop processing of the loop iteration process, obtain the updated image array obtained after the N-1th round of loop processing; and determine the updated image array obtained after the N-1th round of loop processing as the image array during the Nth round of loop processing.

[0210] It should be noted that the description of the device of the embodiment of the present application is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment, so it is not repeated. For technical details not disclosed in the embodiment of the device, please refer to the description of the method embodiment of the present application for understanding.

[0211] The embodiment of the present application provides a computer program product, which includes an executable instruction, which is a computer instruction; the executable instruction is stored in a computer-readable storage medium. When a processor of an electronic device reads the executable instruction from the computer-readable storage medium and the processor executes the executable instruction, the electronic device executes the method described in the embodiment of the present application.

[0212] The present application embodiment provides a storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the method provided by the present application embodiment, for example, Figure 5 The method shown.

[0213] In some embodiments, the storage medium can be a computer-readable storage medium, for example, a ferroelectric random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPR OM), a flash memory, a magnetic surface memory, an optical disk, or a compact disk read-only memory (CD-ROM), etc.; it can also be various devices including one or any combination of the above memories.

[0214] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0215] As an example, executable instructions may, but need not correspond to, a file in a file system, may be stored as part of a file storing other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). As an example, executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed at multiple locations and interconnected by a communication network.

[0216] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Extracting brightness information of at least one pixel in the image to be processed; According to the brightness information of the at least one pixel, an adjustment table applicable to the image to be processed is determined from a preset image array; the adjustment table includes a brightness adjustment value of each pixel of brightness information; For each of the pixel points, based on the brightness information of the pixel point, querying the target brightness adjustment value corresponding to the pixel point from the adjustment table; The target brightness adjustment value of each pixel point is used to perform image brightness processing on the image to be processed to obtain a processed image.

2. The method according to claim 1, characterized in that The step of determining an adjustment table applicable to the image to be processed from a preset image array according to the brightness information of the at least one pixel point comprises: Determining an image brightness mean value of the image to be processed according to the brightness information of the at least one pixel; An adjustment table corresponding to the image brightness mean is queried in the preset image array; the preset image array is a preset image three-dimensional array, and the adjustment table is a two-dimensional enhancement table corresponding to the image brightness mean.

3. The method according to claim 2, characterized in that The preset image three-dimensional array includes a plurality of two-dimensional enhancement tables, each of which corresponds to a brightness range; The step of searching the preset image array for an adjustment table corresponding to the image brightness mean value includes: Determine the brightness interval where the image brightness mean value is located as the target brightness interval; The two-dimensional enhancement table corresponding to the target brightness interval is determined as the queried adjustment table corresponding to the image brightness mean value.

4. The method according to claim 1, characterized in that: The step of searching, for each pixel, the target brightness adjustment value corresponding to the pixel from the adjustment table based on the brightness information of the pixel, comprises: Determine the image coordinates of each pixel point in the image to be processed; Based on the image coordinates, four pixel points located around each pixel point and adjacent to the pixel point are determined as the adjacent pixel points; For each of the pixel points, based on the brightness information of the pixel point and the brightness information of the plurality of adjacent pixel points, the target brightness adjustment value corresponding to the pixel point is searched from the adjustment table.

5. The method according to claim 4, characterized in that The querying, from the adjustment table, a target brightness adjustment value corresponding to the pixel point based on the brightness information of the pixel point and the brightness information of the plurality of adjacent pixel points comprises: Determine a brightness value of an adjacent pixel corresponding to the pixel based on the brightness information of the pixel and the brightness information of the plurality of adjacent pixels; The target brightness adjustment value corresponding to the pixel is queried from the adjustment table using the brightness information of the pixel and the brightness value of the adjacent pixel as indexes.

6. The method according to claim 5, characterized in that The determining, based on the brightness information of the pixel and the brightness information of the plurality of adjacent pixels, the brightness value of adjacent pixels corresponding to the pixel includes: Based on the brightness information of the pixel point and the brightness information of the plurality of adjacent pixel points, determine the average brightness value of the pixel point between the pixel point and each of the adjacent pixel points to obtain the average brightness values ​​of the plurality of pixel points; Calculate the average of the brightness values ​​of the multiple pixels; The average value is rounded to obtain brightness values ​​of adjacent pixels in integer form.

7. The method according to claim 5, characterized in that The method further comprises: Determine the difference between the brightness information of the pixel and the brightness value of the adjacent pixel; When the absolute value of the difference is greater than the difference threshold, determining an index value according to the difference; The target brightness adjustment value corresponding to the pixel point is searched from the adjustment table based on the index value.

8. The method according to any one of claims 1 to 7, characterized in that: Before extracting brightness information of at least one pixel in the image to be processed, the method further includes: Determining the image format of the image to be processed; If the image format is a color channel format, performing image format conversion on the image to be processed to convert the color channel format into a brightness channel format; The step of extracting brightness information of at least one pixel in the image to be processed includes: The information of the brightness channel of the image to be processed in the brightness channel format is extracted to obtain the brightness information of at least one pixel in the image to be processed.

9. The method according to claim 8, characterized in that The step of using the target brightness adjustment value of each pixel point to perform image brightness processing on the image to be processed to obtain a processed image includes: The target brightness adjustment value of each pixel point is used to replace the brightness information of the corresponding pixel point in the brightness channel format to obtain the image data in the updated brightness channel format; Performing image format conversion on the updated brightness channel format image data to obtain updated color channel format image data; Image rendering is performed based on the image data in the updated color channel format to obtain the processed image.

10. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: determining the preset image array by looping and iterating the following steps: Acquire sample data, wherein the sample data includes a sample image to be processed and a label brightness adjustment value; Extracting brightness information of the sample image to be processed to obtain sample brightness information of each sample pixel in the sample image to be processed; According to the sample brightness information of each sample pixel point, a sample adjustment table applicable to the sample to-be-processed image is determined from the image array; the sample adjustment table includes a sample brightness adjustment value of each sample pixel point of the sample brightness information; For each of the sample pixel points, based on the sample brightness information of the sample pixel point, querying the target sample brightness adjustment value corresponding to the sample pixel point from the sample adjustment table; Performing loss calculation on the target sample brightness adjustment value and the label brightness adjustment value corresponding to the sample pixel point to obtain a loss result; Based on the loss result, updating the data in the image array to obtain an updated image array; When the loss result of the loop iteration process meets the preset convergence condition, the loop iteration process is stopped, and the updated image array obtained when the loop iteration process is stopped is determined as the preset image array.

11. The method according to claim 10, characterized in that The loop iteration process includes N rounds of loop processing, where N is an integer greater than 1; the method further includes: Based on the sample brightness information of each sample pixel point in the sample to-be-processed image, determine the image array when performing the first round of loop processing of the loop iteration process; wherein after the first type of loop processing, the sample brightness adjustment value corresponding to each of the sample pixel points is the same as the sample brightness information of the corresponding sample pixel point; When performing the Nth round of loop processing of the loop iteration process, obtaining the updated image array obtained after the N-1th round of loop processing; The updated image array obtained after the N-1th round of loop processing is determined as the image array during the Nth round of loop processing.

12. An image processing device, characterized in that: The device comprises: An information extraction module, used to extract brightness information of at least one pixel in the image to be processed; A determination module, configured to determine an adjustment table applicable to the image to be processed from a preset image array according to the brightness information of the at least one pixel; the adjustment table includes a brightness adjustment value of each pixel of brightness information; A query module, configured to query, for each pixel point, a target brightness adjustment value corresponding to the pixel point from the adjustment table based on the brightness information of the pixel point; The image brightness processing module is used to perform image brightness processing on the image to be processed by adopting the target brightness adjustment value of each pixel point to obtain a processed image.

13. An electronic device, characterized in that: include: A memory for storing executable instructions; A processor, configured to implement the image processing method according to any one of claims 1 to 11 when executing the executable instructions stored in the memory.

14. A computer-readable storage medium, characterized in that: Executable instructions are stored, which are used to cause a processor to execute the executable instructions to implement the image processing method according to any one of claims 1 to 11.

Images