A method and apparatus for low-light image enhancement based on semantic combination

Abstract

This application provides a method, apparatus, computer-readable medium, and electronic device for low-light image enhancement based on semantic integration. The method includes: acquiring a low-light image to be enhanced; the low-light image includes images captured when the light intensity is below a set threshold; extracting local detail information and global structural information from the low-light image based on its spatial information; and generating an enhanced image by performing image convolution and transformation through an image signal processing pipeline based on the detail information and the structural information. This application's solution can efficiently and effectively enhance low-light images while preserving rich semantic information, significantly improving perceptual quality.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a method, apparatus, computer-readable medium, and electronic device for low-light image enhancement based on semantic combination. Background Technology

[0002] Traditional methods for low-light image enhancement mainly include histogram equalization (HE) and Retinex-based methods. Early on, HE-based image enhancement methods were widely used. Although these methods can achieve satisfactory visual results in some situations, their performance largely depends on manually selected priors, and the complex optimization process leads to long inference times, making them unsuitable for real-time tasks. Convolution-based image information extraction relies on the size of the receptive field provided by the convolution kernel and cannot fully capture global information; therefore, it suffers from relatively low-light image enhancement effectiveness. Summary of the Invention

[0003] Embodiments of this application provide a low-light image enhancement method, apparatus, computer-readable medium, and electronic device based on semantic combination, which can at least partially solve the problem of low low-light image enhancement effect.

[0004] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0005] According to one aspect of this application, a low-light image enhancement method based on semantic combination is provided, comprising: acquiring a low-light image to be enhanced; the low-light image includes an image taken when the light intensity is lower than a set threshold; extracting local detail information and global structural information of the low-light image based on spatial information of the low-light image; and generating an enhanced image by performing image convolution and transformation through an image signal processing pipeline based on the detail information and the structural information.

[0006] In this application, based on the aforementioned scheme, the step of extracting local detail information and global structural information of the low-light image based on the spatial information of the low-light image includes: constructing local structural branches and global structural branches through multiple convolutional feature enhancement blocks; constructing a feature extraction structure through the local structural branches and the global structural branches; and extracting local detail information and global structural information of the low-light image based on the spatial information of the low-light image through the feature extraction structure.

[0007] In this application, based on the aforementioned scheme, the convolutional feature enhancement block sequentially includes: a depthwise convolution of a preset size, a first pointwise convolution, a depthwise separable convolution, and a second pointwise convolution; wherein, the depthwise convolution of the preset size is used to encode positional information, and the second pointwise convolution is used to enhance local details of the image.

[0008] In this application, based on the aforementioned scheme, the local structural branch includes a first preset number of convolutional feature enhancement blocks, wherein the size of the separable convolutional kernel is 5, used to mine fine-grained features; the global structural branch includes a second preset number of convolutional feature enhancement blocks, wherein the size of the separable convolutional kernel increases sequentially, used to gradually capture large-scale features.

[0009] In this application, based on the aforementioned scheme, the step of extracting local detail information and global structural information of the low-light image based on the spatial information of the low-light image includes: encoding and reconstructing the low-light image to generate an original image corresponding to the low-light image; and decoding the original image to determine the local detail information and global structural information therein.

[0010] In this application, based on the aforementioned scheme, the step of generating an enhanced image by performing image convolution and transformation through an image signal processing pipeline based on the detailed information and the structural information includes: performing image convolution through an image signal processing pipeline based on the detailed information and the structural information, and applying an activation function after each convolutional layer to generate an enhanced image.

[0011] In this application, based on the aforementioned scheme, the method further includes: constructing a loss function based on smoothing loss, semantic loss, and perceptual loss.

[0012] According to one aspect of this application, a low-light image enhancement apparatus based on semantic combination is provided, comprising:

[0013] An acquisition unit is used to acquire a low-light image to be enhanced; the low-light image includes an image taken when the light intensity is below a set threshold.

[0014] The extraction unit is used to extract local detail information and global structural information of the low-light image based on the spatial information of the low-light image;

[0015] An enhancement unit is used to generate an enhanced image by performing image convolution and transformation through an image signal processing pipeline based on the detailed information and the structural information.

[0016] In this application, based on the aforementioned scheme, the step of extracting local detail information and global structural information of the low-light image based on the spatial information of the low-light image includes: constructing local structural branches and global structural branches through multiple convolutional feature enhancement blocks; constructing a feature extraction structure through the local structural branches and the global structural branches; and extracting local detail information and global structural information of the low-light image based on the spatial information of the low-light image through the feature extraction structure.

[0017] In this application, based on the aforementioned scheme, the convolutional feature enhancement block sequentially includes: a depthwise convolution of a preset size, a first pointwise convolution, a depthwise separable convolution, and a second pointwise convolution; wherein, the depthwise convolution of the preset size is used to encode positional information, and the second pointwise convolution is used to enhance local details of the image.

[0018] In this application, based on the aforementioned scheme, the local structural branch includes a first preset number of convolutional feature enhancement blocks, wherein the size of the separable convolutional kernel is 5, used to mine fine-grained features; the global structural branch includes a second preset number of convolutional feature enhancement blocks, wherein the size of the separable convolutional kernel increases sequentially, used to gradually capture large-scale features.

[0019] In this application, based on the aforementioned scheme, the step of extracting local detail information and global structural information of the low-light image based on the spatial information of the low-light image includes: encoding and reconstructing the low-light image to generate an original image corresponding to the low-light image; and decoding the original image to determine the local detail information and global structural information therein.

[0020] In this application, based on the aforementioned scheme, the step of generating an enhanced image by performing image convolution and transformation through an image signal processing pipeline based on the detailed information and the structural information includes: performing image convolution through an image signal processing pipeline based on the detailed information and the structural information, and applying an activation function after each convolutional layer to generate an enhanced image.

[0021] In this application, based on the aforementioned scheme, the method further includes: constructing a loss function based on smoothing loss, semantic loss, and perceptual loss.

[0022] According to one aspect of this application, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the low-light image enhancement method based on semantic combination as described in the above embodiments.

[0023] According to one aspect of this application, an electronic device is provided, comprising: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the semantically combined low-light image enhancement method as described in the above embodiments.

[0024] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the semantically combined low-light image enhancement method provided in the various alternative implementations described above.

[0025] In the technical solution of this application, a low-light image to be enhanced is acquired; the low-light image includes images taken when the light intensity is below a set threshold; based on the spatial information of the low-light image, local detail information and global structural information of the low-light image are extracted; based on the detail information and the structural information, image convolution and transformation are performed through an image signal processing pipeline to generate an enhanced image. The technical solution of this application can efficiently and effectively enhance low-light images while retaining rich semantic information, significantly improving perceptual quality.

[0026] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0027] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0028] Figure 1 The flowchart of a low-light image enhancement method based on semantic combination is illustrated in one embodiment of this application.

[0029] Figure 2 The diagram illustrates a convolutional feature enhancement block in one embodiment of this application.

[0030] Figure 3 The illustration shows a schematic diagram of a low-light image enhancement device based on semantic combination in one embodiment of this application.

[0031] Figure 4 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0032] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0033] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0034] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0035] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0036] The implementation details of the technical solution of this application are described below:

[0037] Figure 1 A flowchart of a low-light image enhancement method based on semantic combination according to an embodiment of this application is shown. (Refer to...) Figure 1 As shown, this low-light image enhancement method based on semantic combination includes at least steps S110 to S130, which are described in detail below:

[0038] In step S110, a low-light image to be enhanced is acquired; the low-light image includes an image taken when the light intensity is below a set threshold.

[0039] In one embodiment of this application, acquiring the low-light image to be enhanced is a crucial step in image processing tasks, particularly in fields such as computer vision, photography, and security monitoring. Low-light images typically refer to images captured under poor lighting conditions, such as light intensity below a certain set threshold. Such images often suffer from blurred details and distorted colors, making subsequent analysis or processing difficult. Using devices such as cameras or mobile phones to capture images in low-light environments simulates real-world low-light conditions.

[0040] In step S120, based on the spatial information of the low-light image, local detail information and global structural information of the low-light image are extracted.

[0041] In this method, a Transformer-like structure based on convolutional operations automatically extracts the enhancement compensation needed from low-light images. Compared to traditional convolutional neural networks, this design focuses more on global information, can perceive semantic information, and improves the model's robustness and generalization ability. Furthermore, it has lower computational cost compared to the classic Transformer model. Simultaneously, this method utilizes a pre-trained segmentation network, which preserves a large amount of semantic information without significantly increasing computational complexity and feeds this semantic information back into the entire training process.

[0042] In one embodiment of this application, an Enhancement Compensation Extraction (ECE) module is included. This module employs a Transformer-style two-branch design and depthwise separable convolutions to understand the impact of illumination at the pixel level in low-light images and extract information to guide subsequent enhancement. The local branch focuses on image details, while the global branch captures global structural information. Compared to convolutional networks, this approach emphasizes global semantics, enabling the model to capture image information more comprehensively.

[0043] In one embodiment of this application, based on the spatial information of the low-light image, local detail information and global structural information of the low-light image are extracted, including:

[0044] Local and global structural branches are constructed using multiple convolutional feature enhancement blocks;

[0045] The feature extraction structure is constructed using the local structural branches and the global structural branches;

[0046] Based on the spatial information of the low-light image, the feature extraction structure is used to extract local detail information and global structural information of the low-light image.

[0047] In this embodiment, a Transformer-like structure combined with a depthwise separable convolutional dual-branch lightweight enhancement compensation extraction (ECE) module is designed. ECE is highly adaptive, utilizing spatial information from low-light images to monitor subsequent image enhancement, and using local and global branches to focus on image detail and global structural information, respectively.

[0048] In one embodiment of this application, the convolutional feature enhancement block sequentially includes: a depthwise convolution of a preset size, a first pointwise convolution, a depthwise separable convolution, and a second pointwise convolution;

[0049] The preset-size depthwise convolution is used to encode positional information, and the second pointwise convolution is used to enhance local image details.

[0050] The local structure branch includes a first preset number of convolutional feature enhancement blocks, wherein the separable convolutional kernel size is 5, which is used to mine fine-grained features;

[0051] The global structure branch includes a second preset number of convolutional feature enhancement blocks, wherein the size of the separable convolutional kernel increases sequentially to gradually capture large-scale features.

[0052] like Figure 2 As shown, the enhancement compensation extraction aims to understand the pixel-level illumination effects of low-light images and record relevant information in the output. Unlike traditional methods, this module does not adopt the U-Net style of downsampling followed by upsampling, as this would destroy the spatial coherence of the image. Instead, this module chooses to process at the original resolution of the input to preserve detail information to the greatest extent. Furthermore, traditional convolutional networks mainly rely on local receptive fields to extract information, while Transformers can process images from a global perspective and capture global semantic information. Therefore, this module proposes a Transformer-style structure that uses a Conv-transformer-enhanceBlock (CTEB). The structure of the Conv-transformer-enhance Block is as follows: Figure 2 As shown.

[0053] Specifically, the input is expanded to increase the channel dimension through 3×3 convolutions. Building upon existing single-branch structures, this module adds a global structure branch, with each branch consisting of a different number of CTEBs. Given the enormous computational cost of the self-attention mechanism in traditional feature extraction structures like the Transformer when processing high-resolution images, such as… Figure 2As shown, in CTEB, 3×3 depthwise convolutions are first used to encode positional information, then pointwise convolutions, depthwise separable convolutions, and pointwise convolutions are used to replace multi-head self-attention to enhance local details, and finally the feature representation is enhanced by two 1×1 convolutions.

[0054] Specifically, in the local branch, three CETBs are stacked, with a depthwise separable convolution kernel size of 5, to mine fine-grained features; in the global structure branch, two CETBs are used, with convolution kernel sizes of 5 and 7 respectively. Large-scale features are captured by gradually increasing the convolution kernel size. Finally, the channel dimension is reduced by 3×3 convolution, and ReLU / Tanh activation functions are used to generate detail enhancement factors and global enhancement compensation.

[0055] In one embodiment of this application, based on the spatial information of the low-light image, local detail information and global structural information of the low-light image are extracted, including:

[0056] The low-light image is encoded and reconstructed to generate the original image corresponding to the low-light image;

[0057] The original image is decoded to determine local detail information and global structural information.

[0058] Specifically, inspired by image signal processing (ISP), this module constructs a progressive enhancement module for low-light conditions. i Low-light image I taken below i The resulting image is obtained after light enters the CMOS, converts the light signal into an electrical signal, and provides it to the ISP (Image Signal Processing) pipeline G(·) for further image processing. This is assumed to be under normal lighting conditions L. g Normal illumination image I was obtained g The process of low-light image enhancement is to use I i with I g Matching is performed, a mapping relationship is established between the two, and an encoder-decoder structure is applied to input the low-light image into I. i Edit as normal lighting image I g The process is as follows:

[0059] I g =G(F(I) i )) (1)

[0060] Where F is the reconstruction function, which can be considered as an encoder. After reconstruction, the original image data D = F(I) is obtained. i G(·) is the transformation function, which can be considered as a decoder, transforming D into a normally lit image I. g Here, we simplify the role of G(·), focusing primarily on the reconstruction process, which can be represented as:

[0061] F=I⊙M+A (2)

[0062] I⊙M=ω (3)

[0063] ω+A=F (4)

[0064] For the reconstruction process F based on the normal illumination image I, it consists of two separate processes: receiving the detail enhancement factor M and generating the illumination enhancement factor ω from the normal illumination image I (multiplication formula (3)), and then performing pixel-by-pixel addition formula (4) based on the global enhancement compensation A. The progressive enhancement module performs these two processes progressively to make pixel-level adjustments.

[0065] In step S130, based on the detailed information and the structural information, image convolution and transformation are performed through the image signal processing pipeline to generate an enhanced image.

[0066] To reduce the performance overhead of subsequent image enhancement, a Progressive Image Enhancement (PIE) module is introduced. This module uses the low-light image and enhancement compensation from the Enhancement Compensation Extraction (ECE) module as its input. The reconstructed image obtained through the Image Signal Processing (ISP) pipeline is progressively enhanced.

[0067] To preserve semantic information during the enhancement process, an Automatic Semantic Segmentation (ASS) module that does not require expensive segmentation annotations is proposed. ASS receives the enhanced image from the PIE and calculates the segmentation loss using VGG1 6. The resulting segmentation loss is combined with the L1smooth loss and the perceptual loss to form a total loss, which is fed back into the ECE during training for better feature extraction. The combination of these three modules can efficiently and effectively enhance low-light images while preserving rich semantic information, significantly improving perceptual quality, and mitigating common problems such as noise and blur.

[0068] In one embodiment of this application, based on the detailed information and the structural information, an enhanced image is generated by performing image convolution and transformation through an image signal processing pipeline, including:

[0069] Based on the detailed information and the structural information, image convolution is performed through the image signal processing pipeline, and an activation function is applied after each convolutional layer to generate an enhanced image.

[0070] The Automatic Semantic Segmentation (ASS) module aims to perform precise pixel-level segmentation of augmented images, thereby preserving semantic information during progressive image enhancement. This method freezes all layers of the segmentation network during training, using a VGG16 with pre-trained weights as the segmentation network.

[0071] The enhanced image from PIE is processed through a series of convolutional layers, each containing multiple kernels to extract different features from the image. The convolutional operations filter the input image, generating a series of feature maps. An activation function (ReLU) is applied after each convolutional layer to introduce non-linearity and enhance the model's expressive power. After the convolutional layers, max pooling reduces the size of the feature maps. After multiple convolutional and pooling layers, the final feature maps are converted into one-dimensional vectors and connected to a fully connected layer, associating high-level features with the output class. Finally, a classification layer is connected to map the model's output to a class probability distribution and computes the segmentation.

[0072] In one embodiment of this application, a loss function is further constructed based on smoothing loss, semantic loss, and perceptual loss.

[0073] The loss function design includes L1 Smooth Loss, SGE Loss (semantic loss), and perceptual loss. L1 Smooth Loss balances the differences between the generated and target images, making the output smoother, avoiding overfitting, and improving details and texture. SGE Loss preserves the semantic information of the image by calculating classification loss for semantic segmentation. Perceptual loss captures the semantic differences between images, encouraging the generation of images with greater semantic consistency and visual realism. These three loss functions comprehensively constrain both pixel-level and semantic levels, guiding the training process and improving the enhancement effect.

[0074] The overall loss function can be summarized as follows:

[0075] L total =λ1L 1smooth +λ2L sem +λ3L per (5)

[0076] Where λ1=1, λ2=0.05, λ3=0.03.

[0077] Specifically, L 1smooth This indicates that L1 Smooth Loss is a smoothed version of L1 loss, which reduces the impact of outliers on the loss.

[0078]

[0079] Where N is the number of samples, x i and y i ε represents the predicted and target values ​​for the i-th sample, and ε is a small positive number used for numerical stability.

[0080] Specifically, SGE Loss helps preserve the semantic information of an image during the enhancement process. Semantic loss does not require label segmentation and only needs a pre-initialized model.

[0081]

[0082] Where p is the estimated class probability of a pixel by the segmentation network, and W and H represent the width and length of the input. In this paper, the focus coefficients β and γ are selected as 1 and 2, respectively.

[0083] Perceptual loss is based on pre-trained visual perception models, such as the VGG network. Better perceptual quality can be achieved by minimizing the difference between the generated image and the target image in the perceptual feature space. per This can be calculated by measuring the Euclidean or cosine distance between the generated and target images in terms of perceptual features:

[0084]

[0085] Where N is the number of samples, M is the number of feature maps, and x i and y i Let be the predicted and target values ​​of the i-th sample, φ represent the feature extraction function of the pre-trained perceptual model (such as VGG), and ε be a small positive number for numerical stability. This formula measures the perceptual similarity between the predicted and target values ​​by calculating the difference between the feature maps of the perceptual model and averaging the sum of squares of the differences.

[0086] In this application's technical solution, a low-light image to be enhanced is acquired; the low-light image includes images taken when the light intensity is below a set threshold; based on the spatial information of the low-light image, local detail information and global structural information of the low-light image are extracted; based on the detail information and the structural information, image convolution and transformation are performed through an image signal processing pipeline to generate an enhanced image. This application's technical solution can efficiently and effectively enhance low-light images while preserving rich semantic information, significantly improving perceptual quality.

[0087] The following describes an apparatus embodiment of this application, which can be used to execute the semantic-based low-light image enhancement method in the above embodiments of this application. It is understood that the apparatus can be a computer program (including program code) running on a computer device, for example, the apparatus is application software; the apparatus can be used to execute the corresponding steps in the method provided in the embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the semantic-based low-light image enhancement method described above.

[0088] Figure 3A block diagram of a semantically combined low-light image enhancement apparatus according to an embodiment of this application is shown.

[0089] Reference Figure 3 As shown, a semantically combined low-light image enhancement apparatus according to an embodiment of this application includes:

[0090] The acquisition unit 310 is used to acquire a low-light image to be enhanced; the low-light image includes an image taken when the light intensity is lower than a set threshold.

[0091] Extraction unit 320 is used to extract local detail information and global structural information of the low-light image based on the spatial information of the low-light image;

[0092] The enhancement unit 330 is used to generate an enhanced image by performing image convolution and transformation through an image signal processing pipeline based on the detailed information and the structural information.

[0093] In this application, based on the aforementioned scheme, the step of extracting local detail information and global structural information of the low-light image based on the spatial information of the low-light image includes: constructing local structural branches and global structural branches through multiple convolutional feature enhancement blocks; constructing a feature extraction structure through the local structural branches and the global structural branches; and extracting local detail information and global structural information of the low-light image based on the spatial information of the low-light image through the feature extraction structure.

[0094] In this application, based on the aforementioned scheme, the convolutional feature enhancement block sequentially includes: a depthwise convolution of a preset size, a first pointwise convolution, a depthwise separable convolution, and a second pointwise convolution; wherein, the depthwise convolution of the preset size is used to encode positional information, and the second pointwise convolution is used to enhance local details of the image.

[0095] In this application, based on the aforementioned scheme, the local structural branch includes a first preset number of convolutional feature enhancement blocks, wherein the size of the separable convolutional kernel is 5, used to mine fine-grained features; the global structural branch includes a second preset number of convolutional feature enhancement blocks, wherein the size of the separable convolutional kernel increases sequentially, used to gradually capture large-scale features.

[0096] In this application, based on the aforementioned scheme, the step of extracting local detail information and global structural information of the low-light image based on the spatial information of the low-light image includes: encoding and reconstructing the low-light image to generate an original image corresponding to the low-light image; and decoding the original image to determine the local detail information and global structural information therein.

[0097] In this application, based on the aforementioned scheme, the step of generating an enhanced image by performing image convolution and transformation through an image signal processing pipeline based on the detailed information and the structural information includes: performing image convolution through an image signal processing pipeline based on the detailed information and the structural information, and applying an activation function after each convolutional layer to generate an enhanced image.

[0098] In this application, based on the aforementioned scheme, the method further includes: constructing a loss function based on smoothing loss, semantic loss, and perceptual loss.

[0099] In this application's technical solution, a low-light image to be enhanced is acquired; the low-light image includes images taken when the light intensity is below a set threshold; based on the spatial information of the low-light image, local detail information and global structural information of the low-light image are extracted; based on the detail information and the structural information, image convolution and transformation are performed through an image signal processing pipeline to generate an enhanced image. This application's technical solution can efficiently and effectively enhance low-light images while preserving rich semantic information, significantly improving perceptual quality.

[0100] Figure 4 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown.

[0101] It should be noted that the computer system 400 of the electronic device shown in the figure is only an example and should not impose any limitations on the function and scope of use of the embodiments of this application.

[0102] The computer system 400 includes a Central Processing Unit (CPU) 401, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 402 or loaded from Storage Unit 408 into Random Access Memory (RAM) 403, such as executing the methods described in the above embodiments. The RAM 403 also stores various programs and data required for system operation. The CPU 401, ROM 402, and RAM 403 are interconnected via a bus 404. An Input / Output (I / O) interface 405 is also connected to the bus 404.

[0103] The following components are connected to I / O interface 405: an input section 406 including a keyboard, mouse, etc.; an output section 407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to I / O interface 405 as needed. A removable medium 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 410 as needed so that computer programs read from it can be installed into storage section 408 as needed.

[0104] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411. When the computer program is executed by central processing unit (CPU) 401, it performs various functions defined in the system of this application.

[0105] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0106] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0107] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0108] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various alternative implementations described above.

[0109] In another aspect, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

[0110] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0111] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this application.

[0112] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0113] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A low-light image enhancement method based on semantic combination, characterized in that, include: Acquire the low-light image to be enhanced; the low-light image includes images taken when the light intensity is below a set threshold; Based on the spatial information of the low-light image, local detail information and global structural information of the low-light image are extracted; based on the detail information and the structural information, image convolution and transformation are performed through an image signal processing pipeline to generate an enhanced image; Based on the spatial information of the low-light image, local detail information and global structural information of the low-light image are extracted, including: constructing local structural branches and global structural branches through multiple convolutional feature enhancement blocks; constructing a feature extraction structure through the local structural branches and the global structural branches; and extracting local detail information and global structural information of the low-light image through the feature extraction structure based on the spatial information of the low-light image. The convolutional feature enhancement block sequentially includes: a depthwise convolution of a preset size, a first pointwise convolution, a depthwise separable convolution, and a second pointwise convolution; wherein, the depthwise convolution of the preset size is used to encode positional information, and the second pointwise convolution is used to enhance local details of the image; The local structure branch includes a first preset number of convolutional feature enhancement blocks, wherein the separable convolutional kernel size is 5, used to mine fine-grained features; the global structure branch includes a second preset number of convolutional feature enhancement blocks, wherein the separable convolutional kernel size increases sequentially, used to gradually capture large-scale features. Based on the spatial information of the low-light image, extracting local detail information and global structural information of the low-light image includes: encoding and reconstructing the low-light image to generate an original image corresponding to the low-light image; and decoding the original image to determine the local detail information and global structural information therein. Based on the detailed information and the structural information, an enhanced image is generated by performing image convolution and transformation through an image signal processing pipeline, including: performing image convolution through an image signal processing pipeline based on the detailed information and the structural information, and applying an activation function after each convolutional layer to generate an enhanced image.

2. The method according to claim 1, characterized in that, The method also includes constructing a loss function based on smoothing loss, semantic loss, and perceptual loss.

3. A low-light image enhancement apparatus for implementing the semantic-based low-light image enhancement method as described in any one of claims 1 to 2, characterized in that, include: An acquisition unit is used to acquire a low-light image to be enhanced; the low-light image includes an image taken when the light intensity is below a set threshold. The extraction unit is used to extract local detail information and global structural information of the low-light image based on the spatial information of the low-light image; An enhancement unit is used to generate an enhanced image by performing image convolution and transformation through an image signal processing pipeline based on the detailed information and the structural information.

4. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the low-light image enhancement method based on semantic combination as described in any one of claims 1 to 2.

5. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the semantically combined low-light image enhancement method as described in any one of claims 1 to 2.

Images