Image exposure adjustment method and device, equipment, medium and product

Abstract

The invention provides an image exposure adjustment method and device, equipment, a medium and a product, and the method comprises the steps: obtaining exposure features corresponding to an original image, and the exposure features comprise an image quality feature, an image information feature and a global feature of the original image; predicting an adjustment exposure value corresponding to the next frame of image of the original image based on the exposure characteristics; and adjusting the exposure value of the next frame of image based on the adjusted exposure value. According to the invention, the image exposure effect can be improved, and the imaging quality is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image exposure adjustment method, device, equipment, medium and product. Background Art

[0002] Exposure parameters are very important for image quality. Reasonable setting of exposure parameters can ensure that photos can achieve the best results in various shooting environments.

[0003] At present, the image quality score is mainly calculated based on image indicator values, such as grayscale value and contrast, and whether the image exposure is in an appropriate range is judged based on the score. If the image brightness is not in an appropriate range, the corresponding exposure value is adjusted based on prior information so that the image exposure effect meets expectations.

[0004] However, this exposure adjustment method is relatively simple, and the exposure adjustment effect is not good, resulting in poor image quality. Summary of the invention

[0005] Based on the above technical status, the present application provides an image exposure adjustment method, device, equipment, medium and product, which can improve the image exposure adjustment effect and thus improve the image imaging quality.

[0006] In order to achieve the above technical objectives, this application specifically proposes the following technical solutions:

[0007] According to a first aspect of an embodiment of the present application, a method for adjusting image exposure is provided, comprising: obtaining exposure characteristics corresponding to an original image, the exposure characteristics comprising image quality characteristics, image information characteristics and global characteristics of the original image, the image quality characteristics being characteristics characterizing the image quality of the original image, the image information characteristics being characteristics characterizing the image content and structure of the original image, and the global characteristics characterizing the overall characteristics of the original image; predicting an adjusted exposure value corresponding to a next frame image of the original image based on the exposure characteristics; and adjusting the exposure value of the next frame image based on the adjusted exposure value.

[0008] In some implementations, acquiring exposure features corresponding to an original image and predicting an adjusted exposure value corresponding to a next frame image of the original image based on the exposure features includes: inputting the original image into an exposure adjustment model, acquiring exposure features corresponding to the original image by the exposure adjustment model, and predicting an adjusted exposure value corresponding to the next frame image of the original image based on the exposure features.

[0009] In some implementations, the exposure adjustment model includes an image quality feature extraction module, an image information feature extraction module, a global feature extraction module and a fusion module; wherein, the exposure adjustment model obtains the exposure features corresponding to the original image, including: extracting the image quality features of the original image through the image quality feature extraction module; extracting the image information features of the original image through the image information feature extraction module; extracting the global features of the original image through the global feature extraction module; and fusing the image quality features, image information features and global features through the fusion module to obtain the exposure features corresponding to the original image.

[0010] In some implementations, the image quality feature extraction module includes an image size conversion unit, a first image quality feature extraction unit, and a second image quality feature extraction unit; wherein, extracting the image quality feature of the original image through the image quality feature extraction module includes: processing the original image into a first resolution image and a second resolution image through the image size conversion unit, the first resolution being greater than the second resolution; extracting a first image quality feature corresponding to the first resolution image through the first image quality feature extraction unit; and extracting a second image quality feature corresponding to the second resolution image through the second image quality feature extraction unit.

[0011] In some implementations, the first image quality feature extraction unit includes a spatial pyramid pooling convolution unit, a dilated convolution unit, a first fusion unit, a first attention mechanism unit, and a second fusion unit; wherein, extracting the first image quality feature corresponding to the first resolution image through the first image quality feature extraction unit includes: extracting the initial spatial structure feature of the first resolution image through the spatial pyramid pooling convolution unit, and extracting the deep spatial structure feature of the initial spatial structure feature through the dilated convolution unit; fusing the initial spatial structure feature and the deep spatial structure feature through the first fusion unit to obtain a fused spatial structure feature; determining the weights corresponding to each channel and the weights corresponding to each spatial position in the fused spatial structure feature through the first attention mechanism unit, and adjusting the importance of the features corresponding to each channel based on the weights corresponding to each channel in the fused spatial structure feature, and adjusting the importance of the features corresponding to each spatial position in the spatial structure feature based on the weights corresponding to each spatial position, to obtain an adjusted spatial structure feature; fusing the adjusted spatial structure feature and the deep spatial structure feature through the second fusion unit to obtain the first image quality feature corresponding to the first resolution image.

[0012] In some implementations, the second image quality feature extraction unit includes an image basic indicator feature extraction unit and a KAN network; wherein, extracting the second image quality feature corresponding to the second resolution image through the second image quality feature extraction unit includes: extracting each image basic quality indicator feature of the second resolution image through the image basic indicator feature extraction unit; determining the weight corresponding to each of the image basic quality indicator features through the KAN network, and fusing the each image basic quality indicator feature based on the weight of the each image basic quality indicator feature to obtain the second image quality feature corresponding to the second resolution image.

[0013] In some implementations, the image information feature extraction module includes an initial feature extraction unit, a division unit, a first feature extraction unit, a second feature extraction unit and a third fusion unit; wherein, extracting the image information features of the original image through the image information feature extraction module includes: performing feature extraction on the original image through the initial feature extraction unit to obtain initial image information features; performing image block division on the initial image information features through the division unit to obtain initial image block information features, wherein the initial image block information features include initial image information features corresponding to each image block in the original image; performing feature extraction on the initial image block information features through the first feature extraction unit and the second feature extraction unit respectively to obtain first image block information features and second image block information features; and fusing the first image block information features and the second image block information features through the third fusion unit to obtain image information features of the original image.

[0014] In some implementations, the global feature extraction module includes a first global feature extraction unit, a second global feature extraction unit, a second attention mechanism unit and a fourth fusion unit; wherein, extracting the global features of the original image through the global feature extraction module includes: extracting the first initial global features of the original image through the first global feature extraction unit; extracting the second initial global features of the original image through the second global feature extraction unit; performing feature extraction on the fused global features along different spatial directions through the second attention mechanism unit to obtain intermediate local features, wherein the fused global features are features obtained by fusing the first initial global features and the second initial global features; and fusing the intermediate global features and the second initial global features through the fourth fusion unit to obtain the global features of the original image.

[0015] In some implementations, the exposure feature also includes an exposure attribute parameter of the photosensitive device, and the exposure adjustment model also includes an adapter, a KAN network, a fifth fusion unit and a prediction unit; wherein the exposure adjustment model predicts an adjusted exposure value corresponding to the next frame image of the original image based on the exposure feature, including: determining the weights corresponding to the image quality information feature, the global feature and the exposure attribute parameter in the exposure feature through the KAN network, and fusing the image quality information feature, the global feature and the exposure attribute parameter based on the weights to obtain an adjusted exposure feature, wherein the image quality information feature is a fusion feature of the image quality feature and the image information feature; determining the environmental features of the current scene through the adapter based on the adjusted exposure feature and the initial image information feature; fusing the adjusted exposure feature and the environmental features through the fifth fusion unit to obtain a comprehensive exposure feature; and predicting the adjusted exposure value corresponding to the next frame image of the original image based on the comprehensive exposure feature through the prediction unit.

[0016] In some implementations, the exposure adjustment model is trained through the following steps: obtaining exposure feature samples corresponding to original image samples through the exposure adjustment model to be trained, and predicting the adjusted exposure value corresponding to the next frame image sample of the original image sample based on the exposure feature samples to obtain the predicted adjusted exposure value; the exposure feature samples include image quality feature samples, image information feature samples and global feature samples of the original image samples; determining the predicted exposure adjustment loss based on the difference between the predicted adjusted exposure value and the labeled adjusted exposure value corresponding to the original image sample, and converging the exposure adjustment model to be trained based on the predicted exposure adjustment loss to obtain the exposure adjustment model.

[0017] In some implementations, the annotated adjusted exposure value corresponding to the original image sample is obtained by the following steps: obtaining multiple exposure scores corresponding to each of multiple training image samples, each of the multiple training image samples corresponding to a different exposure value, and the multiple training image samples and the original image sample are images taken of the same object; for each training image sample in the multiple training image samples, based on the average value of the multiple exposure scores corresponding to the training image sample, determining the comprehensive exposure score corresponding to the training image sample; selecting the exposure value of the training image sample with the highest comprehensive exposure score among the multiple training image samples, and determining it as the annotated adjusted exposure value corresponding to the original image.

[0018] According to a second aspect of an embodiment of the present application, an image exposure adjustment device is provided, comprising: an acquisition unit, used to acquire exposure characteristics corresponding to an original image, the exposure characteristics including image quality characteristics, image information characteristics and global characteristics of the original image; a prediction unit, used to predict an adjusted exposure value corresponding to a next frame image of the original image based on the exposure characteristics; and an adjustment unit, used to adjust the exposure value of the next frame image based on the adjusted exposure value.

[0019] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory and a processor; the memory is connected to the processor and is used to store programs; the processor is used to implement the image exposure adjustment method as described in the first aspect by running the program in the memory.

[0020] According to a fourth aspect of an embodiment of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the image exposure adjustment method as described in the first aspect is implemented.

[0021] According to a fifth aspect of an embodiment of the present application, a computer program product is provided, comprising computer program instructions, which, when executed by a processor, cause the processor to execute: the image exposure adjustment method as described in the first aspect.

[0022] The embodiments of the present application provide an image exposure adjustment method, device, equipment, medium and product, which obtains the exposure characteristics corresponding to the original image, including the image quality characteristics, image information characteristics and global characteristics of the original image, predicts the adjusted exposure value corresponding to the next frame of the original image based on the exposure characteristics, and adjusts the exposure value of the next frame of the image based on the adjusted exposure value. Since the image quality characteristics are the characteristics that characterize the image quality of the original image, the image information characteristics are the characteristics that characterize the image content and structure of the original image, and the global characteristics characterize the overall characteristics of the original image, it is possible to extract more comprehensive image exposure characteristics with richer feature information, thereby improving the image exposure value adjustment effect and improving the image imaging quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0024] Figure 1 A flowchart of an image exposure adjustment method provided in an embodiment of the present application;

[0025] Figure 2 A schematic diagram of the framework of the exposure adjustment model provided in an embodiment of the present application;

[0026] Figure 3 A schematic diagram of a process of obtaining exposure features corresponding to an original image by an exposure adjustment model provided in an embodiment of the present application;

[0027] Figure 4 A schematic diagram of the framework of the image quality feature extraction module provided in the embodiment of the present application;

[0028] Figure 5 A schematic diagram of a framework of a first image quality feature extraction unit provided in an embodiment of the present application;

[0029] Figure 6 A schematic diagram of a framework of a second image quality feature extraction unit provided in an embodiment of the present application;

[0030] Figure 7 A schematic diagram of the framework of the image information feature extraction module provided in the embodiment of the present application;

[0031] Figure 8 A schematic diagram of a framework of a first feature extraction unit provided in an embodiment of the present application;

[0032] Fig. 9 A schematic diagram of the framework of the global feature extraction module provided in the embodiment of the present application;

[0033] Fig.10 A schematic diagram of the framework of the fusion module provided in the embodiment of the present application;

[0034] Fig.11 A schematic diagram of the framework of the adapter provided in the embodiment of the present application;

[0035] Fig.12 A schematic diagram of a flow chart of a training process of an exposure adjustment model provided in an embodiment of the present application;

[0036] Fig.13 is a structural schematic diagram of an image exposure adjustment device provided in an embodiment of the present application;

[0037] Fig.14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The technical solution proposed in the embodiment of the present application is applicable to any application scenario that requires image exposure adjustment, such as medical imaging, industrial inspection, photography post-processing, news photography, mobile photography, video production, scientific photography, etc. The technical solution of the embodiment of the present application can improve the image exposure adjustment effect in the above application scenarios, thereby improving the image quality.

[0039] The technical solution provided in the embodiments of the present application can be exemplarily applied to hardware devices such as processors, electronic devices, servers (including cloud servers), or packaged into software programs to be run. When the hardware device executes the processing of the technical solution of the embodiments of the present application, or the above-mentioned software program is run, the automatic splitting of the target task and the automatic calling of the application program interface required for the task can be achieved to complete the purpose of the target task. The embodiments of the present application only exemplarily introduce the specific processing of the technical solution of the present application, and do not limit the specific implementation form of the technical solution of the present application. Any technical implementation form that can execute the processing of the technical solution of the present application can be adopted by the embodiments of the present application.

[0040] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0041] Before introducing this application solution, the relevant technologies are first introduced:

[0042] Image exposure adjustment is very important in many application scenarios. Correct exposure can not only achieve the best effect in terms of brightness, contrast and color, improve image quality, but also improve the effect of subsequent image-based analysis and processing. Whether it is medical imaging, industrial inspection, photography post-processing, news photography, mobile photography, video production or scientific photography, proper exposure adjustment can significantly improve image quality.

[0043] Taking medical imaging and industrial inspection as examples, image quality is very important for medical imaging or industrial inspection. In medical imaging scenarios, correct exposure can preserve details in highlight and shadow areas and avoid loss of details caused by overexposure or underexposure. This helps to observe lesions and tissue details, identify tiny lesions, vascular structures and tissue abnormalities, and can also reduce misdiagnosis and missed diagnosis caused by image quality problems.

[0044] In industrial inspection scenarios, correct exposure can ensure that the brightness, contrast and color of the image are optimal. Appropriate exposure can preserve the details of the highlight and shadow areas, avoid the loss of details caused by overexposure or underexposure, and enable operators to clearly observe the wear, cracks, blockages and other faults inside the equipment. During equipment inspection and maintenance, real-time exposure adjustment can ensure that the image is always in the best condition, helping operators to perform precise inspections and repairs.

[0045] Whether it is medical imaging or industrial inspection, endoscopes are used to observe areas that are difficult to observe directly. Endoscopes are divided into medical endoscopes and industrial endoscopes. Medical endoscopes are a type of medical equipment used to observe and operate body cavities in medical diagnosis and treatment, providing access and lighting during endoscopic diagnosis and treatment.

[0046] Industrial endoscopes are tools used for non-destructive testing of industrial equipment and components, allowing users to view and evaluate areas that are difficult to observe directly, such as the internal conditions of precision castings, engines, and other confined spaces. Its working principle is to send a small camera and light source into the target area through a rigid tubular structure pipeline to take real-time photos and videos, and users can view the internal images in real time on an external screen.

[0047] When using industrial endoscopes and medical endoscopes, the lens needs to be inserted deep into the device or the internal organs of the human body, so the image quality is extremely sensitive to lighting conditions. In such a low-light environment, the exposure effect directly affects the user experience. If the exposure parameters are set unreasonably, such as the exposure time is too long, the object will reflect light to a large extent, making it difficult for the user to observe the actual original appearance of the observed object through the endoscope; if the exposure time is too short, the image will be too dark and it will be difficult to obtain image details.

[0048] At present, the adjustment of image exposure effect mainly depends on the image index value, such as gray value, contrast, etc., and the image brightness is determined by calculating the corresponding score of the image. Subsequently, the exposure value is adjusted according to the prior information so that the image exposure effect meets the observer's expectations. This method needs to rely on prior information, but in different application scenarios, the applicability of prior information is limited, resulting in insufficient versatility and adaptability, making this adjustment method only effective for some workpieces to be inspected and having poor robustness. In addition, exposure adjustment based on traditional evaluation methods makes it difficult to achieve accurate effects in the exposure adjustment process.

[0049] At the same time, some exposure adjustment algorithms use object recognition as a dimming reference object and split the dimming process into two stages: first detecting the main object and then adjusting the exposure intensity. This two-stage approach makes it difficult to adjust the exposure in real time, affecting the efficiency and real-time performance of exposure adjustment.

[0050] In addition, some exposure effects are more dependent on hardware, and the mapping relationship between exposure values ​​and exposure effects of different hardware does not match. Therefore, when an algorithm is adapted to different hardware, there will be a large effect deviation, affecting the accuracy and consistency of exposure adjustment.

[0051] In view of this, the embodiments of the present application are dedicated to providing an image exposure adjustment method, device, equipment and storage medium, which can improve the image exposure adjustment effect. Detailed descriptions are given one by one in the following embodiments.

[0052] Exemplary Methods

[0053] Figure 1 Flow chart of an image exposure adjustment method provided in an embodiment of the present application. Figure 1 As shown, the image exposure adjustment method provided in this embodiment includes steps S101-S103:

[0054] S101. Acquire exposure features corresponding to an original image, where the exposure features include image quality features, image information features, and global features of the original image.

[0055] Among them, the original image is the image currently captured by the image acquisition device. Image quality features refer to features used to evaluate the overall quality of the image, such as grayscale value, contrast, color saturation, sharpness, and noise level. Image information features refer to features used to describe the content and structure of the image, such as edge information, texture information, target information, and regional information. Global features refer to features used to describe the overall characteristics of the image, such as average brightness, brightness dynamic range, color balance, spatial distribution, and overall contrast.

[0056] In some embodiments, step S101 may be implemented by an exposure adjustment model. Specifically, step S101 includes: inputting the original image into the exposure adjustment model, and obtaining the exposure features corresponding to the original image by the exposure adjustment model.

[0057] The specific implementation process of obtaining the exposure features corresponding to the original image through the exposure adjustment model is described in detail below with reference to the accompanying drawings:

[0058] Figure 2 This is a schematic diagram of the framework of the exposure adjustment model provided in the embodiment of the present application. Figure 2As shown, the exposure adjustment model includes an image quality feature extraction module 21, an image information feature extraction module 22, a global feature extraction module 23 and a fusion module 24. The image quality feature extraction module 21 is used to extract image quality features from the original image; the image information feature extraction module 22 is used to extract image information features from the original image; the global feature extraction module 23 is used to extract image global features from the original image; and the fusion module 24 is used to fuse the image quality features, image information features and image global features to obtain the exposure features corresponding to the original image.

[0059] based on Figure 2 The framework schematic diagram of the exposure adjustment model shown, the specific implementation of step S101 is as follows:

[0060] Figure 3 The following is a flow chart of obtaining the exposure features corresponding to the original image by the exposure adjustment model provided in the embodiment of the present application. Figure 3 As shown, obtaining the exposure features corresponding to the original image by the exposure adjustment model includes the following steps S301 to S304:

[0061] S301 , extracting image quality features of an original image through the image quality feature extraction module 21 .

[0062] Figure 4 Schematic diagram of the framework of the image quality feature extraction module provided in the embodiment of the present application. Figure 4 As shown, the image quality feature extraction module 21 includes an image size conversion unit 211, a first image quality feature extraction unit 212, and a second image quality feature extraction unit 213. The image size conversion unit 211 is used to perform size conversion on the original image to obtain a first resolution image and a second resolution image; the first image quality feature extraction unit 212 is used to perform feature extraction on the first resolution image to obtain a first image quality feature; and the second image quality feature extraction unit 213 is used to perform feature extraction on the second resolution image to obtain a second image quality feature.

[0063] based on Figure 4 The framework schematic diagram of the image quality feature extraction module shown in FIG. 1 , the specific implementation of step S301 includes the following steps a1 to a3:

[0064] Step a1: Processing the original image into a first resolution image and a second resolution image through the image size conversion unit 211, wherein the first resolution is greater than the second resolution.

[0065] The image size change unit may use a resize function to resize the original image to obtain a first resolution image and a second resolution image. The first resolution is greater than the second resolution. The first resolution image may be understood as a high-resolution image, and the second resolution image may be understood as a low-resolution image. The first resolution may be any resolution value in [640, 1280], such as 960*960. The second resolution may be any resolution value in [320, 640], such as 448*448.

[0066] Step a2: extracting the first image quality feature Ffc corresponding to the first resolution image by the first image quality feature extraction unit 212.

[0067] Figure 5 This is a schematic diagram of the framework of the first image quality feature extraction unit provided in the embodiment of the present application. Figure 5 As shown, the first image quality feature extraction unit 212 includes a spatial pyramid pooling convolution unit (Spatial PyramidPooling Network, SPPnet) 2121, a dialeted convolution unit (dialeted conv) 2122, a first fusion unit 2123, a first attention mechanism unit (cbam) 2124 and a second fusion unit 2125. The spatial pyramid pooling convolution unit 2121 is used to extract the initial spatial structure feature Fspp of the first resolution image; the hole convolution unit 2122 is used to extract the deep spatial structure feature Fck from the initial spatial structure feature Fspp; the first fusion unit 2123 is used to fuse the initial spatial structure feature Fspp and the deep spatial structure feature Fck to obtain the fused spatial structure feature Fsdc; the first attention mechanism unit 2124 is used to determine the weights corresponding to each channel and the weights corresponding to each spatial position in the fused spatial structure feature Fsdc, and adjust the importance of the features corresponding to each channel based on the weights corresponding to each channel in the fused spatial structure feature Fsdc, and adjust the importance of the features corresponding to each spatial position in the fused spatial structure feature Fsdc based on the weights corresponding to each spatial position, to obtain the adjusted spatial structure feature Fsdcc; the second fusion unit 2125 is used to fuse the adjusted spatial structure feature Fsdcc and the deep spatial structure feature Fck to obtain the first image quality feature Ffc corresponding to the first resolution image.

[0068] based on Figure 5 The framework schematic diagram of the first image quality feature extraction unit is shown, and the specific implementation of step a2 includes the following steps a21 to a25:

[0069] Step a21, extracting initial spatial structural features of the first resolution image through the spatial pyramid pooling convolution unit 2121.

[0070] Among them, the spatial pyramid pooling convolution unit uses the SPPnet network, which can process feature maps of any size and output a feature vector of fixed length. The SPPnet unit introduces a spatial pyramid structure to divide the first resolution image into multiple image blocks of different scales, and performs a pooling operation on each image block, and splices the results of these pooling operations to obtain a feature vector of fixed length, that is, the initial spatial structure feature.

[0071] Step a22, extracting the deep spatial structure features of the initial spatial structure features through the hole convolution unit 2122.

[0072] The dilated convolution unit 2122 uses dilated convolution, that is, inserting a certain number of zeros (i.e., "holes") between the convolution kernel elements, so as to expand the receptive field without reducing the image resolution, extract more context information while maintaining high resolution, and obtain deep spatial structure features with richer feature information. Compared with the initial spatial structure features, the deep spatial structure features contain richer feature information.

[0073] Step a23: The initial spatial structure feature and the deep spatial structure feature are fused through the first fusion unit 2123 to obtain a fused spatial structure feature.

[0074] Specifically, the first fusion unit 2123 may fuse the initial spatial structure feature and the deep spatial structure feature using an ADD fusion method to obtain a fused spatial structure feature.

[0075] Step a24, determine the weights corresponding to each channel and the weights corresponding to each spatial position in the fused spatial structure feature through the first attention mechanism unit 2124, and adjust the importance of the features corresponding to each channel based on the weights corresponding to each channel in the fused spatial structure feature, and adjust the importance of the features corresponding to each spatial position in the fused spatial structure feature based on the weights corresponding to each spatial position, to obtain the adjusted spatial structure feature.

[0076] Among them, the fused spatial structural features include various channels. The fused spatial structural features include spatial features and structural features; spatial features refer to the feature information of the original image in the spatial dimension, which is mainly used to describe the spatial relationship between objects in the image, such as point distribution, line distribution, surface distribution and body distribution, which can help understand the position, arrangement and structure of objects in the image. Structural features refer to the feature information of the original image in the structural dimension, such as color, texture and shape in the image.

[0077] The first attention mechanism unit 2124 adopts a CBAM (Convolutional Block Attention Module) structure. CBAM is an attention module integrated in a convolutional neural network, which can enhance the feature expression ability of the model, more accurately strengthen important features and suppress unimportant features.

[0078] Among them, CBAM includes CAM (Channel Attention Module) and SAM (Spatial Attention Module). CAM can be used to determine the weights corresponding to each channel in the spatial structure feature, and based on the weights corresponding to each channel in the spatial structure feature, the features corresponding to each channel in the spatial structure feature are weighted and summed to obtain the first adjusted spatial structure feature, thereby adjusting the importance of each channel.

[0079] Afterwards, the weight corresponding to each spatial position in the spatial structure feature is determined by SAM, and based on the weight corresponding to each spatial position in the spatial structure feature, the features corresponding to each spatial position in the first adjusted spatial structure feature are weighted summed to obtain the second adjusted spatial structure feature, thereby adjusting the importance of each spatial position.

[0080] Step a25: The second fusion unit 2125 fuses the adjusted spatial structure feature and the deep spatial structure feature to obtain a first image quality feature corresponding to the first resolution image.

[0081] Specifically, the second fusion unit 2124 may fuse the adjusted spatial structure feature and the deep spatial structure feature using an ADD fusion method to obtain a first image quality feature corresponding to the first resolution image.

[0082] In the above embodiment, step a21 to step a25 are introductions to the specific implementation of step a2. After step a2, step a3 may also be included.

[0083] Step a3: extracting the second image quality feature corresponding to the second resolution image by the second image quality feature extraction unit 213.

[0084] Figure 6 Schematic diagram of the framework of the second image quality feature extraction unit provided in the embodiment of the present application. Figure 6 As shown, the second image quality feature extraction unit 213 includes an image basic index feature extraction unit 2131 and a KAN network 2132.

[0085] Among them, the image basic indicator feature extraction unit 2131 is used to extract each image basic quality indicator feature of the second resolution image; the KAN network 2132 is used to determine the weight corresponding to each image basic quality indicator feature, and based on the weight of each image basic quality indicator feature, fuse the basic quality indicator features of each image to obtain the second image quality feature Fcks corresponding to the second resolution image.

[0086] based on Figure 6 The schematic diagram of the framework of the second image quality feature extraction unit is shown. Step a3 specifically includes: extracting each basic image quality indicator feature of the second resolution image by the basic image indicator feature extraction unit 2131; determining the weight corresponding to each basic image quality indicator feature by the KAN network 2132, and fusing each basic image quality indicator feature based on the weight of each basic image quality indicator feature to obtain the second image quality feature corresponding to the second resolution image.

[0087] like Figure 6 As shown, each image basic indicator feature extraction unit 2131 may include an average gray value extraction subunit, a local exposure value extraction subunit, a color richness value extraction subunit and a contrast extraction subunit, which are respectively used to extract the average gray value, the local exposure value, the color richness value and the contrast to obtain the basic quality indicator features of each image.

[0088] In some embodiments, a merging unit, an MLP (multi-layer perceptron) and a ReLU layer may be provided between each image basic index feature extraction unit 2131 and the KAN network 2132. The merging unit may be a concate unit, which is used to fuse the basic quality index features of each image. The MLP (multi-layer perceptron) and the ReLU layer are used to process the fusion result into an output feature of a specific dimension size.

[0089] The KAN network 2132 may be connected to a Swish activation function to activate the fusion result after the basic quality indicator features of each image are fused, so as to obtain the second image quality feature corresponding to the second resolution image.

[0090] Continue reading Figure 3 After step S301, the exposure feature corresponding to the original image is obtained by the exposure adjustment model, and step S302 may also be included.

[0091] Step S302 : extracting image information features of the original image through the image information feature extraction module 22 .

[0092] Figure 7 Schematic diagram of the framework of the image information feature extraction module provided in the embodiment of the present application. Figure 7 As shown, the image information feature extraction module 22 includes an initial feature extraction unit 221 , a division unit 222 , a first feature extraction unit 223 , a second feature extraction unit 224 and a third fusion unit 225 .

[0093] based on Figure 7 The schematic diagram of the framework of the image information feature extraction module shown, step S302 specifically includes steps b1 to b5:

[0094] Step b1: extract features from the original image through the initial feature extraction unit 221 to obtain the initial image information feature Fr50.

[0095] In some embodiments, step b1 can perform feature extraction on the second resolution image by the initial feature extraction unit 221 to obtain the initial image information feature Fr50.

[0096] Among them, the initial feature extraction unit 221 can be a Resnet50 network, which is used to extract the initial image information feature Fr50 from the second resolution image.

[0097] Step b2: The initial image information feature Fr50 is divided into image blocks by the dividing unit 222 to obtain the initial image sequence information feature Fps.

[0098] The initial image sequence information feature Fps includes initial image information features corresponding to each image block in the second resolution image.

[0099] The initial image information features are divided into image blocks by a dividing unit (patch split) 222, so that the initial image information features corresponding to each image block in the second resolution image can be obtained.

[0100] Step b3: The first feature extraction unit 223 and the second feature extraction unit 224 respectively extract features of the initial image sequence information feature Fps to obtain a first image sequence information feature Fvmb and a second image sequence information feature Fvb.

[0101] Figure 8 Schematic diagram of the framework of the first feature extraction unit provided in the embodiment of the present application. Figure 8As shown, the first feature extraction unit includes a plurality of vision basic models (Vision Mamba, VIM) connected in series. Taking 4 Vision Mamba as an example, they are respectively recorded as the first Vision Mamba, the second Vision Mamba, the third Vision Mamba, the ADD unit and the fourth Vision Mamba; wherein, the first Vision Mamba, the second Vision Mamba and the third Vision Mamba successively extract the features of the initial image sequence information, and the ADD unit fuses the features Fvm1 extracted by the first Vision Mamba and the features Fvm3 extracted by the third Vision Mamba, and the fused features Fvma are input to the fourth Vision Mamba to continue the feature extraction, thereby obtaining the first image sequence information features Fvmb.

[0102] The second feature extraction unit can be a VIT Block (Vision Transformer Block). The second resolution image is regarded as a serialized input and the self-attention mechanism is used to process the pixel relationship in the image. The initial image sequence information feature is regarded as a series of fixed-size patches, and each patch is converted into a vector representation to form an input sequence. These vectors capture the local features of the image. These vector sequences are then input into a standard Transformer encoder, which contains a self-attention mechanism and a feedforward neural network layer. The self-attention mechanism allows the model to focus on different parts of the image and capture the dependencies between them regardless of their position in the image. In order to retain the position information of pixels or regions in the image, ViT also introduces position encoding. Position encoding provides a unique identifier for each position, allowing the model to understand the positional relationship of elements in the sequence. By treating the image as sequence data, ViT is able to capture long-range dependencies in the image using the self-attention mechanism, thereby obtaining the second image sequence information feature.

[0103] In some embodiments, MLP and ReLU may be connected after the multiple Vision Mamba Blocks and VIT Blocks connected in series to process the features output by the multiple Vision Mamba Blocks and VIT Blocks connected in series to obtain the first image sequence information features and the second image sequence information features.

[0104] Step b4: The first image sequence information feature Fvmb and the second image sequence information feature Fvb are fused by the third fusion unit 225 to obtain the image information feature Fvm of the original image.

[0105] The third fusion unit 225 may be an Attention network. Then step b4 includes: determining the weights corresponding to the first image sequence information feature and the second image sequence information feature through the Attention network, and fusing the first image sequence information feature and the second image sequence information feature based on the weights corresponding to the first image sequence information feature and the second image sequence information feature to obtain the image information feature Fvm of the original image. This process can be expressed as the following formula:

[0106] F VMA =weightA*F VM +weightB*F CKS ;

[0107] In the formula, F VMA represents the image information feature; weightA represents the weight corresponding to the first image sequence information feature; F VM represents the information feature of the first image sequence; weightB represents the weight corresponding to the information feature of the second image sequence; F CKS Represents the second image sequence information feature.

[0108] This embodiment combines the traditional evaluation method with the model learning evaluation method to make the exposure effect evaluation method more robust.

[0109] Continue reading Figure 3 After step S302, the exposure feature corresponding to the original image is obtained by the exposure adjustment model, and step S303 may also be included.

[0110] S303 , extracting global features of the original image through the global feature extraction module 23 .

[0111] Fig. 9 Schematic diagram of the framework of the global feature extraction module provided in the embodiment of the present application. Fig. 9 As shown, the global feature extraction module 23 includes a first global feature extraction unit 231, a second global feature extraction unit 232, a second attention mechanism unit 233 and a fourth fusion unit 234; the first global feature extraction unit 231 is used to extract the first initial global feature of the original image; the second global feature extraction unit 232 is used to extract the second initial global feature of the original image; the second attention mechanism unit 233 is used to determine the weights corresponding to the first initial global feature and the second initial global feature; the fourth fusion unit 234 is used to fuse the first initial global feature and the second initial global feature based on the weights corresponding to the first initial global feature and the second initial global feature, so as to obtain the global feature of the original image.

[0112] based on Fig. 9The schematic diagram of the framework of the global feature extraction module is shown. Step S303 includes the following steps c1 to c4:

[0113] Step c1: extracting the first initial global feature of the original image by the first global feature extraction unit 231.

[0114] In some embodiments, step c1 can extract the first initial global feature from the first resolution image corresponding to the original image by the first global feature extraction unit 231. The first global feature extraction unit can be a ConvMLP network. The ConvMLP network is a network structure that combines a convolutional neural network (CNN) and a multi-layer perceptron (MLP). ConvMLP introduces an MLP layer between convolutional layers, extracts features using convolution operations, and enhances the expressiveness of the features through the MLP layer. Specifically, the features of the first resolution image are extracted through the convolutional layer, and the features extracted by the convolutional layer are processed and fused through the MLP layer connected after the convolutional layer, so that the global features can be better captured, while reducing the number of parameters and the amount of calculation, which can improve the calculation efficiency.

[0115] Step c2: extracting the second initial global feature of the original image by the second global feature extraction unit 232.

[0116] Among them, the second global feature extraction unit 232 can be a RepVGG network, and the structure of RepVGG is composed of multiple building blocks, each building block includes a convolution layer and a normalization (BatchNorm) layer, and a ReLU activation function. The RepVGG network is a lightweight convolutional neural network architecture for image classification. Through structural parameter reconstruction technology (Structural Re-parameterization), the network adopts different structures in the training and reasoning stages, thereby maintaining high accuracy while improving the reasoning speed.

[0117] Step c3: The second attention mechanism unit 233 performs feature extraction on the fused global features along different spatial directions to obtain intermediate local features.

[0118] The fused global feature is a feature obtained by fusing the first initial global feature and the second initial global feature.

[0119] Specifically, the second attention mechanism unit 23 can be a Coordinate Attention network. The Coordinate Attention network is an attention mechanism that embeds position information into channel attention, aiming to improve the performance of lightweight network models. This mechanism decomposes channel attention into two one-dimensional feature encoding processes, aggregating features along two spatial directions respectively, thereby capturing long-range dependencies in one spatial direction while maintaining accurate position information in another spatial direction. The generated feature maps are respectively encoded as a pair of direction-aware and position-sensitive attention maps, which can be complementarily applied to the input feature map to enhance the feature representation of the object of interest.

[0120] Step c4: The fourth fusion unit 234 fuses the intermediate global feature and the second initial global feature to obtain the global feature of the original image.

[0121] Specifically, the fourth fusion unit 234 may fuse the intermediate initial global feature and the second initial global feature in a concatenated fusion manner to obtain the global feature of the original image.

[0122] In this embodiment, the first resolution image and the second resolution image are used simultaneously. Since the second resolution image is smaller in size and has a lower computational complexity, features can be extracted quickly and the exposure of the image can be quickly evaluated. In addition, the second resolution image occupies less memory and can be processed efficiently with limited hardware resources. The first resolution image can capture more detailed information in the image and can extract higher quality features. By combining the features of the first resolution image and the second resolution image, feature information of different scales can be extracted. The two complement each other and can more comprehensively understand the image content and improve the performance of the model in different scenarios.

[0123] Continue reading Figure 3 After step S303, the exposure feature corresponding to the original image is obtained by the exposure adjustment model, and step S304 may also be included.

[0124] S304 , fusing the image quality features, the image information features and the global features through the fusion module 24 to obtain the exposure features corresponding to the original image.

[0125] As mentioned above, the image quality feature includes a first image quality feature and a second image quality feature.

[0126] Fig.10 This is a schematic diagram of the framework of the fusion module provided in the embodiment of the present application. Fig.10As shown, the fusion module 24 includes a third attention mechanism unit 241, a multiplication (MUL) unit 242 and a sixth fusion unit 243; wherein the third attention mechanism unit 241 is used to determine the weights corresponding to the second image quality feature Fcks and the image information feature Fvm, and based on the weights corresponding to the second image quality feature Fcks and the image information feature Fvm, fuse the second image quality feature Fcks and the image information feature Fvm to obtain the initial image quality information feature Fvma; the multiplication unit 242 is used to multiply the first image quality feature Ffc and the initial image quality information feature Fvma to obtain the image quality information feature Fvmc; the sixth fusion unit 243 is used to fuse the image quality information feature Fvmc and the global feature Fccks to obtain the exposure feature Fcvc corresponding to the original image. The sixth fusion unit 243 can adopt a concate fusion method.

[0127] In some embodiments, the exposure feature also includes exposure attribute parameters of the photosensitive device. The exposure attribute parameters of the photosensitive device include exposure parameter values ​​and illumination mapping values. The photosensitive device provides an exposure environment for the image acquisition device to acquire the original image. By adjusting the exposure parameter value of the photosensitive device, the effect of adjusting the image exposure parameter value can be achieved.

[0128] The sixth fusion unit 243 is further used to fuse the image quality information features, the global features and the exposure attribute parameters of the photosensitive device to obtain a fused exposure feature corresponding to the original image.

[0129] Here, the exposure attribute parameters of the photosensitive device provide the model with the exposure sensitivity of the current photosensitive device, so that the model can determine the exposure value that needs to be adjusted for the next frame image for the current photosensitive device based on the exposure sensitivity of the current photosensitive device and the exposure characteristics of the original image.

[0130] S102: predicting an adjusted exposure value corresponding to a next frame of the original image based on the exposure feature.

[0131] In some embodiments, step S102 can be implemented by an exposure adjustment model. Specifically, step S102 includes: the exposure adjustment model predicts the adjusted exposure value corresponding to the next frame of the original image based on the exposure characteristics. Here, the adjusted exposure value corresponding to the next frame of the image refers to the exposure parameter value that needs to be adjusted when the photosensitive device shoots the next frame of the image. By adjusting the exposure parameter value that needs to be adjusted when the photosensitive device shoots the next frame of the image, the next frame of the image can be adjusted to a suitable exposure parameter value to improve the image quality.

[0132] After adjusting the exposure parameter value of the next frame image, the next frame image can be used as the original image, and steps S101 to S103 are re-executed and iterated in sequence, so as to achieve the effect of adjusting the exposure of each frame image.

[0133] Continue reading Figure 2 , the exposure adjustment model also includes an adapter 25, a KAN network 26, a fifth fusion unit 27 and a prediction unit 28; wherein the adapter 25 is used to determine the exposure environment characteristics under the current exposure environment conditions based on the exposure characteristics and the initial image information characteristics; the KAN network 26 is used to determine the weights corresponding to the image quality information characteristics, the global characteristics and the exposure attribute parameters, and fuse the image quality information characteristics, the global characteristics and the exposure attribute parameters based on the weights to obtain the adjusted exposure characteristics; the fifth fusion unit 27 is used to fuse the exposure environment characteristics and the adjusted exposure characteristics to obtain the comprehensive exposure characteristics; the prediction unit 28 is used to predict the adjusted exposure value corresponding to the next frame image of the original image based on the comprehensive exposure characteristics. Wherein, the prediction unit 28 may include MLP and Sigmoid activation functions.

[0134] based on Figure 2 The schematic diagram of the framework of the exposure adjustment model is shown. Step S102 includes: determining the weights corresponding to the image quality information features, global features and exposure attribute parameters in the exposure features through the KAN network 26, and fusing the image quality information features, global features and exposure attribute parameters based on the weights to obtain an adjusted exposure feature, where the image quality information feature is a fusion feature of the image quality feature and the image information feature; determining the environmental features of the current scene based on the adjusted exposure features and the initial image information features through the adapter 25; fusing the adjusted exposure features and the environmental features through the fifth fusion unit 27 to obtain a comprehensive exposure feature; and predicting the adjusted exposure value corresponding to the next frame of the original image based on the comprehensive exposure features through the prediction unit 28.

[0135] Fig.11 This is a schematic diagram of the framework of the adapter provided in the embodiment of the present application. Fig.11 As shown, the adapter 25 includes a seventh fusion unit 251, a MUL unit 252 and a KAN network 253; wherein the seventh fusion unit 251 is used to fuse the comprehensive exposure feature Fcvc and the initial image information feature Fr50 to obtain the initial fused image feature; the MUL unit 252 is used to multiply the initial fused image feature and the initial image information feature to obtain the fused image feature; the KAN network 253 is used to determine the weights corresponding to the initial fused image feature and the initial image information feature, and based on the weights corresponding to the initial fused image feature and the initial image information feature, fuse the initial fused image feature and the initial image information feature to obtain the exposure environment feature.

[0136] An MLP unit and a Sigmoid activation function are also provided between the seventh fusion unit 251 and the MUL unit 252 to perform dimensionality reduction processing on the initial fused image features.

[0137] In this embodiment, the exposure feature characterizes the current exposure information recognized by the model for the original image. The initial image information feature characterizes the image information carried by the second resolution image under the current exposure conditions (ie, the exposure parameter value and light mapping value corresponding to the current photosensitive device).

[0138] In this embodiment, the adapter includes adapters for different scenes, which can improve the adaptability of the model to exposure in different scenes. The model can load adapters for different scenes during the inference process to improve the robustness of the model's exposure adaptation in different scenes. Among them, different scenes refer to different natural lighting environments. Different scenes can include dark light scenes, strong light scenes, normal light scenes, and the like. For different scenes, this embodiment can select an adapter trained in the corresponding scene according to the current scene, thereby improving the adjustment effect of exposure in different scenes.

[0139] In some embodiments, the model can also fuse the exposure attribute parameters of the photosensitive device and the feature Fcvc, and directly predict the adjusted exposure value that needs to be adjusted for the next frame of the image based on the fused features. In other words, the weights corresponding to the image quality information features, global features, and exposure attribute parameters in the exposure features are determined by the KAN network, and the image quality information features, global features, and exposure attribute parameters are fused based on the weights to obtain the adjusted exposure features. Afterwards, the prediction unit predicts the adjusted exposure value corresponding to the next frame of the original image based on the adjusted exposure features.

[0140] Continue reading Figure 1 The exposure adjustment method of this embodiment may further include step S103 after step S102.

[0141] S103: Adjust the exposure value of the next frame image based on the adjusted exposure value.

[0142] The adjusted exposure value output by the exposure adjustment model may be the desired exposure value to be adjusted for the next frame of image, or the exposure value adjustment amount to be adjusted for the next frame of image. The adjusted exposure value may be the exposure duration to be adjusted for the next frame of image.

[0143] The above embodiment introduces the reasoning and application process of the exposure adjustment model. Before the exposure adjustment model is used for reasoning and application, it needs to be trained. The following will introduce the training process of the exposure adjustment model:

[0144] Fig.12The following is a flow chart of the training process of the exposure adjustment model provided in the embodiment of the present application. Fig.12 As shown, the training process of the exposure adjustment model includes the following steps:

[0145] S1101. Obtain exposure feature samples corresponding to original image samples through the exposure adjustment model to be trained.

[0146] The exposure feature samples include image quality feature samples, image information feature samples and global feature samples of the original image samples.

[0147] In this embodiment, the network structure of the exposure adjustment model to be trained may refer to the network structure of the exposure adjustment model in the aforementioned embodiment.

[0148] The exposure adjustment model to be trained includes an image quality feature extraction module, an image information feature extraction module, a global feature extraction module and a fusion module. Then step S1101 includes: extracting image quality feature samples of original image samples through the image quality feature extraction module; extracting image information feature samples of original image samples through the image information feature extraction module; extracting global feature samples of original image samples through the global feature extraction module; fusing image quality feature samples, image information feature samples and global feature samples through the fusion module to obtain exposure feature samples corresponding to the original image samples.

[0149] The image quality feature extraction module includes an image size conversion unit, a first image quality feature extraction unit and a second image quality feature extraction unit; wherein, extracting image quality feature samples of original image samples through the image quality feature extraction module includes: processing the original image samples into first resolution image samples and second resolution image samples through the image size conversion unit, the first resolution being greater than the second resolution; extracting first image quality feature samples corresponding to the first resolution image samples through the first image quality feature extraction unit; and extracting second image quality feature samples corresponding to the second resolution image samples through the second image quality feature extraction unit.

[0150] The first image quality feature extraction unit includes an SPPnet unit, a dilated convolution unit, a first fusion unit, a first attention mechanism unit and a second fusion unit; wherein, extracting the first image quality feature sample corresponding to the first resolution image sample through the first image quality feature extraction unit includes: extracting the initial spatial structure feature sample of the first resolution image sample through the SPPnet unit, and extracting the deep spatial structure feature sample of the initial spatial structure feature sample through the dilated convolution unit; fusing the initial spatial structure feature sample and the deep spatial structure feature sample through the first fusion unit to obtain a fused spatial structure feature sample; determining the weights corresponding to each channel and the weights corresponding to each spatial position in the fused spatial structure feature sample through the first attention mechanism unit, and adjusting the importance of the features corresponding to each channel based on the weights corresponding to each channel in the fused spatial structure feature sample, and adjusting the importance of the features corresponding to each spatial position in the spatial structure feature sample based on the weights corresponding to each spatial position, to obtain an adjusted spatial structure feature sample; fusing the adjusted spatial structure feature sample and the deep spatial structure feature sample through the second fusion unit to obtain a first image quality feature sample corresponding to the first resolution image sample.

[0151] In some embodiments, the second image quality feature extraction unit includes each image basic indicator feature extraction unit and a KAN network; wherein, extracting the second image quality feature sample corresponding to the second resolution image sample by the second image quality feature extraction unit includes: extracting each image basic quality indicator feature sample of the second resolution image sample by each image basic indicator feature extraction unit; determining the weight corresponding to each image basic quality indicator feature sample by the KAN network, and fusing each image basic quality indicator feature sample based on the weight of each image basic quality indicator feature sample to obtain the second image quality feature sample corresponding to the second resolution image sample.

[0152] In some embodiments, the image information feature extraction module includes an initial feature extraction unit, a division unit, a first feature extraction unit, a second feature extraction unit and a third fusion unit; wherein, extracting image information feature samples of the original image samples through the image information feature extraction module includes: extracting features of the original image samples through the initial feature extraction unit to obtain initial image information feature samples; dividing the initial image information feature samples into image blocks through the division unit to obtain initial image block information feature samples, the initial image block information feature samples including initial image information feature samples corresponding to each image block in the original image sample; extracting features of the initial image block information feature samples through the first feature extraction unit and the second feature extraction unit respectively to obtain first image block information feature samples and second image block information feature samples; fusing the first image block information feature samples and the second image block information feature samples through the third fusion unit to obtain image information feature samples of the original image samples.

[0153] It should be noted that the initial feature extraction unit is a pre-trained feature extraction network, which remains frozen during the training process of the exposure adjustment model and does not perform parameter update.

[0154] In some embodiments, the global feature extraction module includes a first global feature extraction unit, a second global feature extraction unit, a second attention mechanism unit, and a fourth fusion unit;

[0155] Among them, the global feature samples of the original image samples are extracted through the global feature extraction module, including: extracting the first initial global feature samples of the original image samples through the first global feature extraction unit; extracting the second initial global feature samples of the original image samples through the second global feature extraction unit; performing feature extraction on the fused global feature samples along different spatial directions through the second attention mechanism unit to obtain intermediate local feature samples, and the fused global feature samples are features obtained by fusion of the first initial global feature samples and the second initial global feature samples; and fusing the intermediate global feature samples and the second initial global feature samples through the fourth fusion unit to obtain the global feature samples of the original image samples.

[0156] S1102: predicting an adjusted exposure value corresponding to a next frame of image samples of the original image samples based on the exposure feature samples using the exposure adjustment model to be trained to obtain a predicted adjusted exposure value.

[0157] In some embodiments, the exposure feature samples also include exposure attribute parameter samples of the photosensitive device, and the exposure adjustment model to be trained also includes an adapter, a KAN network, a fifth fusion unit and a prediction unit; wherein, the exposure adjustment model to be trained predicts the adjusted exposure value corresponding to the next frame image sample of the original image sample based on the exposure feature samples, including: determining the weights corresponding to the image quality feature samples, the image information feature samples, the global feature samples and the exposure attribute parameter samples in the exposure feature samples through the KAN network, and fusing the image quality feature samples, the image information feature samples, the global feature samples and the exposure attribute parameter samples based on the weights to obtain the adjusted exposure feature samples; determining the exposure environment feature samples under the current exposure environment conditions based on the adjusted exposure feature samples and the initial image information feature samples through the adapter; fusing the adjusted exposure feature samples and the exposure environment feature samples through the fifth fusion unit to obtain the comprehensive exposure feature samples; and predicting the adjusted exposure value corresponding to the next frame image sample of the original image sample based on the comprehensive exposure feature samples through the prediction unit.

[0158] By fusing the exposure attribute parameter samples of different photosensitive devices, the model can learn the exposure sensitivity of different photosensitive devices, which facilitates the subsequent model to adapt to different photosensitive devices. That is, according to the exposure attribute parameters in the input model, the exposure sensitivity of the photosensitive device corresponding to the exposure attribute parameters is determined, so that the adjusted exposure value for the next frame of the image can be determined according to the exposure sensitivity and exposure characteristics of the photosensitive device, so as to improve the adjustment effect and adjustment accuracy of the image exposure.

[0159] S1103. Determine a predicted exposure adjustment loss based on a difference between the predicted adjusted exposure value and the labeled adjusted exposure value corresponding to the original image sample, and converge the exposure adjustment model to be trained based on the predicted exposure adjustment loss to obtain an exposure adjustment model.

[0160] Among them, the annotated adjusted exposure value can be obtained by the following steps: obtaining multiple exposure scores corresponding to each of the multiple training image samples, each of the multiple training image samples corresponds to a different exposure value, and the multiple training image samples and the original image sample are images taken of the same object; for each training image sample in the multiple training image samples, based on the average value of the multiple exposure scores corresponding to the training image sample, determining the comprehensive exposure score corresponding to the training image sample; selecting the exposure value of the training image sample with the highest comprehensive exposure score among the multiple training image samples, and determining it as the annotated adjusted exposure value corresponding to the original image.

[0161] For example, by setting different light intensity values ​​for the same object and taking multiple images with different exposure values, multiple training image samples are obtained. Then, a photo with a suitable light intensity is selected as a reference image anchor, and the exposure intensity of the reference image is set to 0.5. At the same time, multiple labelers compare the exposure of multiple training image samples with the reference image, and score the multiple training image samples according to the comparison results.

[0162] For example, taking one labeler among multiple labelers as an example, for each training image sample among multiple training image samples, if the exposure effect of the training image sample is better than that of the reference image, a first score (such as "1") is set for the training image sample; if the exposure effect of the training image sample is worse than that of the reference image, a second score (such as "0") is set for the training image sample. Finally, the comprehensive exposure score of the training image sample is obtained by dividing the total score of the training image sample by the total number of labelers.

[0163] In this way, the comprehensive exposure scores corresponding to the multiple training image samples can be obtained. Then, the exposure value of the training image sample corresponding to the highest comprehensive exposure score is selected as the labeled adjustment exposure value.

[0164] Afterwards, by calculating the difference between the predicted exposure adjustment value and the labeled exposure adjustment value corresponding to the original image sample, the exposure adjustment error of the model can be quantified. This quantified error is the "image exposure adjustment prediction loss". This loss value not only reflects the prediction ability of the model in the current state, but also is an important basis for guiding the subsequent learning direction of the model.

[0165] In order to minimize this prediction loss, we can use optimization algorithms such as gradient descent to gradually adjust the parameters in the model so that the model can better fit the image exposure characteristics in the training data set. As the training process continues, the loss will gradually decrease, indicating that the model's prediction accuracy for image exposure adjustment is improving. When the model's prediction accuracy reaches the preset accuracy or the change in the comprehensive loss value tends to be stable, the model is considered to have converged, and the model obtained at this time is the exposure adjustment model that has been optimized and trained.

[0166] This embodiment adopts a comparative scoring method instead of a direct scoring method, which can avoid the defect that it is difficult to accurately give an image exposure degree score based on the human eye alone, improve the accuracy of labeling and adjusting the exposure value, and thus help model training, improve model accuracy and image exposure adjustment effect.

[0167] It should be noted that in order to adapt the model to different scenes, the corresponding adapter can be trained by selecting the original image samples under a specific scene. In some implementations, the training of the adapter can be synchronized with other components of the system. For example, when training an adapter for a specific scene, in addition to the adapter itself, other parts of the system will also participate in the training process. In order to improve the efficiency of training, other model components other than the adapter can be trained in the initial stage to build a basic exposure adjustment model. When the basic model is trained, it can be fixed, and then the corresponding adapter can be specially trained using image samples from different lighting conditions (such as dark light, strong light, and standard light). In this way, each adapter can better adapt to its corresponding specific scene. Finally, by combining these scene-specific adapters with the basic exposure adjustment model, an exposure adjustment model suitable for various lighting conditions can be created.

[0168] The exposure adjustment model in this embodiment may be a basic exposure adjustment model, or may be a comprehensive model formed by combining a basic model with a specific scene adapter.

[0169] The technical solution of the exposure adjustment method provided in this application has the following beneficial effects:

[0170] (1) By combining deep learning networks (such as Mamba and CNN) with traditional image evaluation algorithms, image exposure adjustment values ​​can be quickly predicted.

[0171] (2) The model outputs the image exposure adjustment value, that is, the exposure value required for the next frame can be predicted based on the current frame without the need for iterative exposure adjustment, which improves the real-time performance of the exposure adjustment algorithm and the efficiency of the overall exposure adjustment process.

[0172] (3) By combining the properties of different photosensitive devices and adding machine property configuration, photosensitive devices with different exposure sensitivities can be adapted, making this method more effective in adjusting the exposure level.

[0173] (4) Through the environmental adapter, data from different environments can be decoupled for training. When implementing reasoning in different environments, the model can be equipped with different adapters, thereby improving the robustness of the model to environmental adaptation.

[0174] Exemplary Devices

[0175] Corresponding to the above-mentioned image exposure adjustment method, an embodiment of the present application also provides an image exposure adjustment device. Fig.13 Schematic diagram of the structure of an image exposure adjustment device provided in an embodiment of the present application. Fig.13As shown, the image exposure adjustment device provided by the embodiment of the present application includes: an acquisition unit 131, a prediction unit 132 and an adjustment unit 133; wherein the acquisition unit 131 is used to acquire the exposure characteristics corresponding to the original image, and the exposure characteristics include the image quality characteristics, image information characteristics and global characteristics of the original image, the image quality characteristics are the characteristics that characterize the image quality of the original image, the image information characteristics are the characteristics that characterize the image content and structure of the original image, and the global characteristics characterize the overall characteristics of the original image; the prediction unit 132 is used to predict the adjusted exposure value corresponding to the next frame image of the original image based on the exposure characteristics; the adjustment unit 133 is used to adjust the exposure value of the next frame image based on the adjusted exposure value.

[0176] In some embodiments, the acquisition unit 131 acquires the exposure features corresponding to the original image, and the prediction unit 132 predicts the adjusted exposure value corresponding to the next frame image of the original image based on the exposure features, including: inputting the original image into an exposure adjustment model, acquiring the exposure features corresponding to the original image by the exposure adjustment model, and predicting the adjusted exposure value corresponding to the next frame image of the original image based on the exposure features.

[0177] In some embodiments, the exposure adjustment model includes an image quality feature extraction module, an image information feature extraction module, a global feature extraction module and a fusion module; wherein, the exposure adjustment model obtains the exposure features corresponding to the original image, including: extracting the image quality features of the original image through the image quality feature extraction module; extracting the image information features of the original image through the image information feature extraction module; extracting the global features of the original image through the global feature extraction module; and fusing the image quality features, image information features and global features through the fusion module to obtain the exposure features corresponding to the original image.

[0178] In some embodiments, the image quality feature extraction module includes an image size conversion unit, a first image quality feature extraction unit and a second image quality feature extraction unit; wherein, extracting the image quality feature of the original image through the image quality feature extraction module includes: processing the original image into a first resolution image and a second resolution image through the image size conversion unit, the first resolution being greater than the second resolution; extracting a first image quality feature corresponding to the first resolution image through the first image quality feature extraction unit; and extracting a second image quality feature corresponding to the second resolution image through the second image quality feature extraction unit.

[0179] In some embodiments, the first image quality feature extraction unit includes an SPPnet unit, a dilated convolution unit, a first fusion unit, a first attention mechanism unit, and a second fusion unit; wherein, extracting the first image quality feature corresponding to the first resolution image through the first image quality feature extraction unit includes: extracting the initial spatial structure feature of the first resolution image through the SPPnet unit, and extracting the deep spatial structure feature of the initial spatial structure feature through the dilated convolution unit; fusing the initial spatial structure feature and the deep spatial structure feature through the first fusion unit to obtain a fused spatial structure feature; determining the weights corresponding to each channel and the weights corresponding to each spatial position in the fused spatial structure feature through the first attention mechanism unit, and adjusting the importance of the features corresponding to each channel based on the weights corresponding to each channel in the fused spatial structure feature, and adjusting the importance of the features corresponding to each spatial position in the spatial structure feature based on the weights corresponding to each spatial position, to obtain an adjusted spatial structure feature; fusing the adjusted spatial structure feature and the deep spatial structure feature through the second fusion unit to obtain the first image quality feature corresponding to the first resolution image.

[0180] In some embodiments, the second image quality feature extraction unit includes an image basic indicator feature extraction unit and a KAN network; wherein, extracting the second image quality feature corresponding to the second resolution image through the second image quality feature extraction unit includes: extracting each image basic quality indicator feature of the second resolution image through the image basic indicator feature extraction unit; determining the weight corresponding to each of the image basic quality indicator features through the KAN network, and fusing the each image basic quality indicator feature based on the weight of the each image basic quality indicator feature to obtain the second image quality feature corresponding to the second resolution image.

[0181] In some embodiments, the image information feature extraction module includes an initial feature extraction unit, a division unit, a first feature extraction unit, a second feature extraction unit and a third fusion unit; wherein, extracting the image information features of the original image through the image information feature extraction module includes: performing feature extraction on the original image through the initial feature extraction unit to obtain initial image information features; performing image block division on the initial image information features through the division unit to obtain initial image block information features, wherein the initial image block information features include initial image information features corresponding to each image block in the original image; performing feature extraction on the initial image block information features through the first feature extraction unit and the second feature extraction unit respectively to obtain first image block information features and second image block information features; and fusing the first image block information features and the second image block information features through the third fusion unit to obtain image information features of the original image.

[0182] In some embodiments, the global feature extraction module includes a first global feature extraction unit, a second global feature extraction unit, a second attention mechanism unit and a fourth fusion unit; wherein, extracting the global features of the original image through the global feature extraction module includes: extracting the first initial global features of the original image through the first global feature extraction unit; extracting the second initial global features of the original image through the second global feature extraction unit; performing feature extraction on the fused global features along different spatial directions through the second attention mechanism unit to obtain intermediate local features, and the fused global features are features obtained by fusing the first initial global features and the second initial global features; fusing the intermediate global features and the second initial global features through the fourth fusion unit to obtain the global features of the original image.

[0183] In some embodiments, the exposure feature also includes exposure attribute parameters of the photosensitive device, and the exposure adjustment model also includes an adapter, a KAN network, a fifth fusion unit and a prediction unit; wherein, the exposure adjustment model predicts the adjusted exposure value corresponding to the next frame image of the original image based on the exposure feature, including: determining the weights corresponding to the image quality information feature, the global feature and the exposure attribute parameter in the exposure feature through the KAN network, and fusing the image quality information feature, the global feature and the exposure attribute parameter based on the weights to obtain an adjusted exposure feature, wherein the image quality information feature is a fusion feature of the image quality feature and the image information feature; determining the environmental features of the current scene through the adapter based on the adjusted exposure feature and the initial image information feature; fusing the adjusted exposure feature and the environmental features through the fifth fusion unit to obtain a comprehensive exposure feature; and predicting the adjusted exposure value corresponding to the next frame image of the original image based on the comprehensive exposure feature through the prediction unit.

[0184] In some embodiments, the exposure adjustment model is trained through the following steps: obtaining exposure feature samples corresponding to the original image samples through the exposure adjustment model to be trained, and predicting the adjusted exposure value corresponding to the next frame image sample of the original image sample based on the exposure feature samples to obtain the predicted adjusted exposure value; the exposure feature samples include image quality feature samples, image information feature samples and global feature samples of the original image samples; determining the predicted exposure adjustment loss based on the difference between the predicted adjusted exposure value and the labeled adjusted exposure value corresponding to the original image sample, and converging the exposure adjustment model to be trained based on the predicted exposure adjustment loss to obtain the exposure adjustment model.

[0185] In some embodiments, the annotated adjusted exposure value corresponding to the original image sample is obtained by the following steps: obtaining multiple exposure scores corresponding to each of multiple training image samples, each of the multiple training image samples corresponds to a different exposure value, and the multiple training image samples and the original image sample are images taken of the same object; for each training image sample in the multiple training image samples, based on the average value of the multiple exposure scores corresponding to the training image sample, determining the comprehensive exposure score corresponding to the training image sample; selecting the exposure value of the training image sample with the highest comprehensive exposure score among the multiple training image samples, and determining it as the annotated adjusted exposure value corresponding to the original image.

[0186] The image exposure adjustment device provided in this embodiment belongs to the same application concept as the image exposure adjustment method provided in the above embodiments of this application, and can execute the image exposure adjustment method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of executing the image exposure adjustment method. For technical details not fully described in this embodiment, please refer to the specific processing content of the image exposure adjustment method provided in the above embodiments of this application, and will not be repeated here.

[0187] The functions implemented by the above acquisition unit 131, prediction unit 132 and adjustment unit 133 may be implemented by the same or different processors, respectively, and are not limited in the embodiments of the present application.

[0188] It should be understood that the units in the above devices can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit of the device, wherein the processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory in the device or a memory outside the device. Alternatively, the units in the device can be implemented in the form of hardware circuits, and the functions of some or all units can be realized by designing the hardware circuits. The hardware circuit can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are realized by designing the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit can be implemented by PLD, taking FPGA as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by the configuration file, so as to realize the functions of some or all of the above units. All units of the above devices can be implemented in the form of a processor calling software, or in the form of hardware circuits, or in part by a processor calling software, and the remaining part is implemented in the form of hardware circuits.

[0189] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and run instructions, such as a CPU, a microprocessor, a GPU, or a DSP; in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0190] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0191] In addition, all or part of the units in the above device can be integrated together, or can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a SOC. The SOC may include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The type of the at least one processor may be different, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0192] Exemplary Electronic Devices

[0193] The present application embodiment provides an electronic device, see Fig.14 As shown, the electronic device includes:

[0194] Memory 200 and processor 210;

[0195] The memory 200 is connected to the processor 210 and is used to store programs;

[0196] The processor 210 is used to implement the image exposure adjustment method disclosed in any of the above embodiments by running the program stored in the memory 200.

[0197] Specifically, the electronic device may further include: a bus, a communication interface 220 , an input device 230 and an output device 240 .

[0198] The processor 210, the memory 200, the communication interface 220, the input device 230 and the output device 240 are connected to each other via a bus.

[0199] A bus may include a pathway that transfers information between components of a computer system.

[0200] The processor 210 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the scheme of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0201] The processor 210 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0202] The memory 200 stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes a computer operation instruction. More specifically, the memory 200 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0203] The input device 230 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0204] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0205] The communication interface 220 may include any transceiver or the like to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0206] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement each step of any image exposure adjustment method provided in the above embodiments of the present application.

[0207] An embodiment of the present application also proposes a chip, which includes a processor and a data interface. The processor reads and runs a program stored in a memory through the data interface to execute the image exposure adjustment method introduced in any of the above embodiments. The specific processing process and its beneficial effects can be found in the introduction to the embodiment of the above-mentioned image exposure adjustment method.

[0208] Exemplary computer program products and storage media

[0209] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the image exposure adjustment method according to various embodiments of the present application described in any of the above embodiments of this specification.

[0210] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0211] In addition, an embodiment of the present application may also be a storage medium on which a computer program is stored, and the computer program is executed by a processor to execute the steps of the image exposure adjustment method according to various embodiments of the present application described in any of the above embodiments of this specification.

[0212] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0213] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0214] The steps in the methods of each embodiment of the present application can be adjusted in order, combined and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0215] The modules and sub-modules in the devices and terminals of the various embodiments of the present application can be combined, divided and deleted according to actual needs.

[0216] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0217] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0218] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.

[0219] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0220] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly by hardware, software units executed by a processor, or a combination of the two. The software units may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0221] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0222] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for adjusting image exposure, characterized in that: include: Acquire exposure features corresponding to the original image, wherein the exposure features include image quality features, image information features, and global features of the original image, wherein the image quality features are features characterizing the image quality of the original image, the image information features are features characterizing the image content and structure of the original image, and the global features characterize the overall features of the original image; Predicting an adjusted exposure value corresponding to a next frame of the original image based on the exposure feature; The exposure value of the next frame image is adjusted based on the adjusted exposure value.

2. The method according to claim 1, characterized in that The acquiring the exposure feature corresponding to the original image and predicting the adjusted exposure value corresponding to the next frame image of the original image based on the exposure feature includes: The original image is input into an exposure adjustment model, the exposure adjustment model obtains exposure features corresponding to the original image, and an adjusted exposure value corresponding to a next frame image of the original image is predicted based on the exposure features.

3. The method according to claim 2, characterized in that The exposure adjustment model includes an image quality feature extraction module, an image information feature extraction module, a global feature extraction module and a fusion module; The exposure adjustment model is used to obtain the exposure features corresponding to the original image, including: Extracting the image quality feature of the original image by the image quality feature extraction module; Extracting the image information features of the original image by the image information feature extraction module; Extracting the global features of the original image by the global feature extraction module; The image quality feature, the image information feature and the global feature are fused through the fusion module to obtain the exposure feature corresponding to the original image.

4. The method according to claim 3, characterized in that The image quality feature extraction module includes an image size conversion unit, a first image quality feature extraction unit and a second image quality feature extraction unit; The step of extracting the image quality feature of the original image by the image quality feature extraction module includes: Processing the original image into a first resolution image and a second resolution image by the image size conversion unit, wherein the first resolution is greater than the second resolution; Extracting a first image quality feature corresponding to the first resolution image by the first image quality feature extraction unit; The second image quality feature corresponding to the second resolution image is extracted by the second image quality feature extraction unit.

5. The method according to claim 4, characterized in that The first image quality feature extraction unit includes a spatial pyramid pooling convolution unit, a hole convolution unit, a first fusion unit, a first attention mechanism unit and a second fusion unit; The extracting, by the first image quality feature extraction unit, a first image quality feature corresponding to the first resolution image includes: Extracting initial spatial structural features of the first resolution image through the spatial pyramid pooling convolution unit, and extracting deep spatial structural features of the initial spatial structural features through the dilated convolution unit; The initial spatial structure feature and the deep spatial structure feature are fused by the first fusion unit to obtain a fused spatial structure feature; Determine the weights corresponding to the channels and the weights corresponding to the spatial positions in the fused spatial structure feature by the first attention mechanism unit, and adjust the importance of the features corresponding to the channels based on the weights corresponding to the channels in the fused spatial structure feature, and adjust the importance of the features corresponding to the spatial positions in the spatial structure feature based on the weights corresponding to the spatial positions, to obtain an adjusted spatial structure feature; The adjusted spatial structure feature and the deep spatial structure feature are fused through the second fusion unit to obtain a first image quality feature corresponding to the first resolution image.

6. The method according to claim 4, characterized in that The second image quality feature extraction unit includes an image basic index feature extraction unit and a KAN network; The extracting, by the second image quality feature extraction unit, the second image quality feature corresponding to the second resolution image includes: Extracting basic image quality indicator features of each image of the second resolution image by the basic image indicator feature extraction unit; The weights corresponding to the basic quality indicator features of each image are determined by the KAN network, and based on the weights of the basic quality indicator features of each image, the basic quality indicator features of each image are fused to obtain a second image quality feature corresponding to the second resolution image.

7. The method according to claim 3, characterized in that The image information feature extraction module includes an initial feature extraction unit, a division unit, a first feature extraction unit, a second feature extraction unit and a third fusion unit; The step of extracting the image information features of the original image by the image information feature extraction module includes: Extracting features from the original image by the initial feature extraction unit to obtain initial image information features; The initial image information feature is divided into image blocks by the dividing unit to obtain initial image block information features, wherein the initial image block information features include initial image information features corresponding to each image block in the original image; Extracting features of the initial image block information features by the first feature extraction unit and the second feature extraction unit respectively to obtain first image block information features and second image block information features; The first image block information feature and the second image block information feature are fused by the third fusion unit to obtain the image information feature of the original image.

8. The method according to claim 3, characterized in that The global feature extraction module includes a first global feature extraction unit, a second global feature extraction unit, a second attention mechanism unit and a fourth fusion unit; The extracting of the global features of the original image by the global feature extraction module includes: Extracting a first initial global feature of the original image by the first global feature extraction unit; Extracting a second initial global feature of the original image by the second global feature extraction unit; The second attention mechanism unit performs feature extraction on the fused global feature along different spatial directions to obtain intermediate local features, wherein the fused global feature is a feature obtained by fusing the first initial global feature and the second initial global feature; The intermediate global feature and the second initial global feature are fused by the fourth fusion unit to obtain the global feature of the original image.

9. The method according to claim 7, characterized in that: The exposure feature also includes exposure attribute parameters of the photosensitive device, and the exposure adjustment model also includes an adapter, a KAN network, a fifth fusion unit and a prediction unit; The exposure adjustment model predicts the adjusted exposure value corresponding to the next frame image of the original image based on the exposure feature, including: Determining weights corresponding to the image quality information feature, the global feature, and the exposure attribute parameter in the exposure feature through the KAN network, and fusing the image quality information feature, the global feature, and the exposure attribute parameter based on the weights to obtain an adjusted exposure feature, wherein the image quality information feature is a fusion feature of the image quality feature and the image information feature; Determining, by the adapter, environmental features in the current scene based on the adjusted exposure features and the initial image information features; fusing the adjusted exposure feature and the environmental feature through the fifth fusion unit to obtain a comprehensive exposure feature; The prediction unit predicts an adjusted exposure value corresponding to a next frame image of the original image based on the comprehensive exposure feature.

10. The method according to any one of claims 1 to 9, characterized in that: The exposure adjustment model is trained by the following steps: Acquire exposure feature samples corresponding to the original image samples through the exposure adjustment model to be trained, and predict the adjusted exposure value corresponding to the next frame image sample of the original image sample based on the exposure feature samples to obtain the predicted adjusted exposure value; the exposure feature samples include image quality feature samples, image information feature samples and global feature samples of the original image samples; A predicted exposure adjustment loss is determined based on a difference between the predicted adjusted exposure value and the labeled adjusted exposure value corresponding to the original image sample, and the exposure adjustment model to be trained is converged based on the predicted exposure adjustment loss to obtain the exposure adjustment model.

11. The method according to claim 10, characterized in that The annotated adjusted exposure value corresponding to the original image sample is obtained by the following steps: Acquire a plurality of exposure scores corresponding to a plurality of training image samples, each of the plurality of training image samples corresponds to a different exposure value, and the plurality of training image samples and the original image sample are images obtained by photographing the same object; For each training image sample among the multiple training image samples, determining a comprehensive exposure score corresponding to the training image sample based on an average value of multiple exposure scores corresponding to the training image sample; The exposure value of the training image sample with the highest comprehensive exposure score among the multiple training image samples is selected to determine the labeled adjusted exposure value corresponding to the original image.

12. An image exposure adjustment device, characterized in that: include: An acquisition unit, configured to acquire exposure characteristics corresponding to an original image, wherein the exposure characteristics include image quality characteristics, image information characteristics, and global characteristics of the original image; A prediction unit, configured to predict an adjusted exposure value corresponding to a next frame image of the original image based on the exposure feature; An adjusting unit is used to adjust the exposure value of the next frame image based on the adjusted exposure value.

13. An electronic device, characterized in that: including memory and processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement the method according to any one of claims 1 to 11 by running the program in the memory.

14. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 11 is implemented.

15. A computer program product, characterized in that The method comprises computer program instructions, which, when executed by a processor, cause the processor to implement the method as claimed in any one of claims 1 to 11.

Images