Image processing method and device, product, equipment and medium

By enhancing content and format transformation of the target image, combined with weight fusion processing, the problem of excessive enhancement of face images during low dynamic range image optimization is solved, and the image optimization effect is improved.

CN118918051AActive Publication Date: 2024-11-08TENCENT TECHNOLOGY (SHENZHEN) CO LTD

Patent Information

Application Number
CN202310481408.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-04-27
Publication Date
2024-11-08
Estimated Expiration
2043-04-27

AI Technical Summary

Technical Problem

When the prior art optimizes images with low dynamic range to high dynamic range, it is easy to lead to excessive enhancement of face images, resulting in poor optimization effects.

Method used

By acquiring the target image, content enhancement processing and image format transformation are performed, and fusion processing is combined with image color weights, position weights and color difference weights to generate and optimize images, retain the original features of the target object, and avoid excessive enhancement.

Benefits of technology

It achieves better optimization processing for the target image, improves optimization effect, and ensures better visual effects of face images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918051A_ABST
    Figure CN118918051A_ABST
Patent Text Reader

Abstract

The invention discloses an image processing method and device, a product, equipment and a medium. The method comprises the steps that a target image containing a target object is acquired; the target image is an image in a first dynamic range, the target image comprises a first object image, and the first object image is a local area image of a target object in the target image; performing enhancement processing on the image content of the target image to obtain a content enhanced image; performing conversion processing on the image format of the first object image to obtain a format conversion image; the format conversion image is an image in a second dynamic range, the second dynamic range is larger than the first dynamic range, and the image content of the format conversion image is the same as the image content of the first object image; performing fusion processing on the format conversion image and the content enhancement image to obtain an optimized image of the target image; the optimized image is an image in the second dynamic range. According to the invention, the optimization effect of the target image can be improved, and a more excellent optimized image of the target image can be obtained.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and in particular to an image processing method, apparatus, product, equipment and medium. Background Art

[0002] Image optimization is extremely important in the field of image processing. Image optimization may include optimizing a low dynamic range image into a high dynamic range image. The dynamic range is the intensity range that light in the image can reach.

[0003] Among them, the low dynamic range image to be optimized may include a face image. In existing applications, when the low dynamic range image is optimized into a high dynamic range image, a uniform content enhancement process is performed on the entire low dynamic range image. The visual effect of the face image is very prominent. When the low dynamic range image is subjected to content enhancement process, the face image in the low dynamic range image is often excessively enhanced, resulting in poor optimization effect of the face image in the low dynamic range image, and further resulting in poor visual effect of the optimized face image. Summary of the invention

[0004] The present application provides an image processing method, apparatus, product, equipment and medium, which can improve the optimization effect of a target image and thereby obtain a better optimized image of the target image.

[0005] On the one hand, the present application provides an image processing method, the method comprising:

[0006] Acquire a target image containing a target object; the target image is an image of a first dynamic range, the target image contains a first object image, and the first object image refers to a local area image of the position of the target object in the target image;

[0007] Performing image content enhancement processing on the target image to obtain a content enhanced image;

[0008] Performing a format conversion process on the image format of the first object image to obtain a format conversion image; the format conversion image is an image in a second dynamic range, the second dynamic range is greater than the first dynamic range, and the image content of the format conversion image is the same as the image content of the first object image;

[0009] The format-converted image and the content-enhanced image are fused to obtain an optimized image of the target image; the optimized image is an image in the second dynamic range.

[0010] On one hand, the present application provides an image processing device, which includes:

[0011] An acquisition module is used to acquire a target image containing a target object; the target image is an image of a first dynamic range, the target image contains a first object image, and the first object image refers to a local area image of the position where the target object is located in the target image;

[0012] An enhancement module, used for enhancing the image content of the target image to obtain a content-enhanced image;

[0013] a transformation module, configured to transform the image format of the first object image to obtain a format-transformed image; the format-transformed image is an image in a second dynamic range, the second dynamic range is greater than the first dynamic range, and the image content of the format-transformed image is the same as the image content of the first object image;

[0014] The fusion module is used to fuse the format-converted image and the content-enhanced image to obtain an optimized image of the target image; the optimized image is an image in the second dynamic range.

[0015] Optionally, the fusion module performs fusion processing on the format-converted image and the content-enhanced image to obtain an optimized image of the target image, including:

[0016] Obtaining a first fusion weight of the format transformed image;

[0017] Based on the first fusion weight, the format transformed image and the content enhanced image are fused to obtain an optimized image.

[0018] Optionally, the first fusion weight is obtained based on any one of the following weights, or the first fusion weight is obtained by fusing at least two of the following weights:

[0019] an image color weight of the format transformed image; the image color weight being obtained based on a difference between a reference color of the target object and an image color of the first object image;

[0020] An image position weight of the format transformed image; the image position weight is obtained based on the position of each pixel in the format transformed image;

[0021] The image color difference weight of the format transformed image; the image color difference weight is obtained based on the pixel difference between the format transformed image and the second object image, and the second object image refers to the local area image of the position of the target object in the content enhanced image.

[0022] Optionally, the image color of the first object image includes the pixel color of each pixel in the first object image; and the process of the fusion module generating the image color weight includes:

[0023] Obtaining a reference pixel point of the target object in the target image, and using the pixel color of the reference pixel point as the reference color of the target object;

[0024] Obtaining a target color distance between a pixel color of each pixel point in the first object image and a reference color of the target object;

[0025] An image color weight is generated based on a target color distance between a pixel color of each pixel point in the first object image and a reference color of the target object.

[0026] Optionally, any pixel in the first object image is a target pixel, there are one or more reference pixels, and one reference pixel has a reference color of the target object;

[0027] The method in which the fusion module obtains the target color distance between the pixel color of each pixel point in the first object image and the reference color of the target object includes:

[0028] Obtain the color distance between the pixel color of the target pixel and the reference color of each reference pixel;

[0029] The color distance with the smallest value among the color distances between the pixel color of the target pixel and the reference colors of one or more reference pixels is used as the target color distance between the pixel color of the target pixel and the reference color of the target object.

[0030] Optionally, the fusion module generates the image color weight based on the target color distance between the pixel color of each pixel point in the first object image and the reference color of the target object, including:

[0031] Using a color weight mapping function, mapping the target color distance between each pixel in the first object image and the reference color of the target object is performed to generate a color weight for each pixel in the first object image;

[0032] The dimension of the format transformation image is the same as that of the first object image, there is a one-to-one correspondence between pixels having the same pixel position in the first object image and the format transformation image, the color weight of any pixel in the first object image is the color weight of the corresponding pixel in the format transformation image, and the image color weight includes the color weight of each pixel in the format transformation image.

[0033] Optionally, the process of generating image position weights by the fusion module includes:

[0034] Acquire a first image length of the format-converted image in a first direction, and generate a first Gaussian distribution based on the first image length;

[0035] Acquire a second image length of the format-converted image in a second direction, and generate a second Gaussian distribution based on the second image length;

[0036] Based on the first Gaussian distribution and the second Gaussian distribution, an image position weight is generated.

[0037] Optionally, the fusion module generates the first Gaussian distribution based on the first image length, including:

[0038] Generate a first initial Gaussian distribution based on the first image length; the first initial Gaussian distribution has a first Gaussian mean, and the first Gaussian mean is half the value of the first image length;

[0039] The first initial Gaussian distribution is normalized to obtain a first Gaussian distribution.

[0040] Optionally, the first initial Gaussian distribution has a first independent variable and a first dependent variable; and the fusion module normalizes the first initial Gaussian distribution to obtain the first Gaussian distribution, including:

[0041] Obtaining a first reference value of a first independent variable, and obtaining a first variable value of a first dependent variable when the value of the first independent variable is the first reference value, wherein the first reference value is not equal to a first Gaussian mean;

[0042] Normalize the values ​​of the first dependent variable within the first value range in the first initial Gaussian distribution to a first constant value, and normalize the values ​​of the first dependent variable within the second value range in the first initial Gaussian distribution to a first target dependent variable, to obtain a first Gaussian distribution;

[0043] Among them, the first value range is the range in which the first dependent variable takes values ​​greater than or equal to the first variable value in the first initial Gaussian distribution, the second value range is the range in which the first dependent variable takes values ​​less than the first variable value in the first initial Gaussian distribution, and the first target dependent variable is obtained based on the first dependent variable and the first variable value.

[0044] Optionally, the fusion module generates a second Gaussian distribution based on the second image length, including:

[0045] Generate a second initial Gaussian distribution based on the second image length; the second initial Gaussian distribution has a second Gaussian mean, and the second Gaussian mean is half the value of the second image length;

[0046] The second initial Gaussian distribution is normalized to obtain a second Gaussian distribution.

[0047] Optionally, the second initial Gaussian distribution has a second independent variable and a second dependent variable; and the fusion module normalizes the second initial Gaussian distribution to obtain the second Gaussian distribution, including:

[0048] Obtaining a second reference value of the second independent variable, and obtaining a second variable value of the second dependent variable when the second independent variable is the second reference value, the second reference value is not equal to the second Gaussian mean;

[0049] Normalizing the values ​​of the second dependent variable within the third value range in the second initial Gaussian distribution to a second constant value, and normalizing the values ​​of the second dependent variable within the fourth value range in the second initial Gaussian distribution to a second target dependent variable, to obtain a second Gaussian distribution;

[0050] Among them, the third value range is the range in which the value of the second dependent variable in the second initial Gaussian distribution is greater than or equal to the value of the second variable, the fourth value range is the range in which the value of the second dependent variable in the second initial Gaussian distribution is less than the value of the second variable, and the second target dependent variable is obtained based on the second dependent variable and the second variable values.

[0051] Optionally, the first Gaussian distribution has a first independent variable, the second Gaussian distribution has a second independent variable, the value range of the first independent variable includes the serial number value of each pixel point of the format transformation image in the first direction, and the value range of the second independent variable includes the serial number value of each pixel point of the format transformation image in the second direction; any pixel point in the format transformation image has a serial number value in the first direction and a serial number value in the second direction; the fusion module generates the image position weight based on the first Gaussian distribution and the second Gaussian distribution, including:

[0052] Performing a product operation on the first Gaussian distribution and the second Gaussian distribution to obtain a target Gaussian distribution; the target Gaussian distribution has a first independent variable and a second independent variable;

[0053] The first independent variable of the target Gaussian distribution is respectively set as the serial number value of each pixel point in the format transformation image in the first direction, and the second independent variable of the target Gaussian distribution is respectively set as the serial number value of each pixel point in the format transformation image in the second direction, to obtain the position weight of each pixel point in the format transformation image;

[0054] The image position weight includes the position weight of each pixel in the format transformation image.

[0055] Optionally, the process of generating the image color difference weight by the fusion module includes:

[0056] Acquire a second object image in the content-enhanced image; the second object image refers to a local area image of the location of the target object in the content-enhanced image;

[0057] Calculate the difference between the pixel values ​​of the pixel points at the same pixel position in the format conversion image and the second object image to obtain the target pixel difference between each pixel point in the format conversion image and the pixel point at the same pixel position in the second object image;

[0058] A color difference weight mapping function is used to map the target pixel difference value corresponding to each pixel point in the format transformation image, and a color difference weight of each pixel point in the format transformation image is generated;

[0059] The image color difference weight includes the color difference weight of each pixel in the format transformation image.

[0060] Optionally, the fusion module fuses the format transformed image and the content enhanced image based on the first fusion weight to obtain an optimized image, including:

[0061] Determine a second fusion weight of a second object image based on the first fusion weight; the second object image refers to a local area image of the location of the target object in the content enhanced image;

[0062] performing weighted processing on the format-converted image based on the first fusion weight to obtain a weighted format-converted image;

[0063] Performing weighted processing on the second object image based on the second fusion weight to obtain a weighted second object image;

[0064] Performing sum processing on the weighted format transformation image and the weighted second object image to obtain a fused object image;

[0065] The images other than the second object image in the fused object image and the content enhanced image are combined to obtain an optimized image.

[0066] Optionally, the process of the above device acquiring the first object image includes:

[0067] Performing object detection on the target image to obtain an object detection frame of the target object in the target image;

[0068] Expanding the object detection frame on the target image to obtain an expanded detection frame;

[0069] The image within the expanded detection frame in the target image is taken as the first object image.

[0070] Optionally, the target image belongs to a video frame in the target video that contains the target object;

[0071] The above device is also used for:

[0072] Generate an optimized video of the target video based on the optimized image;

[0073] The optimized video is transmitted to the video client so that the video client plays the optimized video.

[0074] In one aspect, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes a method in one aspect of the present application.

[0075] In one aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor executes the method in the above aspect.

[0076] According to one aspect of the present application, a computer program product is provided, the computer program product comprising a computer program, the computer program being stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the method provided in various optional modes such as the above-mentioned one aspect.

[0077] The present application can firstly obtain a target image containing a target object; the target image is an image of a first dynamic range, the target image contains a first object image, and the first object image refers to a local area image of the position of the target object in the target image; then, the image content of the target image can be enhanced to obtain a content-enhanced image; and the image format of the first object image can be transformed to obtain a format-transformed image; the format-transformed image is an image of a second dynamic range, the second dynamic range is greater than the first dynamic range, and the image content of the format-transformed image is the same as the image content of the first object image; further, the format-transformed image and the content-enhanced image can be fused to obtain an optimized image of the target image; the optimized image is an image of the second dynamic range. It can be seen that the method proposed in the present application can perform overall content enhancement processing on the target image to obtain a content enhanced image, and can additionally perform format conversion processing on the local area image (i.e., the first object image) where the target object is located in the target image, and the image content of the first object image is kept unchanged during the format conversion processing to obtain a format converted image, and then, the content enhanced image and the format converted image can be fused to obtain an optimized image of the target image, so that in the obtained optimized image, some original features of the target object in the target image are retained through the format conversion image, and the target object in the target image will not be over-enhanced, and the image content of the target image can be enhanced as a whole through the content enhanced image, thereby achieving a better optimization processing of the target image, thereby obtaining a better optimized image of the target image. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0079] Figure 1 It is a structural diagram of a network architecture provided by this application;

[0080] Figure 2 It is a schematic diagram of a scene of image optimization provided by this application;

[0081] Figure 3 It is a flowchart of an image processing method provided by the present application;

[0082] Figure 4 is a schematic diagram of a scene for obtaining a first object image provided by the present application;

[0083] Figure 5 This is a schematic diagram of a scene for image fusion processing provided by the present application;

[0084] Figure 6 This is a schematic diagram of an architecture for optimizing images provided by the present application;

[0085] Figure 7 It is a flowchart of a method for generating image color weights provided by the present application;

[0086] Figure 8 It is a schematic diagram of a scenario for generating image color weights provided by the present application;

[0087] Fig. 9 It is a flowchart of a method for generating image position weights provided by the present application;

[0088] Fig.10 is a curve schematic diagram of a first initial Gaussian distribution provided by the present application;

[0089] Fig.11 is a curve schematic diagram of a second initial Gaussian distribution provided by the present application;

[0090] Fig.12 It is a flowchart of a method for generating image color difference weights provided by the present application;

[0091] Fig.13 This is a schematic diagram of a scene for generating image color difference weights provided by the present application;

[0092] Fig.14It is a structural schematic diagram of an image processing device provided by the present application;

[0093] Fig.15 It is a structural schematic diagram of a computer device provided by this application. DETAILED DESCRIPTION

[0094] The following will be combined with the drawings in this application to clearly and completely describe the technical solutions in this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0095] This application involves artificial intelligence related technologies. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0096] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0097] This application mainly involves computer vision technology in artificial intelligence. Among them, computer vision technology (Computer Vision, CV) Computer vision is a science that studies how to make machines "see". To put it more concretely, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further do graphic processing to make computer processing into images that are more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, automatic driving, smart transportation and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0098] The computer vision technology involved in this application mainly refers to the ability to optimize the target image to obtain an excellent optimized image of the target image. For details, please refer to the following Figure 3 Description in the corresponding embodiment.

[0099] First of all, it needs to be explained that all business data collected by this application (such as target images of target objects and other related data) are collected with the consent and authorization of the users, institutions or enterprises to which the business data belongs, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0100] Here, the relevant technical concepts involved in this application are described:

[0101] Dynamic range: The range of light intensities that an image contains simultaneously, used to describe the range of light intensity distribution in an image from the darkest shadows to the brightest highlights.

[0102] HDR: High Dynamic Range.

[0103] SDR: Standard Dynamic Range, standard dynamic range.

[0104] EOTF: Electro-Optical Transfer Function, electro-optical transfer function.

[0105] OETF: Opto-Electronic Transfer Function, photoelectric conversion function.

[0106] OOTF: Opto-Optical Transfer Function, optical-optical transfer function.

[0107] PQ: Perceptual Quantizer, perceptual quantization.

[0108] HLG: Hybrid Log–Gamma, hybrid log gamma.

[0109] TM:Tone Mapping, tone mapping.

[0110] iTM: Inverse Tone Mapping, inverse tone mapping.

[0111] See also Figure 1 , Figure 1 This is a schematic diagram of a network architecture provided by this application. Figure 1 As shown, the network architecture may include a server 200 and a terminal device cluster, and the terminal device cluster may include one or more terminal devices, and the number of terminal devices is not limited here. Figure 1 As shown, the multiple terminal devices may specifically include terminal device 1, terminal device 2, terminal device 3, ..., terminal device n; Figure 1 As shown, terminal device 1, terminal device 2, terminal device 3, ..., terminal device n can all be connected to the server 200 through a network, so that each terminal device can exchange data with the server 200 through the network connection.

[0112] like Figure 1 The server 200 shown can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal device can be a smart terminal such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart TV, etc. The following takes the communication between the terminal device 1 and the server 200 as an example to describe the specific embodiments of the present application.

[0113] Please also see Figure 2 , Figure 2 This is a schematic diagram of an image optimization scenario provided by this application. Figure 2As shown, the server 200 can obtain a target image to be optimized, the dynamic range of the target image can be a first dynamic range, the target image can include a target object, the target object can be a face, and the local area image of the position of the face in the target image can be called a first object image, that is, the target image includes the first object image.

[0114] First, the server 200 may perform overall enhancement processing on the image content of the target image to obtain a content-enhanced image of the target image. The server 200 may also perform format conversion processing on the first object image in the target image to obtain a format-converted image of the first object image. The dynamic range of the format-converted image may be a second dynamic range, which may be greater than the first dynamic range of the first object image. The image content of the format-converted image is the same as that of the first object image. It can be understood that the visual effect of the format-converted image is the same as that of the first object image.

[0115] Furthermore, the server 200 may fuse the content enhanced image with the format transformed image to obtain an optimized image of the target image. The server 200 may also send the optimized image to the terminal device 1 so that the terminal device 1 can display the optimized image of the target image to enhance the visual effect of the user viewing the image.

[0116] The optimized image of the target image obtained by the method provided in the present application can retain some of the original features of the target object in the target image in the optimized image through the fused format transformation image, preventing the target object from being over-enhanced (because the human eye is very sensitive to the vision of the human face, and slight changes in skin color or content can be perceived). The fused content enhancement image can also enhance the overall image content of the target image, thereby improving the optimization effect of the target image and obtaining a more superior optimized image of the target image.

[0117] See also Figure 3 , Figure 3 The figure is a flowchart of an image processing method provided by the present application. The execution subject in the embodiment of the present application may be an image processing device, which may be a computer device or a computer device cluster composed of multiple computer devices. The computer device may be a server, a terminal device, or other devices, without limitation. Figure 3 As shown, the method may include:

[0118] Step S101, obtaining a target image including a target object; the target image is an image in a first dynamic range, and the target image includes a first object image, which refers to a local area image of the target object in the target image.

[0119] Optionally, the image processing device can acquire a target image containing a target object, where the target image is an image to be optimized (i.e., needs to be optimized), and the target image can be an image with a first dynamic range, i.e., the dynamic range of the target image can be referred to as the first dynamic range. If the first dynamic range can be the dynamic range of SDR, the target image can be an SDR image.

[0120] Among them, the target object can be any object that needs to be subjected to specific optimization processing. The specific optimization processing can be an optimization processing by retaining some original features of the target object in the target image (such as the original color or content, etc.) to prevent the target object from being over-optimized (such as over-enhanced) during the optimization process of the target image, thereby resulting in problems such as insufficient aesthetics and poor visual effects of the optimized target object.

[0121] The type of target object may be different depending on the actual optimization scenario.

[0122] For example, since the human eye is very sensitive to changes in the image of a face (i.e., the human eye has a very strong perception of the visual effect of a face), if the face is over-enhanced, the face will become unclear and the skin color will become unrealistic. Therefore, in this scenario, the target object can be any face, and through specific optimization processing of the face, the original skin color and facial content of the face in the target image can be retained to a certain extent.

[0123] For another example, the target object can also be any object (such as a table, sofa, dye, cabinet, etc.). In this scenario, the target image can be a photo of the object (it can be a photo taken by the user of the object he owns, or it can be a photo taken by the merchant of the product). When optimizing the photo, on the basis of overall enhancement of the background of the object in the photo, the original object features of the object when it was actually photographed (such as the original color or original visual features of the object) can be retained to a certain extent, so as to ensure that the optimized object has stronger authenticity.

[0124] Similarly, the target object can also be the face of any animal, the entire body of any animal, etc. The specific target object can be determined based on the actual optimization scenario and there is no restriction on this.

[0125] Optionally, the target image may include one or more target objects. Since the method of performing specific optimization processing on each target object is independent and the same, it can be understood that the following is explained by taking the process of performing specific optimization processing on one target object as an example.

[0126] The target image may include a first object image, which may be a local area image of the target object in the target image. As described below, the method for obtaining the first object image may include but is not limited to:

[0127] The image processing device can perform object detection (also called target detection, where the target is the target object) on the target image to obtain a detection box (bounding box) at the location of the target object in the target image. The detection box can be called an object detection box.

[0128] For example, the image processing device can use a trained detection model to perform object detection on the target image to obtain a detection frame of the position of the target object in the target image. The detection model can be any trained model that can be used to detect the target object (such as the position of the target object).

[0129] Furthermore, the image processing device may perform expansion processing (ie, enlargement) on the object detection frame on the target image to obtain an expanded object detection frame, and the expanded object detection frame may be referred to as an expanded detection frame.

[0130] The image processing device may take the image within the expanded detection frame in the target image as the first object image. Therefore, it can be understood that the first object image not only includes the target object, but also includes a partial background image of the target object in the target image. Since the first object image may be an image of a local area in the target image, the size (such as dimension) of the first object image may be smaller than the size (such as dimension) of the target image.

[0131] The present application expands the object detection frame of the target object and obtains the first object image by expanding the detection frame. This allows the target object to have a partial background image around it for optimized transition when the target object is subjected to specific optimization processing. This can make the optimization of the target object more natural, and can also make the optimized target object and the optimized background image fit better (i.e., the fusion effect is better), without causing the optimized target object to have an obvious sense of boundary. For details, please refer to the following description.

[0132] Optionally, the principle of expanding the object detection frame to obtain an expanded detection frame can be shown in the following formula:

[0133] W R =(1+2α)×W B (1)

[0134] H R =(1+2α)×H B (2)

[0135] Among them, the object detection box can be a rectangular box, W B is the width of the object detection box, W R H is the width of the expanded detection frame. B is the height of the object detection box, H R is the height of the expanded detection frame. α is an adjustable parameter (which can be called an expansion parameter), through which the degree of expansion of the object detection frame (such as the ratio) can be adjusted. The larger α is, the greater the degree of expansion of the object detection frame, the larger the size of the expanded detection frame, and the larger the first object image obtained. The larger the first object image is, the larger the spatial transition area (such as the background area of ​​the target object in the first object image) when the target object is subsequently optimized. Therefore, it can be understood that α can be used to adjust the size of the spatial transition area when the target object is optimized. Optionally, the recommended value of α can be 1 / 5.

[0136] See also Figure 4 , Figure 4 is a schematic diagram of a scene for obtaining a first object image provided by the present application. Figure 4 As shown, the target object may be a human face, and the image processing device may perform face detection on the target image, that is, may detect and obtain an object detection frame of the target object in the target image.

[0137] Furthermore, the image processing device can perform expansion processing on the object detection frame on the target image (i.e., enlargement processing, the specific expansion ratio can be determined according to the actual application scenario, and can expand outward from the center of the object detection frame, such as expanding from the center of the object detection frame in all four directions), to obtain an expanded detection frame of the object detection frame.

[0138] Furthermore, the image processing device may regard the image within the expanded detection frame in the target image as a first object image, where the first object image includes the target object and a partial image around the target object.

[0139] Step S102, enhancing the image content of the target image to obtain a content-enhanced image.

[0140] Optionally, the image processing device can enhance the entire image content of the target image (such as the entire image screen) (which can be referred to as content up-conversion of the target image) to obtain a content-enhanced image. The enhancement processing can be understood as optimizing the image content of the target image.

[0141] For example, the content enhanced image can be obtained by enhancing the contrast, brightness and saturation of the image content of the target image (such as increasing the contrast, brightness and saturation). The enhancement of the image content of the target image can be achieved by adopting a corresponding image enhancement method according to the actual application scenario, and there is no limitation on this.

[0142] For example, the image processing device can select a traditional inverse tone mapping algorithm or an AI-based (such as based on a trained enhancement model, which can be a trained model for content enhancement of images) inverse tone mapping algorithm as needed to enhance the image content of the target image to obtain a content-enhanced image.

[0143] The dimension of the content-enhanced image (ie, the image dimension) is the same as that of the target image, and the number of pixels included in the content-enhanced image is also the same as that of the target image.

[0144] It is understandable that, since the content-enhanced image is obtained by enhancing the image content of the target image, the image content of the content-enhanced image is different from the image content of the target image, and the visual effect of the content-enhanced image is also different from the visual effect of the target image.

[0145] Furthermore, it should be noted that, in the process of enhancing the image content of the target image, the image format of the target image may also be transformed (such as included) to transform the target image from a first dynamic range to a second dynamic range, so that the obtained content-enhanced image may also be an image of the second dynamic range, and the second dynamic range is greater than the first dynamic range. For example, the second dynamic range may be an HDR dynamic range, and the content-enhanced image may be an HDR image. The process of transforming the image format of the target image may refer to the process of transforming the image format of the first object image described in the following step S103, and the principles of the two processes may be the same.

[0146] In other words, the image format of the content enhanced image may be the same as the image format of the optimized image of the target image in step S104 described below.

[0147] Step S103, transforming the image format of the first object image to obtain a format-transformed image; the format-transformed image is an image in a second dynamic range, the second dynamic range is greater than the first dynamic range, and the image content of the format-transformed image is the same as that of the first object image.

[0148] Optionally, the image processing device may transform the image format of the first object image (which may be referred to as performing a format up-conversion on the first object image) to obtain a format-transformed image. Since the first object image is an image in a first dynamic range, the image format transformation processing may be to transform the first object image from an image in a first dynamic range to an image in a second dynamic range, that is, the dynamic range of the format-transformed image may be a second dynamic range, which is greater than the first dynamic range.

[0149] It can be understood that since the dynamic range represents the intensity range of all light contained in the image (which can also be understood as the range of brightness and darkness), the larger the dynamic range, the larger the range of light intensity that can be contained in the image, the dark areas of the image can be darker and purer, and the bright areas of the image can be brighter with more details and layers, so that the visual effect of the image can be richer, and therefore the overall effect of the image can also be better.

[0150] If it is necessary to optimize the SDR image (such as the target image) into an HDR image, the second dynamic range can be the dynamic range of HDR, that is, the image format of the first object image is transformed, that is, the image format of the first object image is transformed from the image format of SDR to the image format of HDR, and the image format of the format-converted image is the image format of HDR.

[0151] However, it should be noted that, compared with the first object image, only the image format of the format-converted image has changed, and the image content of the format-converted image remains the same as that of the first object image, so that the visual effect of the format-converted image is the same as that of the first object image in a standard display environment. The format-converted image can be obtained by representing the image format of the first object image as the image format of an HDR image.

[0152] Optionally, the process of transforming (i.e. converting) the image format of the first object image (which may include the format of the color gamut of the image and the format of the light of the image) to obtain a format-transformed image may include:

[0153] Assuming that the first object image can be an RGB image (a color image) with a BT.709 color gamut (a color gamut standard) and a BT.709 gamma curve (a curve used to represent light), the first object image needs to be transformed into an RGB image with a BT.2020 color gamut (a color gamut standard) and a PQ transfer curve (a curve used to convert the format of light into the format of light for HDR) (the transformed image is a format-converted image).

[0154] First, the image processing device can perform EOTF conversion (i.e., electro-optical conversion) processing on the first object image obtained above, that is, apply the RGB channels of the first object image (including the R channel (red color channel), the G channel (green color channel), and the B channel (blue color channel)) to the EOTF conversion corresponding to the inverse transform of the BT.709 gamma curve, respectively, so as to convert the light on the RGB channels of the first object image into linear light. The image after the EOTF conversion of the first object image can be called an electro-optical conversion image.

[0155] Next, the image processing device may perform OOTF conversion (i.e., perform optical-optical conversion, such as converting linear scene light into display light for display) on the electro-optical conversion image, that is, apply the electro-optical conversion image to the OOTF conversion corresponding to the BT.709 gamma curve, and the image after the optical-optical conversion of the electro-optical conversion image may be referred to as an optical-optical conversion image. This step can make the visual effect of the format conversion image obtained by the subsequent conversion the same as the visual effect of the first object image.

[0156] Furthermore, the image processing device can perform color gamut conversion on the optical-to-optical conversion image, such as converting the optical-to-optical conversion image from the BT.709 color gamut to the BT.2020 color gamut, that is, representing the BT.709 color gamut of the optical-to-optical conversion image as the BT.2020 color gamut, and the image obtained after the color gamut conversion of the optical-to-optical conversion image can be referred to as a color gamut conversion image. The BT.2020 color gamut is larger than the BT.709 color gamut, and by performing color gamut conversion on the optical-to-optical conversion image, when the color gamut conversion image is subsequently processed, the color gamut of the processed color gamut conversion image can reach a larger range of values.

[0157] Finally, the image processing device can perform OETF conversion (i.e., photoelectric conversion) on the color gamut conversion image, that is, apply the color gamut conversion image to the PQ OETF curve (a curve used to convert the format of light into the format of HDR light) to convert the light of the color gamut conversion image into nonlinear light (the intensity of nonlinear light is generally linear with the subjective visual intensity of the human eye, that is, nonlinear light is more in line with the human eye's perception of light intensity), and a format conversion image (also called a format conversion image) can be obtained.

[0158] Among them, the color gamut of the above-mentioned first object image (such as BT.709 color gamut) and the color gamut of the format-converted image (such as BT.2020 color gamut) are only examples. The color gamut of the first object image and the color gamut of the format-converted image can also be set to other color gamuts. If it is other color gamuts, the color gamut used as an example in the above conversion process can be replaced with the other color gamut for conversion.

[0159] The same is true for the light format. The light format of the first object image (such as the light format represented by the BT.709 gamma curve) and the light format of the format conversion image (such as the light format represented by the PQ transmission curve) are only examples. The light format of the first object image and the light format of the format conversion image can also be other formats. If they are other formats, the light format given as an example in the above conversion process can be replaced with the other format for conversion.

[0160] By converting the format of the color gamut and the format of light of the first object image, the dynamic range of the first object image can be converted, that is, the first dynamic range of the first object image is converted into a second dynamic range. The format conversion of the color gamut and the format conversion of light both play a role in the conversion of the dynamic range, but the format conversion of light plays a major role in the conversion of the dynamic range.

[0161] Through the above process, the image format of the first object image is transformed, thereby obtaining a format-transformed image, wherein the dimension of the format-transformed image is the same as that of the first object image, and the number of pixels (i.e., image pixels) contained in the format-transformed image is the same as that of the first object image.

[0162] Step S104, fusing the format-converted image and the content-enhanced image to obtain an optimized image of the target image; the optimized image is an image in the second dynamic range.

[0163] Optionally, the image processing device may perform a fusion process on the format-converted image and content-enhanced image acquired above to obtain an optimized image of the target image, and the optimized image is also an image of the second dynamic range. For example, the optimized image may also be an HDR image. As the name implies, the optimized image is the image obtained after performing the above-mentioned optimization process on the target image, and the dimension of the optimized image may be the same as that of the target image.

[0164] The process in which the image processing device performs fusion processing on the format-converted image and the content-enhanced image to obtain an optimized image of the target image may be the following process:

[0165] The image processing device can obtain the fusion weight of the format transformed image, and the fusion weight of the format transformed image can be called the first fusion weight. As the name suggests, the first fusion weight is the weight for fusing the format transformed image when the format transformed image and the content enhanced image are fused.

[0166] The first fusion weight may be obtained by the image color weight of the format transformation image, the image position weight of the format transformation image and / or the image color difference weight of the format transformation image. The image color weight, the image position weight and the image color difference weight may each be represented as a corresponding matrix (which may be a two-dimensional matrix). In other words, the image color weight, the image position weight and the image color difference weight may all be weight matrices. The dimension of the weight matrix may be the same as the dimension of the format transformation image, such as the number of elements included in the weight matrix may be the same as the number of pixels included in the format transformation image, as described below.

[0167] Optionally, the image color weight may be obtained based on the difference (i.e., color difference) between the reference color of the target object and the image color of the first object image, i.e., the image color weight may be used to reflect the color difference between the reference color of the target object and the image color of the first object image. The process of generating the image color weight may refer to the following Figure 7 Detailed description in the corresponding embodiment.

[0168] An element (i.e., a value) in the image color weight can be called a color weight, and each pixel in the format transformation image can correspond to a color weight in the image color weight, that is, each pixel in the format transformation image can correspond to each element in the image color weight (i.e., each color weight), such as the color weight of any pixel in the format transformation image can be the color weight in the image color weight that has the same position as the pixel. For example, the color weight of the pixel at the 1st row and the 1st column in the format transformation image can be the color weight at the 1st row and the 1st column in the image color weight (i.e., the element at the 1st row and the 1st column); and the color weight of the pixel at the 2nd row and the 2nd column in the format transformation image can be the color weight at the 2nd row and the 2nd column in the image color weight (i.e., the element at the 2nd row and the 2nd column), and so on.

[0169] Optionally, the image position weight may be obtained based on the different positions of each pixel in the format transformed image, that is, the image position weight may be used to reflect the position of each pixel in the format transformed image. The process of generating the image position weight may refer to the following Fig. 9 Detailed description in the corresponding embodiment.

[0170] Similarly, an element (i.e., a value) in the image position weight can be called a position weight, and each pixel in the format-transformed image can correspond to a position weight in the image position weight, that is, each pixel in the format-transformed image can correspond to each element in the image position weight (i.e., each position weight), such as the position weight of any pixel in the format-transformed image can be the position weight in the image position weight that has the same position as the pixel. For example, the position weight of the pixel at the 1st row and 1st column in the format-transformed image can be the position weight at the 1st row and 1st column in the image position weight (i.e., the element at the 1st row and 1st column); and the position weight of the pixel at the 2nd row and 2nd column in the format-transformed image can be the position weight at the 2nd row and 2nd column in the image position weight (i.e., the element at the 2nd row and 2nd column), and so on.

[0171] Optionally, the image color difference weight can be obtained based on the difference between the pixel value of the pixel point in the format transformation image and the pixel value of the pixel point in the second object image (i.e., the pixel difference). Since the pixel value of the pixel point itself is used to reflect the chromaticity of the pixel point, the image color difference weight can be used to reflect the color difference between the pixel point in the format transformation image and the pixel point in the second object image. The second object image may refer to a local area image of the position of the target object in the content enhanced image. The dimension of the second object image may be the same as that of the first object image. The position of the first object image in the target image may also be the same as the position of the second object image in the content enhanced image. Here, it can be understood that both the target image and the content enhanced image may contain the target object, but the target object in the content enhanced image may be an object after the target object in the target image is enhanced. The process of generating the image color difference weight can be referred to as follows: Fig.12 Detailed description in the corresponding embodiment.

[0172] Similarly, an element (i.e., a value) in the image color difference weight can be called a color difference weight, and each pixel in the format-transformed image can correspond to a color difference weight in the image color difference weight, that is, each pixel in the format-transformed image can correspond to each element in the image color difference weight (i.e., each color difference weight), such as the color difference weight of any pixel in the format-transformed image can be the color difference weight in the image color difference weight that has the same position as the pixel. For example, the color difference weight of the pixel at the 1st row and 1st column in the format-transformed image can be the color difference weight at the 1st row and 1st column in the image color difference weight (i.e., the element at the 1st row and 1st column); and the color difference weight of the pixel at the 2nd row and 2nd column in the format-transformed image can be the color difference weight at the 2nd row and 2nd column in the image color difference weight (i.e., the element at the 2nd row and 2nd column), and so on.

[0173] Optionally, the first fusion weight can be obtained based on any one of the above-mentioned image color weights, image position weights, and image color difference weights. It can be understood that the above-mentioned image color weights can retain part of the original color of the target object in the target image in the optimized image (such as if the target object is a face, then the original skin color of the face is retained), and then the image position weight and the image color difference weight can be used to improve the effect of optimizing the target object on the basis of retaining part of the original color of the target object through the image color weight. Therefore, if the first fusion weight is obtained based on any one of the above-mentioned image color weights, image position weights, and image color difference weights, the image color weight can be preferred to obtain the first fusion weight. That is, it can be understood that the image color weight can be the most important weight among the image color weights, image position weights, and image color difference weights.

[0174] Optionally, if the first fusion weight is obtained based on the image color weight, the image processing device may filter the image color weight (such as performing guided filtering) to obtain the first fusion weight, so as to avoid the presence of sudden elements in the first fusion weight (such as the sudden appearance of a particularly large weight or a particularly small weight, etc.), so that the subsequent fusion between the format-converted image and the content-enhanced image is smoother and more natural. If the first fusion weight is obtained based on either the image position weight or the image color difference weight, the principle of obtaining the first fusion weight based on either weight may also be the same as the principle of obtaining the first fusion weight through the image color weight here.

[0175] Alternatively, the first fusion weight may be obtained by fusing any two of the image color weight, the image position weight, and the image color difference weight (such as performing a matrix dot multiplication operation). Similarly, since the image color weight is more important, if the first fusion weight is obtained by fusing any two of the image color weight, the image position weight, and the image color difference weight, it may be preferred to perform a fusion process (such as a dot multiplication operation) on the image color weight and the image position weight to obtain the first fusion weight, or it may be preferred to perform a fusion process (such as a dot multiplication operation) on the image color weight and the image color difference weight to obtain the first fusion weight.

[0176] Optionally, after fusing any two of the image color weights, image position weights, and image color difference weights (such as dot multiplication), an initial fusion weight can be obtained first, and then the image processing device can filter the initial fusion weight (such as guided filtering) to obtain a first fusion weight. Similarly, this can avoid sudden changes in the first fusion weight (such as the sudden appearance of a particularly large weight or a particularly small weight, etc.), making the subsequent fusion between the format-converted image and the content-enhanced image smoother and more natural.

[0177] Alternatively, the first fusion weight may be obtained by fusing the three weights of image color weight, image position weight and image color difference weight together (such as matrix dot multiplication operation). Since the first fusion weight obtained by fusing the image color weight, image position weight and image color difference weight together is the weight obtained after multi-dimensional consideration of the format transformation image, the first fusion weight obtained in this way is usually more accurate, and the first fusion weight can usually be obtained in this way. If it is to reduce the amount of calculation or based on other considerations, the above method (such as the method of obtaining the first fusion weight through any one or two of the image color weight, image position weight and image color difference weight) can be adopted to obtain the first fusion weight.

[0178] Optionally, after fusing the three weights of image color weight, image position weight and image color difference weight together (such as dot multiplication operation), an initial fusion weight can be obtained first, and then the image processing device can filter the initial fusion weight (such as guided filtering) to obtain the first fusion weight. Similarly, this can avoid the existence of sudden elements in the first fusion weight (such as the sudden appearance of a particularly large weight or a particularly small weight, etc.), so that the subsequent fusion between the format-converted image and the content-enhanced image is smoother and more natural.

[0179] Among them, since guided filtering is a filtering method that preserves edges (i.e., retains edges), in the first fused weight obtained after guided filtering of the above-mentioned weights (such as any one of the image color weights, image position weights, and image color difference weights, or the weights after fusion processing of at least two of the image color weights, image position weights, and image color difference weights), for the filtered weight, its edge information (i.e., the weight information at the edge) will not be lost, thereby ensuring the integrity of the first fused weight, so that the dimension of the first fused weight obtained after filtering is the same as the dimension of the format transformed image. Optionally, the above-mentioned guided filtering can also be replaced by other filtering that can achieve the same effect, and there is no limitation on this.

[0180] The first fusion weight obtained above can also be expressed as a matrix (such as a two-dimensional matrix), the dimension of the first fusion weight can be the same as the dimension of the format transformation image, an element (i.e., a value) in the first fusion weight can be called a weight, each element (i.e., each weight) in the first fusion weight can correspond to each pixel in the format transformation image one by one, and a pixel in the format transformation image can have a weight in the first fusion weight, such as the weight of any pixel in the format transformation image in the first fusion weight can be the weight in the first fusion weight that has the same position as the pixel. For example, for the first fusion weight, the weight of the pixel at the 1st row and the 1st column in the format transformation image can be the weight at the 1st row and the 1st column in the first fusion weight (i.e., the element at the 1st row and the 1st column); and the weight of the pixel at the 2nd row and the 2nd column in the format transformation image can be the weight at the 2nd row and the 2nd column in the first fusion weight (i.e., the element at the 2nd row and the 2nd column), and so on.

[0181] Furthermore, the image processing device can fuse the format-converted image and the content-enhanced image using the first fusion weight to obtain an optimized image of the target image, as described below.

[0182] First, the image processing device can also obtain the fusion weight of the second object image through the first fusion weight, and the fusion weight of the second object image can be called the second fusion weight. The second object image is a local area image of the position of the target object in the content enhanced image, and the size of the second object image (such as image dimension) is the same as the size of the format transformation image. The first object image, the second object image and the format transformation image can all correspond to the image in the same position area (such as the area of ​​the above-mentioned expansion detection frame) in the target image, that is, the first object image, the second object image and the format transformation image can all be local area images of the same size of the target object.

[0183] Optionally, the image processing device may multiply each target weight in the first fusion weight by a fusion coefficient (also referred to as a fusion parameter) to obtain a target fusion weight (also referred to as a weight matrix), such as the element at the first row and first column of the target fusion weight is obtained by multiplying the element at the first row and first column of the first fusion weight by the fusion coefficient; and, the element at the second row and second column of the target fusion weight is obtained by multiplying the element at the second row and second column of the first fusion weight by the fusion coefficient, and so on.

[0184] Optionally, the above-mentioned fusion coefficient is an adjustable parameter, and the adjustable numerical range of the fusion coefficient can be [0, 1], which can be adjusted according to actual needs. By adjusting the fusion coefficient, the retention intensity (i.e., the degree of retention) of the original creative intent of the target object (such as the original features of the target object in the target image (such as the original color features or content features)) can be adjusted. The larger the fusion coefficient, the greater the retention intensity of the original creative intent of the target object. Conversely, the smaller the fusion coefficient, the smaller the retention intensity of the original creative intent of the target object.

[0185] Therefore, there can also be a one-to-one correspondence between each pixel point in the format transformation image and each element (which can be called weight) in the target fusion weight. For example, the weight of any pixel point in the format transformation image in the target fusion weight can be the weight in the target fusion weight that has the same position as the pixel point. For example, for the target fusion weight, the weight of the pixel point at the 1st row and the 1st column in the format transformation image can be the weight at the 1st row and the 1st column in the target fusion weight (i.e., the element at the 1st row and the 1st column); and the weight of the pixel point at the 2nd row and the 2nd column in the format transformation image can be the weight at the 2nd row and the 2nd column in the target fusion weight (i.e., the element at the 2nd row and the 2nd column).

[0186] Furthermore, the image processing device can use a constant 1 (which can be called a fusion weight adjustment parameter or a fusion ratio adjustment parameter) to subtract each element (i.e., each weight) in the target fusion weight to obtain a second fusion weight. For example, the element at the first row and first column in the second fusion weight (which can also be called a weight) is the element obtained by subtracting the element at the first row and first column of the target fusion weight from the constant 1 (i.e., subtracting the element value of the element at the first row and first column of the target fusion weight); the element at the second row and second column in the second fusion weight is the element obtained by subtracting the element at the second row and second column of the target fusion weight from the constant 1, and so on.

[0187] Therefore, the second fusion weight can also be expressed as a matrix (such as a two-dimensional matrix), the dimension of the second fusion weight can be the same as the dimension of the second object image, an element in the second fusion weight (i.e., a numerical value) can also be called a weight, and each element in the second fusion weight (i.e., each weight) can also correspond to each pixel in the second object image one-to-one, such as the weight of any pixel in the second object image in the second fusion weight can be the weight in the second fusion weight that has the same position as the any pixel. For example, for the second fusion weight, the weight of the pixel at the 1st row and 1st column in the second object image can be the weight at the 1st row and 1st column in the second fusion weight (i.e., the element at the 1st row and 1st column); and the weight of the pixel at the 2nd row and 2nd column in the second object image can be the weight at the 2nd row and 2nd column in the second fusion weight (i.e., the element at the 2nd row and 2nd column), and so on.

[0188] Furthermore, the image processing device may perform fusion processing on the format transformed image and the second object image by using the target fusion weight and the second fusion weight to obtain a fused object image, including:

[0189] It can be understood that by subtracting the weights in the target fusion weights by the above-mentioned constant 1 (i.e., the fusion weight adjustment parameter), the weights in the second fusion weights can be negatively correlated with the weights in the target fusion weights. The larger the weights in the target fusion weights, the smaller the weights in the second fusion weights. Therefore, the fusion ratio between the format-transformed image and the second object image can be adjusted by the second fusion weights and the target fusion weights. The larger the target fusion weight, the larger the fusion ratio of the format-transformed image, and the smaller the fusion ratio of the second object image. Conversely, the larger the second fusion weight, the larger the fusion ratio of the second object image, and the smaller the fusion ratio of the format-transformed image, as described below.

[0190] First, the image processing device may perform weighted processing on the format-converted image based on the target fusion weight obtained by the first fusion weight, so as to obtain a weighted format-converted image.

[0191] For example, the image processing device may multiply (i.e., weight) the pixel value of each pixel point in the format conversion image by the weight corresponding to each pixel point in the target fusion weight, so as to obtain the weighted format conversion image. For example, the pixel value of the pixel point at the first row and the first column in the weighted format conversion image may be obtained by multiplying the pixel value of the pixel point at the first row and the first column in the format conversion image by the element (which may be referred to as the weight) at the first row and the first column in the target fusion weight.

[0192] Similarly, the image processing device may also perform weighted processing on the second object image based on the second fusion weight to obtain a weighted second object image.

[0193] For example, the image processing device may multiply (i.e., weight) the pixel value of each pixel in the second object image by the weight corresponding to each pixel in the second fusion weight to obtain a weighted second object image. For example, the pixel value of the pixel at the first row and first column in the weighted second object image may be obtained by multiplying the pixel value of the pixel at the first row and first column in the second object image by the element at the first row and first column in the second fusion weight.

[0194] The dimension of the weighted format conversion image obtained above is the same as the dimension of the weighted second object image. The image processing device can perform sum processing on the weighted format conversion image and the weighted second object fusion to obtain a fused object image.

[0195] For example, the image processing device can add the pixel value of each pixel in the weighted format transformation image with the pixel value of the pixel at the same position in the weighted second object image, so as to obtain the fused object image. The pixel value of the pixel at the 1st row and 1st column in the fused object image can be obtained by adding the pixel value of the pixel at the 1st row and 1st column in the weighted format transformation image and the pixel value of the pixel at the 1st row and 1st column in the weighted second object image.

[0196] Furthermore, the image processing device can combine the fused object image and the image in the content enhanced image except the second object image to obtain an optimized image of the target image, that is, the optimized image is composed of the fused object image and the image in the content enhanced image except the second object image. It can be understood that the optimized image is obtained by replacing the second object image in the content enhanced image with the fused object image.

[0197] See also Figure 5 , Figure 5 This is a schematic diagram of a scene for image fusion processing provided by this application. Figure 5 As shown, the target object may include a first object image, and a format-converted image may be obtained by performing format conversion processing on the first object image.

[0198] The image processing device enhances the image content of the target image to obtain a content-enhanced image, which may include a second object image. It can be understood that the second object image is an image obtained by enhancing the image content of the first object image.

[0199] Furthermore, the image processing device can perform image fusion processing on the format-converted image and the second object image by using the first fusion weight to obtain a fused object image. Then, the image processing device can replace the second object image in the content-enhanced image with the fused object image to obtain an optimized image of the target image.

[0200] By using the target fusion weight of the format conversion image and the second fusion weight of the second object image, a balance can be achieved between the format conversion image and the second object image when the format conversion image and the second object image are fused.

[0201] It can be understood that the first fusion weight can be used to determine the fusion ratio of the format transformation image to the second object image when the format transformation image is fused with the second object image (i.e., the degree of retention of the format transformation image). The larger the first fusion weight, the greater the fusion ratio and the degree of retention of the format transformation image in the above-mentioned fusion object image. Conversely, the smaller the first fusion weight, the smaller the fusion ratio and the degree of retention of the format transformation image in the above-mentioned fusion object image.

[0202] Similarly, the second fusion weight can be used to determine the fusion ratio of the second object image when the second object image is fused with the format-transformed image (i.e., the degree of retention of the second object image). The larger the second fusion weight, the greater the fusion ratio and the degree of retention of the second object image in the above-mentioned fused object image. Conversely, the smaller the second fusion weight, the smaller the fusion ratio and the degree of retention of the second object image in the above-mentioned fused object image.

[0203] Alternatively, in another feasible implementation manner, the second object image and the format-converted image may not be fused separately, as described below.

[0204] Optionally, the dimension of the format transformation image may be padded with 0 pixels to the same dimension as the target image (i.e., the same dimension as the content enhanced image) to obtain a padded format transformation image, in which all images except the format transformation image obtained after format transformation of the first object image may be composed of 0 pixels.

[0205] In addition, similarly, the size (i.e., dimension) of the above-mentioned target fusion weight can be padded to the same size as the padded format transformation image (i.e., the same size as the target image) through 0 elements (i.e., 0 weights) to obtain the padded target fusion weight, so that the corresponding weights of the 0 pixels filled in the padded format transformation image in the padded target fusion weight are also 0.

[0206] When the dimensions of the format-transformed image and the dimensions of the target fusion weight are filled to be the same as the dimensions of the target image, there is no need to obtain the second object image in the content-enhanced image, nor is there any need to perform additional fusion processing on the second object image and the format-transformed image. In this case, each element (i.e., each weight) in the filled target fusion weight can be directly subtracted from the constant 1 to obtain the fusion weight of the content-enhanced image (which can be correspondingly referred to as the filled second fusion weight). The filled second fusion weight is the same as the dimension of the content-enhanced image, and each pixel in the content-enhanced image corresponds one-to-one to the weight at the same position in the filled second fusion weight. In this case, the corresponding weights of the images other than the second object image in the content-enhanced image (i.e., the pixels of the images other than the second object image) in the filled second fusion weight are all 1.

[0207] Furthermore, the padded format transformation image and the padded target fusion weight can be directly multiplied at corresponding positions (i.e., the same positions) (such as multiplying the pixel value of the pixel at the 1st row and 1st column in the padded format transformation image with the element at the 1st row and 1st column in the padded target fusion weight, and so on), and an image obtained by weighting the padded format transformation image (which can be called a weighted padded format transformation image) can be obtained, and the dimension of the weighted padded format transformation image is the same as that of the target image.

[0208] Similarly, the padded second fusion weight in this case can be directly multiplied with the corresponding position (i.e., the same position) of the content-enhanced image (such as multiplying the pixel value of the pixel at the 1st row and 1st column in the content-enhanced image with the element at the 1st row and 1st column in the padded second fusion weight, and so on), to obtain a weighted content-enhanced image, the dimension of which is the same as that of the target image.

[0209] Furthermore, the image processing device can directly add the weighted filling format transformation image here and the weighted content enhancement image at the corresponding position (i.e. the same position) (such as adding the pixel value of the pixel point at the 1st row and 1st column in the weighted filling format transformation image to the pixel value of the pixel point at the 1st row and 1st column in the weighted content enhancement image, and so on), and directly obtain the optimized image of the target image. This method is used to fuse the format transformation image and the content enhancement image. Since there is no need to perform additional fusion processing on the second object image and the format transformation image, the convenience and efficiency of the fusion processing of the format transformation image and the content enhancement image can be improved. As shown in the following formula:

[0210] I HDR =W f ×k×HF +(1-W f ×k)×H C (3)

[0211] Among them, I HDR represents the optimized image, k represents the fusion coefficient, W f It can be expressed as filling the first fusion weight with 0 weight to a weight with the same dimension as the target image (which can be called the first fusion weight after filling), k×W f That is, the target fusion weight after filling, H F Represents the format transformed image after padding. 1-W f ×k means that the second fusion weight after filling is obtained by subtracting each weight in the filled target fusion weight from the constant 1. C That is, the content-enhanced image.

[0212] It can be understood that the optimized images finally obtained by the above two methods (including the method of zero filling or the method of not filling with zero) are the same. The specific method used to fuse the format conversion image and the content enhancement image can be selected according to the actual application scenario, and there is no restriction on this.

[0213] The present application can also be applied to scenarios where video optimization is required: the target image can be a video frame containing a target object in a target video, the target video can be any video that needs to be optimized, there can be one or more video frames containing a target object in the target video, that is, there can be one or more target images, and each target image can be optimized according to the process described above to obtain an optimized image of each target image.

[0214] As for the video frames that do not contain the target objects in the target video, the image content of each video frame that does not contain the target object can be enhanced based on the above-mentioned method of enhancing the image content of the target image to obtain optimized images of each video frame that does not contain the target object. That is, for the video frame that does not contain the target object, the image content of the video frame can be directly enhanced to obtain the optimized image of the video frame.

[0215] Through the above process, the optimized image of each video frame in the target video (which can be called optimized video frame) can be obtained. The optimized video of the target video can be constructed by the optimized video frames of each video frame in the target video. The optimized video contains the optimized video frames of each video frame in the target video.

[0216] The image processing device can transmit the optimized video to the video client so that the video client can play the optimized video. That is, the application can also improve the optimization effect of the target video, thereby improving the viewing effect of the target video.

[0217] See also Figure 6 , Figure 6 This is a schematic diagram of an architecture for optimizing images provided by this application. Figure 6 As shown, the SDR image here is the target image, and the target object may be a human face. The image processing device may perform face detection on the SDR image to obtain a first object image at the position of the target object in the target image.

[0218] The image processing device can calculate the image color weight (i.e., the above-mentioned image color weight) through the first object image, and can calculate the image position weight through the size of the first object image (such as width and height, or the size of the format transformation image obtained through the first object image, the size of the first object image is the same as the size of the format transformation image).

[0219] The image processing device can also perform format up-conversion (i.e., format up-conversion) on the first object image to obtain a format-converted image, and can enhance the image content of the SDR image here (i.e., content up-conversion) to obtain a content-enhanced image.

[0220] Next, the image processing device can calculate the image color difference weight through the format conversion image and the content enhancement image. Furthermore, the image processing device can perform a fusion process (such as a dot multiplication process) and then a filtering process (such as a guided filtering process) on the image color weight, image position weight and image color difference weight obtained above, and obtain the first fusion weight of the format conversion image.

[0221] Furthermore, the image processing device can use the first fusion weight to perform image fusion processing on the format-converted image and the content-enhanced image, so as to obtain an optimized image of the SDR image here (that is, the HDR image here).

[0222] Based on the above description, the present application proposes an inverse tone mapping method for retaining the original creative intent for the target object in the SDR image (such as image retention through format conversion) in the optimized HDR image (such as the optimized image of the target image), effectively maintaining the original creative intent for the target object in the SDR image, and performing spatial smoothing through three weights, namely, image color weight (i.e., image color weight), image position weight, and image color difference weight, effectively solving the problems of over-enhancement and visual distortion of the target object when optimizing the SDR image to the HDR image, and improving the optimization effect of optimizing the SDR image to the HDR image.

[0223] The present application can firstly obtain a target image containing a target object; the target image is an image of a first dynamic range, the target image contains a first object image, and the first object image refers to a local area image of the position of the target object in the target image; then, the image content of the target image can be enhanced to obtain a content-enhanced image; and the image format of the first object image can be transformed to obtain a format-transformed image; the format-transformed image is an image of a second dynamic range, the second dynamic range is greater than the first dynamic range, and the image content of the format-transformed image is the same as the image content of the first object image; further, the format-transformed image and the content-enhanced image can be fused to obtain an optimized image of the target image; the optimized image is an image of the second dynamic range. It can be seen that the method proposed in the present application can perform overall content enhancement processing on the target image to obtain a content enhanced image, and can additionally perform format conversion processing on the local area image (i.e., the first object image) where the target object is located in the target image, and the image content of the first object image is kept unchanged during the format conversion processing to obtain a format converted image, and then, the content enhanced image and the format converted image can be fused to obtain an optimized image of the target image, so that in the obtained optimized image, some original features of the target object in the target image are retained through the format conversion image, and the target object in the target image will not be over-enhanced, and the image content of the target image can be enhanced as a whole through the content enhanced image, thereby achieving a better optimization processing of the target image, thereby obtaining a better optimized image of the target image.

[0224] See also Figure 7 , Figure 7 This is a flowchart of a method for generating image color weights provided by this application. Figure 7 , the method may include:

[0225] Step S201: obtaining a reference pixel of a target object in a target image, and using the pixel color of the reference pixel as the reference color of the target object.

[0226] Optionally, the image processing device can obtain a reference pixel point of the target object in the target image (i.e., a pixel point that can characterize the color of the target object), and can use the pixel color of the reference pixel point (which can be the H component (i.e., hue) of the reference pixel point) as the reference color of the target object.

[0227] For example, if the target object is a face, the reference pixel point may be a pixel point on the face of the target image, the number of the reference pixel points may be one or more, and a reference pixel point has a reference color of the target object, which is not limited. As follows, the process of obtaining the reference pixel point of the target object may include:

[0228] In the above-mentioned process of performing object detection on the target image to obtain the object detection frame of the target object in the target image, the position of the key points of the target object in the target image can also be synchronously detected (such as the above-mentioned trained detection model can also be used to detect the position of the key points of the target object in the image). For example, the key points of the target object can include the key points of the five senses of the face, and the key points of the five senses can include the key points of the left eye, the right eye, the left corner of the mouth and the right corner of the mouth of the face. Then, the position of the key points of the target object can include the position of the left eye key point, the right eye key point, the left corner of the mouth key point and the right corner of the mouth key point of the face.

[0229] Furthermore, the image processing device can obtain the reference pixel of the target object through the position of the key point of the target object: for example, the image processing device can connect the left eye key point and the left corner of the mouth key point of the face in the target image, and take the pixel at the midpoint of the line in the target image as a reference pixel of the target object, and the reference pixel is a reference pixel taken on the left side of the face. For example, the coordinates of the left eye key point and the left corner of the mouth key point can be added and then divided by 2 to obtain the coordinates of the midpoint of the line connecting the left eye key point and the left corner of the mouth key point, that is, the coordinates of the reference pixel on the left face.

[0230] Similarly, the image processing device can connect the right eye key point and the right corner of mouth key point of the face in the target image, and take the pixel point at the midpoint of the line in the target image as a reference pixel point of the target object, and the reference pixel point is a reference pixel point taken on the right face of the face. For example, the coordinates of the right eye key point and the right corner of mouth key point can be added and then divided by 2 to obtain the coordinates of the midpoint of the line connecting the right eye key point and the right corner of mouth key point, that is, the coordinates of the reference pixel point on the right face.

[0231] Through the above process, a reference pixel point is obtained on the left and right sides of the face respectively. In this way, even if there is a slight difference between the colors of the left and right sides of the face, the colors of the left and right sides of the face can be considered at the same time, so as to consider the color of the entire face. Alternatively, as needed, reference pixels at more positions (such as the forehead, chin, etc.) can be obtained on the target object (such as the face) in the target image, and there is no restriction on this.

[0232] Step S202: obtaining a target color distance between the pixel color of each pixel point in the first object image and a reference color of the target object.

[0233] Optionally, the image processing device may also obtain the pixel color of each pixel in the first object image, and the pixel color of each pixel in the first object image may be the H component (ie, hue) of each pixel.

[0234] Optionally, the process of the image processing device obtaining the H component of each pixel point in the first object image may include: first, the image processing device may perform filtering processing on the first object image (such as Gaussian filtering or other filtering methods) to filter out image noise in the first object image to obtain a filtered object image.

[0235] Furthermore, the image processing device can convert the filtering object image into the HSV color space (a color space) to obtain a color space conversion image. In the HSV color space, each pixel in the color space conversion image has an H component (hue component), an S component (saturation component), and a V component (brightness component). The image processing device can use the H component of each pixel in the color space conversion image as the H component of each pixel at the same position in the first object image. For example, the H component of the pixel at the 1st row and 1st column in the color space conversion image can be used as the H component of the pixel at the 1st row and 1st column in the first object image, and so on.

[0236] Optionally, it can be understood that the reference pixel point of the above-mentioned target object also belongs to the pixel point in the first object image, and the H component of the reference pixel point of the target object can also be the H component of the reference pixel point in the first object image obtained here (that is, the H component of the pixel point at the same position corresponding to the reference pixel point in the color space conversion image).

[0237] The image processing device can obtain the target color distance between the pixel color of each pixel in the first object image and the reference color of the target object. The target color distance can be used to reflect the difference between the pixel color of the pixel in the first object image and the reference color. The larger the target color distance, the greater the difference.

[0238] Any pixel in the first object image may be referred to as a target pixel. Here, a process of obtaining a target color distance between a pixel color of the target pixel and a reference color of the target object is taken as an example for explanation, as described below.

[0239] The image processing device can obtain the color distance between the pixel color of the target pixel and the reference color of each reference pixel. The color distance between the target pixel and the reference color of a reference pixel can be the absolute value of the difference between the numerical value of the hue of the target pixel and the numerical value of the hue of the reference pixel (that is, the color distance is not negative).

[0240] Furthermore, the image processing device can use the color distance with the smallest value (i.e. considering the minimum color difference) among the color distances between the pixel color of the target pixel point and the reference color of each reference pixel point as the target color distance between the pixel color of the target pixel point and the reference color of the target object.

[0241] Through the above principle, the image processing device can obtain the target color distance between the pixel color of each pixel in the first object image and the reference color of the target object. The target color distance between the pixel color of a pixel and the reference color of the target object can be called the target color distance corresponding to the pixel.

[0242] Step S203 : generating an image color weight based on a target color distance between a pixel color of each pixel point in the first object image and a reference color of the target object.

[0243] Optionally, the image processing device may obtain a color weight mapping function, where the color weight mapping function may be a pre-set appropriate function for generating color weights.

[0244] The image processing device can use the color weight mapping function to map the target color distance between each pixel in the first object image and the reference color of the target object, thereby generating the color weight of each pixel in the first object image.

[0245] It can be understood that when the color weight mapping function is used to map the target color distance to the corresponding color weight, the smaller the target color distance, the larger the mapped color weight, and conversely, the larger the target color distance, the smaller the mapped color weight.

[0246] This is because, if the target color distance between the pixel color of a pixel and the reference color is smaller, it means that the color difference between the pixel color of the pixel and the reference color is smaller, and the probability that the pixel is the pixel of the target object in the first object image (such as a face pixel) is greater. The greater the probability of being the pixel of the target object, the greater the color weight of the pixel can be, so that the color of the pixel can be retained to a greater extent in the future. As shown in the following formula, the color weight mapping function can be (but not limited to, and can also be other suitable functions) the following function:

[0247] e (-γ×d) (4)

[0248] Among them, e is a natural constant (a transcendental number), γ represents an adjustable parameter, the larger the γ is, the smaller the mapped color weight is, and the smaller the fusion strength brought by the subsequent color weight to the format conversion image is. γ can be adjusted according to the actual application scenario, and the recommended value can be 2. d represents the position where the target color distance can be substituted. For example, the target color distance between the pixel color of the target pixel and the reference color can be substituted into d in the color weight mapping function to obtain the color weight of the target pixel.

[0249] Among them, the dimension of the first object image is the same as that of the format transformation image, and there can be a one-to-one correspondence between the pixels with the same pixel position in the first object image and the format transformation image. The color weight of any pixel in the first object image generated above is the color weight of the pixel corresponding to the pixel in the format transformation image.

[0250] For example, the color weight of the pixel at the 1st row and 1st column in the first object image can be the color weight of the pixel at the 1st row and 1st column in the format transformation image, the color weight of the pixel at the 2nd row and 2nd column in the first object image can be the color weight of the pixel at the 2nd row and 2nd column in the format transformation image, and so on.

[0251] The image color weight can be obtained by transforming the color weight of each pixel in the format transformed image. The image color weight includes the color weight of each pixel in the format transformed image.

[0252] See also Figure 8 , Figure 8 This is a schematic diagram of a scenario for generating image color weights provided by this application. Figure 8As shown, it is assumed that the first object image includes pixel points s1 to s9, pixel point s1 corresponds to target color distance j1, pixel point s2 corresponds to target color distance j2, pixel point s3 corresponds to target color distance j3, pixel point s4 corresponds to target color distance j4, pixel point s5 corresponds to target color distance j5, pixel point s6 corresponds to target color distance j6, pixel point s7 corresponds to target color distance j7, pixel point s8 corresponds to target color distance j8, and pixel point s9 corresponds to target color distance j9.

[0253] The image processing device can use a color weight mapping function to map the target color distance j1 to obtain the color weight q1 of the pixel s1, use a color weight mapping function to map the target color distance j2 to obtain the color weight q2 of the pixel s2, use a color weight mapping function to map the target color distance j3 to obtain the color weight q3 of the pixel s3, use a color weight mapping function to map the target color distance j4 to obtain the color weight q4 of the pixel s4, use a color weight mapping function to map the target color distance j5 to obtain the color weight q5 of the pixel s5, use a color weight mapping function to map the target color distance j6 to obtain the color weight q6 of the pixel s6, use a color weight mapping function to map the target color distance j7 to obtain the color weight q7 of the pixel s7, use a color weight mapping function to map the target color distance j8 to obtain the color weight q8 of the pixel s8, and use a color weight mapping function to map the target color distance j9 to obtain the color weight q9 of the pixel s9.

[0254] The image color weight can be obtained by using the above color weights q1 to q9, and the image color weight can be expressed as a weight matrix.

[0255] By adopting the above-mentioned image color weights obtained in the present application, the original color features of the target object in the target image can be retained to a greater extent in the subsequent process, thereby improving the optimization effect of the target object in the target image.

[0256] See also Fig. 9 , Fig. 9 This is a flowchart of a method for generating image position weights provided by this application. Fig. 9 , the method may include:

[0257] Step S301 : acquiring a first image length of a format-converted image in a first direction, and generating a first Gaussian distribution based on the first image length.

[0258] Optionally, the image processing device may obtain the image length of the format conversion image in the first direction, and the image length of the format conversion image in the first direction may be referred to as the first image length. The first direction may be a transverse direction (such as a horizontal direction), and the first image length may be the width of the format conversion image. The first image length may be equal to the number of pixels in the format conversion image in the first direction, that is, may be equal to the number of pixels in the format conversion image in the transverse direction (such as the total number of columns of pixels in the format conversion image).

[0259] Each pixel in the format conversion image may have a corresponding serial number value in the first direction. The serial number value of a pixel in the first direction may be the value of the column number of the pixel minus 1, and the column number starts from the 1st column. For example, if a pixel is a pixel in the 1st column of the format conversion image, the serial number value of the pixel in the first direction may be 0. If a pixel is a pixel in the 5th column of the format conversion image, the serial number value of the pixel in the first direction may be 4, that is, the serial number value of each pixel in the first direction may start from 0 (the serial number value of each pixel is discrete). The serial number value of each pixel in the first direction reflects the position of each pixel in the first direction of the format conversion image.

[0260] That is to say, the serial number value of the first pixel point in the first direction (or the first column of pixels) in the format conversion image in the first direction is 0, the serial number value of the second pixel point in the first direction (or the second column of pixels) in the format conversion image in the first direction is 1, and so on.

[0261] The image processing device may generate a Gaussian distribution for optimizing the format-converted image from a first direction by using the first image length. The Gaussian distribution in the first direction may be referred to as a first Gaussian distribution. The process of generating the first Gaussian distribution may include:

[0262] First, the image processing device can generate a first initial Gaussian distribution through the first image length. The mean (i.e., expected value) of the first initial Gaussian distribution can be called the first Gaussian mean. The first Gaussian mean can be half the value of the first image length, that is, the first Gaussian mean can be equal to half (i.e., 1 / 2) of the first image length. The first initial Gaussian distribution can have an independent variable and a dependent variable. The independent variable of the first initial Gaussian distribution can be called the first independent variable, and the dependent variable of the first initial Gaussian distribution can be called the first dependent variable. The value range of the first independent variable can include the serial number values ​​of each pixel point in the first direction in the format transformation image. For example, the value range of the first independent variable can be [0, W R -1],W Ris the length of the first image, 0 can represent the serial number value of the first pixel point of the format conversion image in the first direction, W R -1 may represent the sequence number value of the last pixel point of the format conversion image in the first direction.

[0263] By setting the first Gaussian mean to half the length of the first image, the center position of the first initial Gaussian distribution can be located in the middle of the width of the format transformation image, that is, when the first independent variable takes the serial number values ​​of each pixel point in the first direction of the format transformation image, the serial number values ​​of each pixel point in the first direction can be symmetrically distributed on both sides of the center position of the first initial Gaussian distribution.

[0264] Among them, the first initial Gaussian distribution also has a standard deviation. Optionally, the standard deviation can also be equal to half of the length of the first image, or the standard deviation can also be adjusted to other values. The larger the standard deviation is set, the smoother (i.e., less intense) the function value of the first initial Gaussian distribution can change.

[0265] Furthermore, the image processing device may perform a normalization process (such as a normalization process) on the first initial Gaussian distribution to obtain a first Gaussian distribution. The process may include:

[0266] The image processing device can obtain a reference value of the first independent variable, which can be referred to as a first reference value. The first reference value can be used to set a spatial range for smooth transition fusion of the format conversion image in the first direction. The first reference value is not equal to the first Gaussian mean value. The first reference value can be set according to the actual application scenario. For example, the first reference value can be βW R , W R is the length of the first image, β is an adjustable parameter, and the adjustable range of β can be [0, 1 / 2], which is used to adjust the range of the spatial transition area in the first direction and the transition severity of the transition area. The larger β is, the larger the range of the spatial transition area is, the smaller the transition severity is, and the smoother the transition is. For example, β can be 1 / 6.

[0267] Furthermore, the image processing device can obtain the value of the first dependent variable when the first independent variable takes the first reference value (which can be called the first variable value), that is, the first initial Gaussian distribution can be a one-dimensional Gaussian function, and the first variable value can be the function value (that is, the value of the first dependent variable) obtained after the first dependent variable takes the first reference value.

[0268] The image processing device can normalize the value of the first dependent variable within the first value range in the first initial Gaussian distribution to a first constant value, and the first constant value can be equal to 1. The image processing device can also normalize the value of the first dependent variable within the second value range in the first initial Gaussian distribution to a first target dependent variable, thereby obtaining the first Gaussian distribution.

[0269] Among them, the first value range is the range in which the first dependent variable takes values ​​greater than or equal to the first variable value in the first initial Gaussian distribution (i.e., the tip of the middle part of the first initial Gaussian distribution, i.e., the peak), and the second value range is the range in which the first dependent variable takes values ​​less than the first variable value in the first initial Gaussian distribution (i.e., the curve parts on both sides of the first initial Gaussian distribution except the tip of the middle part).

[0270] Optionally, the first target dependent variable may be obtained by the first dependent variable and the first variable value, such as the first target dependent variable may be the ratio between the first dependent variable and the first variable value (such as the dependent variable obtained by dividing the first dependent variable by the first variable value).

[0271] The first Gaussian distribution obtained through the above process can be that the function value in the middle range (i.e., the first value range) is 1, and the function value on both sides of the range (i.e., the second value range) is the value of the first target dependent variable. The closer the function value on both sides of the range is to the boundary (i.e., the edge), the smaller it is, and then gradually approaches 0.

[0272] Therefore, the first Gaussian distribution can be used to make the position weight of the pixel points in the middle position of the format conversion image in the first direction larger (such as equal to 1), and the position weight of the pixel points closer to the edge position on both sides of the format conversion image in the first direction can be smaller (such as a smooth transition change (i.e., transition reduction) from 1 to 0), so that the format conversion image can be smoothly fused in the first direction, and there will be no obvious boundary between the target object or the fused object image in the optimized image of the target image finally optimized and the other parts of the image in the optimized image in the first direction, and a smooth transition of the local area image where the target object is located is achieved (i.e., the edge part image of the local area image where the target object is located (such as the format conversion image) is transitionally optimized). As shown in the following formula:

[0273]

[0274] Among them, x is the first independent variable of the first initial Gaussian distribution, G wn (x) is the first Gaussian distribution (also with the first independent variable), G w (x) is the first initial Gaussian distribution, βW R is the first reference value, Gw (βW R ) is to set x to βW R The value of the first dependent variable (i.e. G w (x), that is, the value of the first variable mentioned above), G w (βW R ) is a point in the first initial Gaussian distribution, W R is the length of the first image. G w (x) <G w (βW R ) is the second value range mentioned above, G w (x)≥G w (βW R ) is the first value range mentioned above.

[0275] See also Fig.10 , Fig.10 is a curve diagram of a first initial Gaussian distribution provided by the present application. Fig.10 As shown, when the first independent variable takes the value When , the first dependent variable of the first initial Gaussian distribution takes the maximum value (i.e., the peak value), and the peak value is at the center of the first initial Gaussian distribution.

[0276] The value of the first independent variable is βW R ~(1-β)W R When the value of the first dependent variable is within the range, the value of the first dependent variable is greater than or equal to the value of the first variable, that is, the first value range is the value of the first dependent variable For example, if β is equal to 1 / 6, then βW R W R / 6, (1-β)W R 5W R / 6.

[0277] Furthermore, the value of the first independent variable is 0~βW R and (1-β)W R ~W R -1, the value of the first dependent variable is less than the value of the first variable, that is, the second value range is the value of the first dependent variable is less than G w (βW R )’s scope.

[0278] Step S302 : acquiring a second image length of the format-converted image in a second direction, and generating a second Gaussian distribution based on the second image length.

[0279] Optionally, the principle of generating the second Gaussian distribution is the same as the principle of generating the first Gaussian distribution, except that the direction considered is different. The second Gaussian distribution considers the optimization of the format conversion image in the second direction.

[0280] Similarly, the image processing device can obtain the image length of the format-converted image in the second direction, and the image length of the format-converted image in the second direction can be referred to as the second image length. The second direction can be the longitudinal direction (such as the vertical direction), and the second image length can be the height of the format-converted image. The second image length can be equal to the number of pixels in the second direction in the format-converted image, that is, it can be equal to the number of pixels in the longitudinal direction in the format-converted image (such as the total number of rows of pixels in the format-converted image).

[0281] Each pixel in the format conversion image may also have a corresponding serial number value in the second direction. The serial number value of a pixel in the second direction may be the value of the row number of the pixel minus 1, and the row number starts from the 1st row. For example, if a pixel is a pixel in the 1st row of the format conversion image, the serial number value of the pixel in the second direction may be 0. If a pixel is a pixel in the 5th row of the format conversion image, the serial number value of the pixel in the second direction may be 4, that is, the serial number value of each pixel in the second direction may start from 0 (the serial number value of each pixel is discrete). The serial number value of each pixel in the second direction reflects the position of each pixel in the second direction of the format conversion image.

[0282] That is to say, the serial number value of the first pixel point in the second direction (or the first row of pixels) in the format transformed image in the second direction is 0, the serial number value of the second pixel point in the second direction (or the second row of pixels) in the format transformed image in the first direction is 1, and so on.

[0283] The image processing device may generate a Gaussian distribution for optimizing the format-converted image in a second direction by using the second image length. The Gaussian distribution in the second direction may be referred to as a second Gaussian distribution. The process of generating the second Gaussian distribution may include:

[0284] First, the image processing device can generate a second initial Gaussian distribution through the second image length. The mean (i.e., expected value) of the second initial Gaussian distribution can be called a second Gaussian mean. The second Gaussian mean can be half the value of the second image length, i.e., the second Gaussian mean can be equal to half (i.e., 1 / 2) of the second image length. The second initial Gaussian distribution can have an independent variable and a dependent variable. The independent variable of the second initial Gaussian distribution can be called a second independent variable. The dependent variable of the second initial Gaussian distribution can be called a second dependent variable. The value range of the second independent variable can include the serial number values ​​of each pixel point in the second direction in the format transformation image. For example, the value range of the second independent variable can be [0, H R -1], H R is the length of the second image, 0 is the serial number of the first pixel point in the second direction of the format conversion image, H R -1 is the serial number value of the last pixel point of the format conversion image in the second direction.

[0285] By setting the second Gaussian mean to half the length of the second image, the center position of the second initial Gaussian distribution can be located in the middle of the height of the format transformation image, that is, when the second independent variable takes the serial number values ​​of each pixel point in the second direction of the format transformation image, the serial number values ​​of each pixel point in the second direction can be symmetrically distributed on both sides of the center position of the second initial Gaussian distribution.

[0286] The second initial Gaussian distribution also has a standard deviation. Optionally, the standard deviation may be equal to half the length of the second image, or the standard deviation may be adjusted to other values. The larger the standard deviation is set, the smoother (i.e., less intense) the function value of the second initial Gaussian distribution may change.

[0287] Furthermore, the image processing device may perform a normalization process (such as a normalization process) on the second initial Gaussian distribution to obtain a second Gaussian distribution. The process may include:

[0288] The image processing device can obtain a reference value of the second independent variable, which can be referred to as a second reference value. The second reference value can also be used to set the spatial range for smooth transition fusion of the format conversion image in the second direction. The second reference value is not equal to the second Gaussian mean value. The second reference value can be set according to the actual application scenario. For example, the second reference value can be βH R , H Ris the length of the second image, β is the above-mentioned adjustable parameter, and the adjustable range of β can be [0, 1 / 2], which is used to adjust the range of the spatial transition area in the second direction and the transition severity of the transition area. The larger β is, the larger the range of the spatial transition area is, the smaller the transition severity is, and the smoother the transition is. For example, β can be 1 / 6.

[0289] Furthermore, the image processing device can obtain the value of the second dependent variable when the second independent variable takes the second reference value (which can be called the second variable value), that is, the second initial Gaussian distribution can also be a one-dimensional Gaussian function, and the second variable value can be the function value (that is, the value of the second dependent variable) obtained after the second dependent variable takes the second reference value.

[0290] The image processing device can normalize the values ​​of the second dependent variable within the third value range in the second initial Gaussian distribution to a second constant value, and the second constant value can be equal to 1. The image processing device can also normalize the values ​​of the second dependent variable within the fourth value range in the second initial Gaussian distribution to a second target dependent variable, thereby obtaining a second Gaussian distribution.

[0291] Among them, the third value range is the range in which the second dependent variable takes values ​​greater than or equal to the second variable value in the second initial Gaussian distribution (i.e., the tip of the middle part of the second initial Gaussian distribution, i.e., the peak), and the fourth value range is the range in which the second dependent variable takes values ​​less than the second variable value in the second initial Gaussian distribution (i.e., the curve parts on both sides of the second initial Gaussian distribution except the tip of the middle part).

[0292] Optionally, the second target dependent variable may be obtained by the second dependent variable and the second variable value, such as the second target dependent variable may be the ratio between the second dependent variable and the second variable value (such as the dependent variable obtained by dividing the second dependent variable by the second variable value).

[0293] The second Gaussian distribution obtained through the above process can also be that the function value in the middle range (i.e., the third value range) is 1, and the function value on both sides (i.e., the fourth value range) is the value of the second target dependent variable. The closer the function value on both sides is to the boundary (i.e., the edge), the smaller it is, and then gradually approaches 0.

[0294] Therefore, the second Gaussian distribution can be used to make the position weight of the pixel points in the middle position of the format conversion image in the second direction larger (such as equal to 1), and the position weight of the pixel points in the format conversion image that are closer to the edge position on both sides in the second direction can be smaller (such as smoothly transitioning from 1 (i.e., transitional reduction) to 0), so that the format conversion image can be smoothly fused in the second direction, and there will be no obvious boundary between the target object or the fused object image in the optimized image of the target image finally optimized and the other parts of the image in the optimized image in the second direction, so as to achieve a smooth transition of the local area image where the target object is located (i.e., the edge part image of the local area image where the target object is located is transitionally optimized). As shown in the following formula:

[0295]

[0296] Among them, y is the second independent variable of the second initial Gaussian distribution, G hn (y) is the second Gaussian distribution (also with a second independent variable), G h (y) is the second initial Gaussian distribution, βH R is the second reference value, G h (βH R ) means that y is set to βH R The value of the second dependent variable (i.e. G h (y), that is, the value of the second variable mentioned above), G h (βH R ) is a point in the second initial Gaussian distribution, H R is the length of the second image. h (y) <G h (βH R ) is the fourth value range mentioned above, G h (y)≥G h (βH R ) is the third value range mentioned above.

[0297] See also Fig.11 , Fig.11 is a curve diagram of a second initial Gaussian distribution provided by the present application. Fig.11 As shown, when the second independent variable takes the value When , the second dependent variable of the second initial Gaussian distribution takes the maximum value (i.e., the peak value), and the peak value is at the center of the second initial Gaussian distribution.

[0298] The value of the second independent variable is βH R ~(1-β)H RWhen the value of the second dependent variable is within the range, the value of the second dependent variable is greater than or equal to the value of the second variable, that is, the value of the first dependent variable in the third value range is For example, if β is equal to 1 / 6, then βH R H R / 6, (1-β)H R 5H R / 6.

[0299] Furthermore, the value of the second independent variable is 0~βH R and (1-β)H R ~H R -1, the value of the second dependent variable is less than the value of the second variable, that is, the fourth value range is the value of the second dependent variable is less than G h (βH R )’s scope.

[0300] Step S303: Generate image position weights based on the first Gaussian distribution and the second Gaussian distribution.

[0301] Optionally, it can be known from the above process that any pixel point in the format transformation image has a serial number value in the first direction and a serial number value in the second direction, the value range of the first independent variable may include the serial number values ​​of each pixel point in the first direction in the format transformation image, and the value range of the second independent variable may include the serial number values ​​of each pixel point in the second direction in the format transformation image. The following describes the process of generating image position weights through the first Gaussian distribution and the second Gaussian distribution:

[0302] The image processing device may multiply the first Gaussian distribution and the second Gaussian distribution to obtain a target Gaussian distribution, and the target Gaussian distribution may be a two-dimensional Gaussian distribution, as shown in the following formula:

[0303] W P (x,y)=G wn (x)×G hn (y) (7)

[0304] Among them, W P (x, y) is the target Gaussian distribution, G wn (x) is the first Gaussian distribution, G hn (y) is the second Gaussian distribution.

[0305] Since the first Gaussian distribution has a first independent variable and the second Gaussian distribution has a second independent variable, the target Gaussian distribution may have both the first independent variable and the second independent variable. The dependent variable of the target Gaussian distribution may be the product of the dependent variable of the first Gaussian distribution and the dependent variable of the second Gaussian distribution.

[0306] The image processing device can take the first independent variable in the target Gaussian distribution as the serial value of each pixel point in the format transformation image in the first direction, and can take the second independent variable in the target Gaussian distribution as the serial value of each pixel point in the format transformation image in the second direction, so as to obtain the position weight of each pixel point in the format transformation image.

[0307] For example, the image processing device can take the first independent variable in the target Gaussian distribution as the serial value of any pixel point in the format transformation image in the first direction, and can take the second independent variable in the target Gaussian distribution as the serial value of any pixel point in the second direction. The function value of the target Gaussian distribution obtained in this way (which can be understood as the value of the dependent variable of the target Gaussian distribution at this time) can be the position weight of any pixel point.

[0308] Optionally, the first Gaussian distribution in the first direction and the second Gaussian distribution in the second direction in the present application can also be designed as other distribution functions that can achieve the same effect, so that the designed distribution function in the first direction (similar to the first Gaussian distribution) can achieve the effect that the position weight of the pixel points in the middle position of the first direction in the format transformation image is large, and the position weight of the pixel points closer to the edge decreases smoothly (such as the position weight of the pixel points at the extreme edge eventually decreases to 0), and the designed distribution function in the second direction (similar to the second Gaussian distribution) can achieve the effect that the position weight of the pixel points in the middle position of the second direction in the format transformation image is large, and the position weight of the pixel points closer to the edge also decreases smoothly (such as the position weight of the pixel points at the extreme edge eventually decreases to 0).

[0309] Furthermore, the position weight of each pixel in the format transformed image constitutes the image position weight, that is, the image position weight includes the position weight of each pixel in the format transformed image.

[0310] By using the above-mentioned image position weights generated by the present application, the position weights of the pixels located at the middle position in the format transformation image can be larger (such as equal to 1), and the position weights of the pixels located at the edge position in the format transformation image can be gradually reduced to approach 0 (such as gradually reduced from 1 to 0), so that the image features located at the middle position in the format transformation image (such as image color) can be retained more, while the image features located at the edge position in the format transformation image can be retained less, so that the image features located at the edge position in the format transformation image can be subsequently fused with more image features at the corresponding edge position in the second object image (that is, more image features in the second object image are fused at the edge position), so that the image at the edge position of this part of the image of the fused object image in the optimized image can be more naturally and smoothly transitioned with the images of other parts of the optimized image (such as the image other than the second object image in the content enhanced image), that is, the fused object image or the target object will not have a visual effect of having a boundary in the optimized image.

[0311] See also Fig.12 , Fig.12 This is a flowchart of a method for generating image color difference weights provided by this application. Fig.12 , the method may include:

[0312] Step S401 , obtaining a second object image in the content-enhanced image; the second object image refers to a local area image of the location of the target object in the content-enhanced image.

[0313] Optionally, the image processing device may acquire a second object image, which is a local area image of the position of the target object in the content enhanced image. The dimension of the target image may be the same as that of the content enhanced image, the dimension of the first object image may be the same as that of the second object image, and the position of the first object image in the target image may be the same as that of the second object image in the content enhanced image.

[0314] Step S402, performing difference calculation on the pixel values ​​of the pixel points at the same pixel position in the format conversion image and the second object image, and obtaining the target pixel difference value between each pixel point in the format conversion image and the pixel point at the same pixel position in the second object image.

[0315] Optionally, both the format conversion image and the second object image may be RGB images, and the pixels in the format conversion image may have pixel values ​​on three color channels, namely, the R channel, the G channel, and the B channel, and the second object image may also have pixel values ​​on three color channels, namely, the R channel, the G channel, and the B channel (i.e., the components on these three color channels).

[0316] The image processing device can perform corresponding subtraction (i.e., difference) on the pixel values ​​of the pixel points in the format-converted image on each color channel and the pixel values ​​of the pixel points at the same position in the second object image on each color channel, and thus obtain the difference between the pixel values ​​of the pixel points in the format-converted image on each color channel and the pixel values ​​of the pixel points at the same position in the second object image on each color channel (which can be called pixel difference, and the pixel difference can be used to reflect the color difference between the pixels).

[0317] Among them, the processing principle for the pixel values ​​of the pixel points on the above-mentioned each color channel (i.e., the difference processing) can be the same and performed separately and independently. Any pixel point in the format transformation image has a pixel difference value on each color channel with the pixel point at the same pixel position corresponding to the any pixel point in the second object image. Furthermore, the maximum value of the three pixel differences between the any pixel point on the three color channels and the pixel point at the same pixel position in the second image (i.e., considering the maximum color difference) can be used as the target pixel difference value between the any pixel point and the pixel point at the same pixel position in the second object image.

[0318] For example, the pixel value on the R channel of the pixel point s1 at the 1st row and 1st column in the format transformation image can be subtracted from the pixel value on the R channel of the pixel point s2 at the 1st row and 1st column in the second object image, and then the absolute value can be taken to obtain the pixel difference c1 between the pixel value on the R channel of the pixel point s1 and the pixel value on the R channel of the pixel point s2. The pixel difference c1 can be used to reflect the color difference between the pixel point s1 and the pixel point s2 on the R channel.

[0319] Similarly, the pixel value on the G channel of the pixel point s1 at the 1st row and 1st column in the format transformation image can be subtracted from the pixel value on the G channel of the pixel point s2 at the 1st row and 1st column in the second object image, and then the absolute value can be taken to obtain the pixel difference c2 between the pixel value on the G channel of the pixel point s1 and the pixel value on the G channel of the pixel point s2. The pixel difference c2 can be used to reflect the color difference between the pixel point s1 and the pixel point s2 on the G channel.

[0320] Similarly, the pixel value of the pixel point s1 at the 1st row and 1st column in the format conversion image on the B channel can be subtracted from the pixel value of the pixel point s2 at the 1st row and 1st column in the second object image on the B channel, and then the absolute value can be taken to obtain the pixel difference c3 between the pixel value of the pixel point s1 on the B channel and the pixel value of the pixel point s2 on the B channel. The pixel difference c3 can be used to reflect the color difference between the pixel point s1 and the pixel point s2 on the B channel.

[0321] Furthermore, the pixel difference with the largest value among the above-mentioned pixel difference c1, pixel difference c2 and pixel difference c3 can be used as the target pixel difference between pixel point s1 and pixel point s2. The target pixel difference takes into account the maximum pixel color difference between pixel point s1 and pixel point s2. The target pixel difference can be called the target pixel difference corresponding to pixel point s1 in the format transformation image.

[0322] Step S403 : using a color difference weight mapping function, mapping processing is performed on the target pixel difference value corresponding to each pixel point in the format conversion image, so as to generate a color difference weight of each pixel point in the format conversion image.

[0323] Optionally, the image processing device may obtain a color difference weight mapping function, where the color difference weight mapping function may be a preset appropriate function for generating color difference weights.

[0324] The image processing device can use the color difference weight mapping function to perform mapping processing on the target pixel difference value corresponding to each pixel point in the format transformation image, so as to generate the color difference weight of each pixel point in the format transformation image.

[0325] It can be understood that when the color difference weight mapping function is used to map the target pixel difference value to the corresponding color difference weight, the smaller the target pixel difference value is, the smaller the color difference between the pixel point to which the target pixel difference value belongs in the format transformation image and the pixel point at the same position in the second object image is, and therefore, the mapped color difference weight can be smaller; conversely, the larger the target pixel difference value is, the larger the color difference between the pixel point to which the target pixel difference value belongs in the format transformation image and the pixel point at the same position in the second object image is, and therefore, the mapped color difference weight can be larger.

[0326] This is because, if the color difference between two pixels is smaller, the color difference between the two pixels can be adjusted by a smaller color difference weight. Conversely, if the color difference between the two pixels is larger, a larger color difference weight is needed to adjust the color difference between the two pixels, so that a balance can be achieved between the color of the pixels in the subsequent format-transformed image (the color reflected by the pixel values ​​in the color channel) and the color of the pixels in the second object image (the color reflected by the pixel values ​​in the color channel), so as to reduce the overall image color difference between the format-transformed image and the second object image, so that the fused object image obtained by fusing the format-transformed image and the second object image and other images in the content-enhanced image except the second object image can achieve overall consistency, so that the effect (such as visual effect) between the original features in the format-transformed image and the content-enhanced features in the second object image obtained by the optimization of the fused object image is more balanced, so that the optimized image of the target image finally obtained also has overall visual consistency.

[0327] As shown in the following formula, the color difference weight mapping function f(z) may be (but not limited to, and may also be other suitable functions) the following function:

[0328]

[0329] The color difference weight mapping function designed above is obtained by translating the sigmoid function (a function of an S-shaped curve) downward by 0.5 and then magnifying the vertical axis by 2 times. In actual application scenarios, other color difference weight mapping functions (such as sine function or linear function, etc.) that can achieve the same effect can also be designed. e is a natural constant (a transcendental number), and z is the position where the target pixel difference can be substituted. For example, the target pixel difference corresponding to any pixel point in the format transformation image can be substituted into z in the color difference weight mapping function to obtain the color difference weight of any pixel point.

[0330] The image color difference weight can be obtained by using the color difference weight of each pixel in the format transformation image. The image color difference weight includes the color difference weight of each pixel in the format transformation image.

[0331] See also Fig.13 , Fig.13 This is a schematic diagram of a scene for generating image color difference weights provided by this application. Fig.13As shown, it is assumed that the format conversion image includes pixel points s11 to s99, pixel point s11 corresponds to the target pixel difference value c1, pixel point s22 corresponds to the target pixel difference value c2, pixel point s33 corresponds to the target pixel difference value c3, pixel point s44 corresponds to the target pixel difference value c4, pixel point s55 corresponds to the target pixel difference value c5, pixel point s66 corresponds to the target pixel difference value c6, pixel point s77 corresponds to the target pixel difference value c7, pixel point s88 corresponds to the target pixel difference value c8, and pixel point s99 corresponds to the target pixel difference value c9.

[0332] The image processing device can use the color difference weight mapping function to map the target pixel difference c1 to obtain the color difference weight q11 of the pixel point s11, use the color difference weight mapping function to map the target pixel difference c2 to obtain the color difference weight q22 of the pixel point s22, use the color difference weight mapping function to map the target pixel difference c3 to obtain the color difference weight q33 of the pixel point s33, use the color difference weight mapping function to map the target pixel difference c4 to obtain the color difference weight q44 of the pixel point s44, and use the color difference weight mapping function to map the target pixel difference c5 to obtain the color difference weight q56 of the pixel point s57. c5 is mapped to obtain the color difference weight q55 of pixel s55, the color difference weight mapping function is used to map the target pixel difference c6 to obtain the color difference weight q66 of pixel s66, the color difference weight mapping function is used to map the target pixel difference c7 to obtain the color difference weight q77 of pixel s77, the color difference weight mapping function is used to map the target pixel difference c8 to obtain the color difference weight q88 of pixel s88, the color difference weight mapping function is used to map the target pixel difference c9 to obtain the color difference weight q99 of pixel s99.

[0333] The image color difference weight can be obtained by using the above color difference weights q11 to q99, and the image color difference weight can be expressed as a weight matrix.

[0334] By adopting the above-mentioned image color difference weight obtained in the present application, the optimized image of the target image obtained by subsequent optimization can have visual consistency, thereby improving the overall optimization effect and visual effect of the target image.

[0335] See also Fig.14 , Fig.14 140 is a schematic diagram of the structure of an image processing device provided by the present application. The image processing device 140 may include: an acquisition module 1401 , an enhancement module 1402 , a transformation module 1403 and a fusion module 1404 .

[0336] An acquisition module 1401 is used to acquire a target image containing a target object; the target image is an image in a first dynamic range, the target image contains a first object image, and the first object image refers to a local area image of the position of the target object in the target image;

[0337] An enhancement module 1402 is used to enhance the image content of the target image to obtain a content-enhanced image;

[0338] The transformation module 1403 is used to transform the image format of the first object image to obtain a format-transformed image; the format-transformed image is an image in a second dynamic range, the second dynamic range is greater than the first dynamic range, and the image content of the format-transformed image is the same as the image content of the first object image;

[0339] The fusion module 1404 is used to fuse the format-converted image and the content-enhanced image to obtain an optimized image of the target image; the optimized image is an image in the second dynamic range.

[0340] Optionally, the fusion module 1404 performs fusion processing on the format-converted image and the content-enhanced image to obtain an optimized image of the target image, including:

[0341] Obtaining a first fusion weight of the format transformed image;

[0342] Based on the first fusion weight, the format transformed image and the content enhanced image are fused to obtain an optimized image.

[0343] Optionally, the first fusion weight is obtained based on any one of the following weights, or the first fusion weight is obtained by fusing at least two of the following weights:

[0344] an image color weight of the format transformed image; the image color weight being obtained based on a difference between a reference color of the target object and an image color of the first object image;

[0345] An image position weight of the format transformed image; the image position weight is obtained based on the position of each pixel in the format transformed image;

[0346] The image color difference weight of the format transformed image; the image color difference weight is obtained based on the pixel difference between the format transformed image and the second object image, and the second object image refers to the local area image of the position of the target object in the content enhanced image.

[0347] Optionally, the image color of the first object image includes the pixel color of each pixel in the first object image; the process of the fusion module 1404 generating the image color weight includes:

[0348] Obtaining a reference pixel point of the target object in the target image, and using the pixel color of the reference pixel point as the reference color of the target object;

[0349] Obtaining a target color distance between a pixel color of each pixel point in the first object image and a reference color of the target object;

[0350] An image color weight is generated based on a target color distance between a pixel color of each pixel point in the first object image and a reference color of the target object.

[0351] Optionally, any pixel in the first object image is a target pixel, there are one or more reference pixels, and one reference pixel has a reference color of the target object;

[0352] The fusion module 1404 obtains the target color distance between the pixel color of each pixel point in the first object image and the reference color of the target object, including:

[0353] Obtain the color distance between the pixel color of the target pixel and the reference color of each reference pixel;

[0354] The color distance with the smallest value among the color distances between the pixel color of the target pixel and the reference colors of one or more reference pixels is used as the target color distance between the pixel color of the target pixel and the reference color of the target object.

[0355] Optionally, the fusion module 1404 generates the image color weight based on the target color distance between the pixel color of each pixel point in the first object image and the reference color of the target object, including:

[0356] Using a color weight mapping function, mapping the target color distance between each pixel in the first object image and the reference color of the target object is performed to generate a color weight for each pixel in the first object image;

[0357] The dimension of the format transformation image is the same as that of the first object image, there is a one-to-one correspondence between pixels having the same pixel position in the first object image and the format transformation image, the color weight of any pixel in the first object image is the color weight of the corresponding pixel in the format transformation image, and the image color weight includes the color weight of each pixel in the format transformation image.

[0358] Optionally, the process of generating the image position weight by the fusion module 1404 includes:

[0359] Acquire a first image length of the format-converted image in a first direction, and generate a first Gaussian distribution based on the first image length;

[0360] Acquire a second image length of the format-converted image in a second direction, and generate a second Gaussian distribution based on the second image length;

[0361] Based on the first Gaussian distribution and the second Gaussian distribution, an image position weight is generated.

[0362] Optionally, the fusion module 1404 generates the first Gaussian distribution based on the first image length, including:

[0363] Generate a first initial Gaussian distribution based on the first image length; the first initial Gaussian distribution has a first Gaussian mean, and the first Gaussian mean is half the value of the first image length;

[0364] The first initial Gaussian distribution is normalized to obtain a first Gaussian distribution.

[0365] Optionally, the first initial Gaussian distribution has a first independent variable and a first dependent variable; the fusion module 1404 normalizes the first initial Gaussian distribution to obtain the first Gaussian distribution, including:

[0366] Obtaining a first reference value of a first independent variable, and obtaining a first variable value of a first dependent variable when the value of the first independent variable is the first reference value, wherein the first reference value is not equal to a first Gaussian mean;

[0367] Normalize the values ​​of the first dependent variable within the first value range in the first initial Gaussian distribution to a first constant value, and normalize the values ​​of the first dependent variable within the second value range in the first initial Gaussian distribution to a first target dependent variable, to obtain a first Gaussian distribution;

[0368] Among them, the first value range is the range in which the first dependent variable takes values ​​greater than or equal to the first variable value in the first initial Gaussian distribution, the second value range is the range in which the first dependent variable takes values ​​less than the first variable value in the first initial Gaussian distribution, and the first target dependent variable is obtained based on the first dependent variable and the first variable value.

[0369] Optionally, the fusion module 1404 generates the second Gaussian distribution based on the second image length, including:

[0370] Generate a second initial Gaussian distribution based on the second image length; the second initial Gaussian distribution has a second Gaussian mean, and the second Gaussian mean is half the value of the second image length;

[0371] The second initial Gaussian distribution is normalized to obtain a second Gaussian distribution.

[0372] Optionally, the second initial Gaussian distribution has a second independent variable and a second dependent variable; the fusion module 1404 normalizes the second initial Gaussian distribution to obtain the second Gaussian distribution, including:

[0373] Obtaining a second reference value of the second independent variable, and obtaining a second variable value of the second dependent variable when the second independent variable is the second reference value, the second reference value is not equal to the second Gaussian mean;

[0374] Normalizing the values ​​of the second dependent variable within the third value range in the second initial Gaussian distribution to a second constant value, and normalizing the values ​​of the second dependent variable within the fourth value range in the second initial Gaussian distribution to a second target dependent variable, to obtain a second Gaussian distribution;

[0375] Among them, the third value range is the range in which the value of the second dependent variable in the second initial Gaussian distribution is greater than or equal to the value of the second variable, the fourth value range is the range in which the value of the second dependent variable in the second initial Gaussian distribution is less than the value of the second variable, and the second target dependent variable is obtained based on the second dependent variable and the second variable values.

[0376] Optionally, the first Gaussian distribution has a first independent variable, the second Gaussian distribution has a second independent variable, the value range of the first independent variable includes the serial number value of each pixel point of the format transformation image in the first direction, and the value range of the second independent variable includes the serial number value of each pixel point of the format transformation image in the second direction; any pixel point in the format transformation image has a serial number value in the first direction and a serial number value in the second direction; the fusion module generates the image position weight based on the first Gaussian distribution and the second Gaussian distribution, including:

[0377] Performing a product operation on the first Gaussian distribution and the second Gaussian distribution to obtain a target Gaussian distribution; the target Gaussian distribution has a first independent variable and a second independent variable;

[0378] The first independent variable of the target Gaussian distribution is respectively set as the serial number value of each pixel point in the format transformation image in the first direction, and the second independent variable of the target Gaussian distribution is respectively set as the serial number value of each pixel point in the format transformation image in the second direction, to obtain the position weight of each pixel point in the format transformation image;

[0379] The image position weight includes the position weight of each pixel in the format transformation image.

[0380] Optionally, the process of generating the image color difference weight by the fusion module 1404 includes:

[0381] Acquire a second object image in the content-enhanced image; the second object image refers to a local area image of the location of the target object in the content-enhanced image;

[0382] Calculate the difference between the pixel values ​​of the pixel points at the same pixel position in the format conversion image and the second object image to obtain the target pixel difference between each pixel point in the format conversion image and the pixel point at the same pixel position in the second object image;

[0383] A color difference weight mapping function is used to map the target pixel difference value corresponding to each pixel point in the format transformation image, and a color difference weight of each pixel point in the format transformation image is generated;

[0384] The image color difference weight includes the color difference weight of each pixel in the format transformation image.

[0385] Optionally, the fusion module 1404 performs fusion processing on the format transformed image and the content enhanced image based on the first fusion weight to obtain a method for optimizing the image, including:

[0386] Determine a second fusion weight of a second object image based on the first fusion weight; the second object image refers to a local area image of the location of the target object in the content enhanced image;

[0387] performing weighted processing on the format-converted image based on the first fusion weight to obtain a weighted format-converted image;

[0388] Performing weighted processing on the second object image based on the second fusion weight to obtain a weighted second object image;

[0389] Performing sum processing on the weighted format transformation image and the weighted second object image to obtain a fused object image;

[0390] The images other than the second object image in the fused object image and the content enhanced image are combined to obtain an optimized image.

[0391] Optionally, the process of the device 140 acquiring the first object image includes:

[0392] Performing object detection on the target image to obtain an object detection frame of the target object in the target image;

[0393] Expanding the object detection frame on the target image to obtain an expanded detection frame;

[0394] The image within the expanded detection frame in the target image is taken as the first object image.

[0395] Optionally, the target image belongs to a video frame in the target video that contains the target object;

[0396] The device 140 is also used for:

[0397] Generate an optimized video of the target video based on the optimized image;

[0398] The optimized video is transmitted to the video client so that the video client plays the optimized video.

[0399] According to one embodiment of the present application, Figure 3 The steps involved in the image processing method shown can be represented by Fig.14 The various modules in the image processing device 140 shown in the figure are executed. For example, Figure 3 The step S101 shown in FIG. 1 can be performed by Fig.14 The acquisition module 1401 in is executed, Figure 3 The step S102 shown in FIG. 1 can be performed by Fig.14 The enhancement module 1402 in is executed; Figure 3 The step S103 shown in FIG. 1 can be performed by Fig.14 The transformation module 1403 in is executed, Figure 3 The step S104 shown in FIG. 1 can be performed by Fig.14 The fusion module 1404 in is executed.

[0400] The present application can firstly obtain a target image containing a target object; the target image is an image of a first dynamic range, the target image contains a first object image, and the first object image refers to a local area image of the position of the target object in the target image; then, the image content of the target image can be enhanced to obtain a content-enhanced image; and the image format of the first object image can be transformed to obtain a format-transformed image; the format-transformed image is an image of a second dynamic range, the second dynamic range is greater than the first dynamic range, and the image content of the format-transformed image is the same as the image content of the first object image; further, the format-transformed image and the content-enhanced image can be fused to obtain an optimized image of the target image; the optimized image is an image of the second dynamic range. It can be seen that the device proposed in the present application can perform overall content enhancement processing on the target image to obtain a content enhanced image, and can additionally perform format conversion processing on the local area image (i.e., the first object image) where the target object is located in the target image, and the image content of the first object image is kept unchanged during the format conversion processing to obtain a format converted image, and then, the content enhanced image and the format converted image can be fused to obtain an optimized image of the target image, so that in the obtained optimized image, some original features of the target object in the target image are retained through the format conversion image, and the target object in the target image will not be over-enhanced, and the image content of the target image can be enhanced as a whole through the content enhanced image, thereby achieving a more superior optimization processing of the target image, thereby obtaining a more excellent optimized image of the target image.

[0401] According to one embodiment of the present application, Fig.14The various modules in the image processing device 140 shown can be separately or all combined into one or several units to constitute, or one (some) of the units can be further divided into multiple smaller sub-units in function, and the same operation can be achieved without affecting the realization of the technical effects of the embodiments of the present application. The above modules are divided based on logical functions. In practical applications, the functions of a module can also be implemented by multiple units, or the functions of multiple modules can be implemented by one unit. In other embodiments of the present application, the image processing device 140 may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0402] According to one embodiment of the present application, the program can be executed by running on a general-purpose computer device such as a computer including a central processing unit (CPU), a random access memory medium (RAM), a read-only memory medium (ROM), and other processing elements and storage elements. Figure 3 The computer program of each step involved in the corresponding method shown in is constructed as follows Fig.14 The image processing apparatus 140 shown in the figure can be used to implement the image processing method of the embodiment of the present application. The above computer program can be recorded on a computer-readable recording medium, for example, and loaded into the above computing device through the computer-readable recording medium and run therein.

[0403] See also Fig.15 , Fig.15 Schematic diagram of the structure of a computer device provided by this application. Fig.15 As shown, the computer device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, in some embodiments, the computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1005 may optionally be at least one storage device located away from the aforementioned processor 1001. As Fig.15 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.

[0404] exist Fig.15 In the computer device 1000 shown, the network interface 1004 can provide a network communication function; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0405] Acquire a target image containing a target object; the target image is an image of a first dynamic range, the target image contains a first object image, and the first object image refers to a local area image of the position of the target object in the target image;

[0406] Performing image content enhancement processing on the target image to obtain a content enhanced image;

[0407] Performing a format conversion process on the image format of the first object image to obtain a format conversion image; the format conversion image is an image in a second dynamic range, the second dynamic range is greater than the first dynamic range, and the image content of the format conversion image is the same as the image content of the first object image;

[0408] The format-converted image and the content-enhanced image are fused to obtain an optimized image of the target image; the optimized image is an image in the second dynamic range.

[0409] It should be understood that the computer device 1000 described in the embodiment of the present application can execute the above Figure 3 The description of the above-mentioned image processing method in the corresponding embodiment can also be performed Fig.14 The description of the image processing device 140 in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated here either.

[0410] In addition, it should be pointed out here that: the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program, and when the processor executes the computer program, it can execute the above Figure 3 The description of the image processing method in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated. For technical details not disclosed in the computer storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0411] As an example, the above-mentioned computer program may be deployed on one computer device for execution, or deployed on multiple computer devices located at one location for execution, or executed on multiple computer devices distributed at multiple locations and interconnected by a communication network. Multiple computer devices distributed at multiple locations and interconnected by a communication network may constitute a blockchain network.

[0412] The computer-readable storage medium may be an internal storage unit of the computer device, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Further, the computer-readable storage medium may also include both an internal storage unit and an external storage device of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0413] The present application provides a computer program product, which includes a computer program, which is stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the above-mentioned Figure 3 The description of the above-mentioned image processing method in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0414] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, device, product, or equipment that includes a series of steps or units is not limited to the listed steps or modules, but optionally includes steps or modules that are not listed, or optionally includes other step units inherent to these processes, methods, devices, products, or equipment.

[0415] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0416] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a target image containing a target object; the target image is an image of a first dynamic range, the target image contains a first object image, and the first object image refers to a local area image of the position of the target object in the target image; Performing image content enhancement processing on the target image to obtain a content enhanced image; Performing a format conversion process on the image format of the first object image to obtain a format conversion image; the format conversion image is an image in a second dynamic range, the second dynamic range is greater than the first dynamic range, and the image content of the format conversion image is the same as the image content of the first object image; The format-converted image and the content-enhanced image are fused to obtain an optimized image of the target image; the optimized image is an image in the second dynamic range.

2. The method according to claim 1, characterized in that The fusing process of the format-converted image and the content-enhanced image to obtain the optimized image of the target image includes: Obtaining a first fusion weight of the format transformed image; Based on the first fusion weight, the format-converted image and the content-enhanced image are fused to obtain the optimized image.

3. The method according to claim 2, characterized in that The first fusion weight is obtained based on any one of the following weights, or the first fusion weight is obtained by fusing at least two of the following weights: an image color weight of the format transformed image; the image color weight being obtained based on a difference between a reference color of the target object and an image color of the first object image; The image position weight of the format transformed image; the image position weight is obtained based on the position of each pixel in the format transformed image; The image color difference weight of the format transformed image; the image color difference weight is obtained based on the pixel difference between the format transformed image and a second object image, wherein the second object image refers to a local area image of the position of the target object in the content enhanced image.

4. The method according to claim 3, characterized in that The image color of the first object image includes the pixel color of each pixel in the first object image; The process of generating the image color weight includes: Acquire a reference pixel of the target object in the target image, and use the pixel color of the reference pixel as the reference color of the target object; Obtaining a target color distance between a pixel color of each pixel point in the first object image and a reference color of the target object; The image color weight is generated based on a target color distance between a pixel color of each pixel point in the first object image and a reference color of the target object.

5. The method according to claim 4, characterized in that Any pixel point in the first object image is a target pixel point, there are one or more reference pixels points, and one reference pixel point has a reference color of the target object; The obtaining of the target color distance between the pixel color of each pixel point in the first object image and the reference color of the target object includes: Obtaining the color distance between the pixel color of the target pixel and the reference color of each reference pixel; The color distance with the smallest value among the color distances between the pixel color of the target pixel and the reference colors of one or more reference pixels is used as the target color distance between the pixel color of the target pixel and the reference color of the target object.

6. The method according to claim 4, characterized in that The step of generating the image color weight based on the target color distance between the pixel color of each pixel point in the first object image and the reference color of the target object comprises: Using a color weight mapping function, mapping the target color distance between each pixel in the first object image and the reference color of the target object is performed to generate a color weight for each pixel in the first object image; The dimension of the format transformation image is the same as that of the first object image, there is a one-to-one correspondence between pixels having the same pixel position in the first object image and the format transformation image, the color weight of any pixel in the first object image is the color weight of the corresponding pixel in the format transformation image, and the image color weight includes the color weight of each pixel in the format transformation image.

7. The method according to claim 3, characterized in that The process of generating the image position weight includes: Acquire a first image length of the format-converted image in a first direction, and generate a first Gaussian distribution based on the first image length; acquiring a second image length of the format-converted image in a second direction, and generating a second Gaussian distribution based on the second image length; The image position weight is generated based on the first Gaussian distribution and the second Gaussian distribution.

8. The method according to claim 7, characterized in that The generating a first Gaussian distribution based on the first image length comprises: Generate a first initial Gaussian distribution based on the first image length; the first initial Gaussian distribution has a first Gaussian mean, and the first Gaussian mean is half the value of the first image length; The first initial Gaussian distribution is normalized to obtain the first Gaussian distribution.

9. The method according to claim 8, characterized in that The first initial Gaussian distribution has a first independent variable and a first dependent variable; the first initial Gaussian distribution is normalized to obtain the first Gaussian distribution, including: Obtaining a first reference value of the first independent variable, and obtaining a first variable value of the first dependent variable when the first independent variable takes the first reference value, wherein the first reference value is not equal to the first Gaussian mean; Normalizing the values ​​of the first dependent variable within a first value range in the first initial Gaussian distribution to a first constant value, and normalizing the values ​​of the first dependent variable within a second value range in the first initial Gaussian distribution to a first target dependent variable, to obtain the first Gaussian distribution; Among them, the first value range is the range in which the first dependent variable takes values ​​greater than or equal to the first variable value in the first initial Gaussian distribution, the second value range is the range in which the first dependent variable takes values ​​less than the first variable value in the first initial Gaussian distribution, and the first target dependent variable is obtained based on the first dependent variable and the first variable values.

10. The method according to claim 7, characterized in that The generating a second Gaussian distribution based on the second image length comprises: generating a second initial Gaussian distribution based on the second image length; the second initial Gaussian distribution has a second Gaussian mean, and the second Gaussian mean is half the value of the second image length; The second initial Gaussian distribution is normalized to obtain the second Gaussian distribution.

11. The method according to claim 10, characterized in that The second initial Gaussian distribution has a second independent variable and a second dependent variable; the normalizing the second initial Gaussian distribution to obtain the second Gaussian distribution includes: Obtaining a second reference value of the second independent variable, and obtaining a second variable value of the second dependent variable when the second independent variable takes the second reference value, wherein the second reference value is not equal to the second Gaussian mean; Normalizing the values ​​of the second dependent variable within the third value range in the second initial Gaussian distribution to a second constant value, and normalizing the values ​​of the second dependent variable within the fourth value range in the second initial Gaussian distribution to a second target dependent variable, to obtain the second Gaussian distribution; Among them, the third value range is the range in which the value of the second dependent variable in the second initial Gaussian distribution is greater than or equal to the value of the second variable, the fourth value range is the range in which the value of the second dependent variable in the second initial Gaussian distribution is less than the value of the second variable, and the second target dependent variable is obtained based on the values ​​of the second dependent variable and the second variable.

12. The method according to claim 7, characterized in that The first Gaussian distribution has a first independent variable, the second Gaussian distribution has a second independent variable, the value range of the first independent variable includes the serial number value of each pixel point of the format transformation image in the first direction, and the value range of the second independent variable includes the serial number value of each pixel point of the format transformation image in the second direction; any pixel point in the format transformation image has a serial number value in the first direction and a serial number value in the second direction; the generating the image position weight based on the first Gaussian distribution and the second Gaussian distribution includes: Performing a product operation on the first Gaussian distribution and the second Gaussian distribution to obtain a target Gaussian distribution; the target Gaussian distribution has the first independent variable and the second independent variable; The first independent variable of the target Gaussian distribution is respectively set as the serial number value of each pixel point in the format transformation image in the first direction, and the second independent variable of the target Gaussian distribution is respectively set as the serial number value of each pixel point in the format transformation image in the second direction, to obtain the position weight of each pixel point in the format transformation image; The image position weight includes the position weight of each pixel in the format transformation image.

13. The method according to claim 3, characterized in that The process of generating the image color difference weight includes: Acquire a second object image in the content-enhanced image; the second object image refers to a local area image of the position where the target object is located in the content-enhanced image; Performing a difference calculation on the pixel values ​​of the pixel points at the same pixel position in the format conversion image and the second object image to obtain a target pixel difference value between each pixel point in the format conversion image and the pixel point at the same pixel position in the second object image; Using a color difference weight mapping function, mapping processing is performed on the target pixel difference value corresponding to each pixel point in the format transformation image to generate a color difference weight for each pixel point in the format transformation image; The image color difference weight includes the color difference weight of each pixel in the format conversion image.

14. The method according to claim 2, characterized in that The fusing the format-converted image and the content-enhanced image based on the first fusion weight to obtain the optimized image includes: Determine a second fusion weight of a second object image based on the first fusion weight; the second object image refers to a local area image of the position where the target object is located in the content enhanced image; performing weighted processing on the format-converted image based on the first fusion weight to obtain a weighted format-converted image; Performing weighted processing on the second object image based on the second fusion weight to obtain a weighted second object image; Performing sum processing on the weighted format transformed image and the weighted second object image to obtain a fused object image; The fused object image and the image in the content enhanced image except the second object image are combined to obtain the optimized image.

15. The method according to claim 1, characterized in that The process of acquiring the first object image includes: Performing object detection on the target image to obtain an object detection frame of the target object in the target image; Expanding the object detection frame on the target image to obtain an expanded detection frame; The image within the expanded detection frame in the target image is used as the first object image.

16. The method according to claim 1, characterized in that The target image belongs to a video frame of a target video that contains the target object; The method further comprises: Based on the optimized image, generating an optimized video of the target video; The optimized video is transmitted to a video client, so that the video client plays the optimized video.

17. An image processing device, characterized in that: The device comprises: An acquisition module, used for acquiring a target image containing a target object; the target image is an image of a first dynamic range, the target image contains a first object image, and the first object image refers to a local area image of the position of the target object in the target image; An enhancement module, used for enhancing the image content of the target image to obtain a content-enhanced image; a transformation module, configured to transform the image format of the first object image to obtain a format-transformed image; the format-transformed image is an image in a second dynamic range, the second dynamic range is greater than the first dynamic range, and the image content of the format-transformed image is the same as the image content of the first object image; A fusion module is used to fuse the format-converted image and the content-enhanced image to obtain an optimized image of the target image; the optimized image is an image in the second dynamic range.

18. A computer program product, comprising a computer program, which implements the steps of the method according to any one of claims 1 to 16 when executed by a processor.

19. A computer device, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 16.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the method according to any one of claims 1 to 16.

Citation Information

Patent Citations

  • High dynamic range image combining

    CN102257531A

  • Image processing method and device, storage medium and electronic equipment

    CN109741280A

  • Lossy compression coding method, device and system-on-chip

    CN110536138A

  • Image processing method and device based on artificial intelligence, medium and electronic equipment

    CN110796593A

  • Image processing method and device, electronic equipment and computer readable storage medium

    CN111325657A

Cited By

  • Image processing method and device, electronic equipment, storage medium and program product

    CN119693251A

  • Image Processing Method, Apparatus, Electronic Device, Storage Medium, and Program Product

    CN119693251B

  • Method and device for defining imaging dynamic range of sound source signal

    CN120340522A