Image processing method, apparatus, product, device, and medium

Abstract

The application discloses an image processing method, device, product, equipment and medium, and the method comprises the following steps: acquiring a target image containing a target object; the target image is an image of a first dynamic range, and the target image contains a first object image, which is a local area image of the target object in the target image; performing enhancement processing on the image content of the target image to obtain a content enhanced image; performing transformation processing on the image format of the first object image to obtain a format transformed image; the format transformed image is an image of a second dynamic range, the second dynamic range is greater than the first dynamic range, and the image content of the format transformed image is the same as that of the first object image; performing fusion processing on the format transformed image and the content enhanced image to obtain an optimized image of the target image; and the optimized image is an image of the second dynamic range. By using the application, the optimization effect of the target image can be improved, and a more excellent optimized image of the target image can be obtained.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more particularly to an image processing method, apparatus, product, device, and medium. Background Technology

[0002] Image optimization is extremely important in the field of image processing. Image optimization can include optimizing images with low dynamic range to images with high dynamic range, which is the range of light intensity that can be achieved in an image.

[0003] Among them, the low dynamic range image to be optimized may include face images. In existing applications, when optimizing the low dynamic range image into a high dynamic range image, the low dynamic range image is subjected to uniform content enhancement processing across the entire image. Since the visual effect of face images is very prominent, when performing content enhancement processing on the low dynamic range image, the face images in the low dynamic range image are often over-enhanced, resulting in poor optimization effect of the face images in the low dynamic range image, which in turn leads to poor visual effect of the optimized face image. Summary of the Invention

[0004] This application provides an image processing method, apparatus, product, device, and medium that can improve the optimization effect on a target image, thereby obtaining a superior optimized image of the target image.

[0005] This application provides an image processing method, which includes:

[0006] Obtain a target image containing the target object; the target image is an image with a first dynamic range, and the target image contains a first object image, which refers to a local region image of the location of the target object in the target image;

[0007] The image content of the target image is enhanced to obtain a content-enhanced image;

[0008] The image format of the first object image is transformed to obtain a format-transformed image; the format-transformed image is an image with a second dynamic range, which is greater than the first dynamic range, and the image content of the format-transformed image is the same as the image content of the first object image.

[0009] The format-transformed image and the content-enhanced image are fused to obtain an optimized image of the target image; the optimized image is an image with the second dynamic range.

[0010] This application provides an image processing apparatus, which includes:

[0011] The acquisition module is used to acquire a target image containing a target object; the target image is an image with a first dynamic range, and the target image contains a first object image, which refers to a local region image of the location of the target object in the target image;

[0012] The enhancement module is used to enhance the image content of the target image to obtain a content-enhanced image;

[0013] The transformation module is used to transform the image format of the first object image to obtain a format-transformed image; the format-transformed image is an image with a second dynamic range, which is greater than the first dynamic range, and the image content of the format-transformed image is the same as the image content of the first object image.

[0014] The fusion module is used to fuse format-transformed images and content-enhanced images to obtain an optimized image of the target image; the optimized image is an image with the second dynamic range.

[0015] Optionally, the fusion module can perform fusion processing on the format-transformed image and the content-enhanced image to obtain an optimized image of the target image in the following ways:

[0016] Obtain the first fusion weight of the format-transformed image;

[0017] Based on the first fusion weight, the format-transformed image and the content-enhanced image are fused to obtain an optimized image.

[0018] Optionally, the first fusion weight is obtained based on any of the following weights, or the first fusion weight is obtained by fusing at least two of the following weights:

[0019] Image color weighting for format-transformed images; image color weighting is obtained based on the difference between the reference color of the target object and the image color of the first object image;

[0020] Image position weights for format-transformed images; image position weights are obtained based on the position of each pixel in the format-transformed image.

[0021] Image color difference weighting for format-transformed images; image color difference weighting is obtained based on the pixel difference between the format-transformed image and the second object image, which refers to the local region image of the target object in the content-enhanced image.

[0022] Optionally, the image color of the first object image includes the pixel color of each pixel in the first object image; the process of the fusion module generating image color weights includes:

[0023] Obtain the reference pixel of the target object in the target image, and use the pixel color of the reference pixel as the reference color of the target object;

[0024] Obtain the target color distance between the pixel color of each pixel in the first object image and the reference color of the target object;

[0025] Image color weights are generated based on the target color distance between the pixel color of each pixel in the first object image and the reference color of the target object.

[0026] Optionally, any pixel in the first object image is the target pixel, and there are one or more reference pixels, each of which has a reference color of the target object.

[0027] The fusion module obtains the target color distance between the pixel color of each pixel in the first object image and the reference color of the target object using the following methods:

[0028] Obtain the color distance between the pixel color of the target pixel and the reference color of each reference pixel;

[0029] The smallest color distance among the color distances between the target pixel's pixel color and the reference colors of one or more reference pixels is taken as the target color distance between the target pixel's pixel color and the reference color of the target object.

[0030] Optionally, the fusion module generates image color weights based on the target color distance between the pixel color of each pixel in the first object image and the reference color of the target object, including:

[0031] A color weight mapping function is used to map the target color distance between each pixel in the first object image and the reference color of the target object, thereby generating the color weight of each pixel in the first object image.

[0032] The format-transformed image has the same dimension as the first object image. There is a one-to-one correspondence between pixels in the first object image and the format-transformed image that have the same pixel position. The color weight of any pixel in the first object image is the color weight of the corresponding pixel in the format-transformed image. The image color weight includes the color weight of each pixel in the format-transformed image.

[0033] Optionally, the process of generating image location weights by the fusion module includes:

[0034] Obtain the first image length of the format-transformed image in the first direction, and generate a first Gaussian distribution based on the first image length;

[0035] Obtain the second image length of the format-transformed image in the second direction, and generate a second Gaussian distribution based on the second image length;

[0036] Image position weights are generated based on the first and second Gaussian distributions.

[0037] Optionally, the fusion module generates a first Gaussian distribution based on the first image length in the following ways:

[0038] A first initial Gaussian distribution is generated based on the first image length; the first initial Gaussian distribution has a first Gaussian mean, which is half the length of the first image.

[0039] Normalize the initial Gaussian distribution to obtain the first Gaussian distribution.

[0040] Optionally, the first initial Gaussian distribution has a first independent variable and a first dependent variable; the fusion module normalizes the first initial Gaussian distribution to obtain the first Gaussian distribution in the following ways:

[0041] Obtain the first reference value of the first independent variable, and obtain the first variable value of the first dependent variable when the first independent variable takes the first reference value. The first reference value is not equal to the first Gaussian mean.

[0042] The values ​​of the first dependent variable within the first range of the first initial Gaussian distribution are normalized to the first constant value, and the values ​​of the first dependent variable within the second range of the first initial Gaussian distribution are normalized to the first target dependent variable, thus obtaining the first Gaussian distribution;

[0043] The first value range is the range in which the value of the first dependent variable in the first initial Gaussian distribution is greater than or equal to the value of the first variable; the second value range is the range in which the value of the first dependent variable in the first initial Gaussian distribution is less than the value of the first variable; and the first target dependent variable is obtained based on the first dependent variable and the value of the first variable.

[0044] Optionally, the fusion module generates a second Gaussian distribution based on the second image length in the following ways:

[0045] A second initial Gaussian distribution is generated based on the second image length; the second initial Gaussian distribution has a second Gaussian mean, which is half the length of the second image.

[0046] The second initial Gaussian distribution is normalized to obtain the second Gaussian distribution.

[0047] Optionally, the second initial Gaussian distribution has a second independent variable and a second dependent variable; the fusion module normalizes the second initial Gaussian distribution to obtain the second Gaussian distribution in the following ways:

[0048] Obtain the second reference value of the second independent variable, and obtain the second variable value of the second dependent variable when the second independent variable takes the second reference value. The second reference value is not equal to the second Gaussian mean.

[0049] The values ​​of the second dependent variable within the third range of the second initial Gaussian distribution are normalized to the second constant value, and the values ​​of the second dependent variable within the fourth range of the second initial Gaussian distribution are normalized to the second target dependent variable, thus obtaining the second Gaussian distribution;

[0050] The third range of values ​​is the range in which the second dependent variable takes values ​​greater than or equal to the value of the second variable in the second initial Gaussian distribution. The fourth range of values ​​is the range in which the second dependent variable takes values ​​less than the value of the second variable in the second initial Gaussian distribution. The second target dependent variable is obtained based on the values ​​of the second dependent variable and the second variable.

[0051] Optionally, the first Gaussian distribution has a first independent variable, and the second Gaussian distribution has a second independent variable. The range of values ​​for the first independent variable includes the index value of each pixel in the format-transformed image in the first direction, and the range of values ​​for the second independent variable includes the index value of each pixel in the format-transformed image in the second direction. Each pixel in the format-transformed image has an index value in the first direction and an index value in the second direction. The fusion module generates image position weights based on the first and second Gaussian distributions in the following ways:

[0052] The target Gaussian distribution is obtained by multiplying the first Gaussian distribution and the second Gaussian distribution; the target Gaussian distribution has a first independent variable and a second independent variable.

[0053] The first independent variable of the target Gaussian distribution is taken as the index value of each pixel in the format-transformed image in the first direction, and the second independent variable of the target Gaussian distribution is taken as the index value of each pixel in the format-transformed image in the second direction, so as to obtain the position weight of each pixel in the format-transformed image.

[0054] The image position weights include the position weights of each pixel in the format-transformed image.

[0055] Optionally, the process of generating image color difference weights by the fusion module includes:

[0056] Obtain the second object image in the content-enhanced image; the second object image refers to the local region image of the location of the target object in the content-enhanced image;

[0057] The pixel values ​​at the same pixel position in the format-transformed image and the second object image are subtracted to obtain the target pixel difference between each pixel in the format-transformed image and the pixel at the same pixel position in the second object image.

[0058] A color difference weighting mapping function is used to map the target pixel difference value corresponding to each pixel in the format-transformed image, thereby generating the color difference weight of each pixel in the format-transformed image.

[0059] Among them, the image color difference weight includes the color difference weight of each pixel in the format-transformed image.

[0060] Optionally, the fusion module performs fusion processing on the format-transformed image and the content-enhanced image based on the first fusion weight to obtain an optimized image, including:

[0061] The second fusion weight of the second object image is determined based on the first fusion weight; the second object image refers to the local region image of the location of the target object in the content-enhanced image;

[0062] The format-transformed image is weighted based on the first fusion weight to obtain the weighted format-transformed image;

[0063] The second object image is weighted based on the second fusion weight to obtain the weighted second object image.

[0064] The weighted format-transformed image and the weighted second object image are summed to obtain the fused object image.

[0065] The images other than the second object image in the fused object image and the content enhancement image are combined to obtain the optimized image.

[0066] Optionally, the process of acquiring the first object image by the above-mentioned apparatus includes:

[0067] Perform object detection on the target image to obtain the object detection bounding box of the target object in the target image;

[0068] The object detection box is expanded on the target image to obtain the expanded detection box;

[0069] The image within the expanded detection box in the target image is used as the first object image.

[0070] Optionally, the target image belongs to a video frame in the target video that contains the target object;

[0071] The above-mentioned device is also used for:

[0072] An optimized video is generated based on the optimized image, representing the target video.

[0073] The video will be optimized for transmission to the video client, enabling the video client to play the optimized video.

[0074] This application provides a computer device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the method of this application.

[0075] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the method described in the above-mentioned aspect.

[0076] According to one aspect of this application, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the methods provided in the various alternative embodiments described above.

[0077] This application first acquires a target image containing a target object; the target image is an image with a first dynamic range, and the target image contains a first object image, which refers to a local region image of the target object's location in the target image; then, the image content of the target image can be enhanced to obtain a content-enhanced image; and the image format of the first object image can be transformed to obtain a format-transformed image; the format-transformed image is an image with a second dynamic range, the second dynamic range being greater than the first dynamic range, and the image content of the format-transformed image is the same as the image content of the first object image; furthermore, the format-transformed image and the content-enhanced image can be fused to obtain an optimized image of the target image; the optimized image is an image with a second dynamic range. Therefore, the method proposed in this application can perform overall content enhancement processing on the target image to obtain a content-enhanced image. It can also perform format transformation processing on the local region image (i.e., the first object image) where the target object is located in the target image, while keeping the image content of the first object image unchanged to obtain a format-transformed image. Furthermore, the content-enhanced image and the format-transformed image can be fused to obtain an optimized image of the target image. In the optimized image, the format-transformed image retains some of the original features of the target object in the target image, preventing the target object in the target image from being over-enhanced. Moreover, the content-enhanced image can also achieve overall enhancement of the image content of the target image. Therefore, a superior optimization processing of the target image is achieved, resulting in a superior optimized image of the target image. Attached Figure Description

[0078] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0079] Figure 1 This is a schematic diagram of a network architecture provided in this application;

[0080] Figure 2 This is a schematic diagram of an image optimization scenario provided in this application;

[0081] Figure 3 This is a flowchart illustrating an image processing method provided in this application;

[0082] Figure 4 This is a schematic diagram of a scene for obtaining an image of a first object, as provided in this application;

[0083] Figure 5 This is a schematic diagram of a scene for image fusion processing provided in this application;

[0084] Figure 6 This is a schematic diagram of an architecture for optimizing images provided in this application;

[0085] Figure 7 This is a flowchart illustrating a method for generating image color weights provided in this application;

[0086] Figure 8 This is a schematic diagram of a scenario for generating image color weights provided in this application;

[0087] Figure 9 This is a flowchart illustrating a method for generating image position weights provided in this application;

[0088] Figure 10 This is a schematic diagram of a first initial Gaussian distribution provided in this application;

[0089] Figure 11 This is a schematic diagram of a second initial Gaussian distribution provided in this application;

[0090] Figure 12 This is a flowchart illustrating a method for generating image color difference weights provided in this application;

[0091] Figure 13 This is a schematic diagram of a scenario for generating image color difference weights provided in this application;

[0092] Figure 14This is a schematic diagram of the structure of an image processing device provided in this application;

[0093] Figure 15 This is a schematic diagram of the structure of a computer device provided in this application. Detailed Implementation

[0094] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0095] This application relates to technologies related to artificial intelligence (AI). AI is the theory, methods, technology, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce new intelligent machines that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0096] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0097] This application primarily concerns computer vision technology within artificial intelligence. Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing and measuring targets, and further processes the images to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision researches related theories and technologies, attempting to establish artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), autonomous driving, intelligent transportation, and other technologies, as well as common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0098] The computer vision technology involved in this application mainly refers to the ability to optimize target images to obtain superior optimized images, as detailed below. Figure 3 The description in the corresponding embodiments.

[0099] First, it should be noted that all business data collected in this application (such as target images and other related data containing the target object) was collected with the consent and authorization of the user, organization, or enterprise to which the business data belongs, and the collection, use, and processing of the relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0100] The relevant technical concepts involved in this application are described here:

[0101] Dynamic range: The range of light intensity contained in an image at the same time, used to describe the range of light intensity distribution in an image from the darkest shadows to the brightest highlights.

[0102] HDR: High Dynamic Range.

[0103] SDR: Standard Dynamic Range.

[0104] EOTF: Electro-Optical Transfer Function.

[0105] OETF: Opto-Electronic Transfer Function.

[0106] OOTF: Opto-Optical Transfer Function.

[0107] PQ: Perceptual Quantizer.

[0108] HLG: Hybrid Log–Gamma.

[0109] TM: Tone Mapping.

[0110] iTM: Inverse Tone Mapping.

[0111] Please see Figure 1 , Figure 1 This is a schematic diagram of a network architecture provided in this application. For example... Figure 1 As shown, the network architecture may include server 200 and a cluster of terminal devices. The cluster of terminal devices may include one or more terminal devices; the number of terminal devices is not limited here. Figure 1 As shown, multiple terminal devices may specifically include terminal device 1, terminal device 2, terminal device 3, ..., terminal device n; as... Figure 1 As shown, terminal device 1, terminal device 2, terminal device 3, ..., terminal device n can all connect to server 200 via the network, so that each terminal device can interact with server 200 through the network connection.

[0112] like Figure 1 The server 200 shown can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal device can be a smartphone, tablet, laptop, desktop computer, smart TV, or other smart terminal. The following description uses the communication between terminal device 1 and server 200 as an example to illustrate the specific implementation of this application.

[0113] Please see also Figure 2 , Figure 2 This is a scene illustration of image optimization provided in this application. For example... Figure 2As shown, server 200 can obtain the target image to be optimized. The dynamic range of the target image can be the first dynamic range. The target image can contain a target object, which can be a face. The local area image of the face in the target image can be called the first object image, that is, the target image contains the first object image.

[0114] First, server 200 can perform overall image enhancement processing on the target image to obtain an enhanced image of the target image. Server 200 can also perform format transformation processing on a first object image within the target image to obtain a format-transformed image of the first object image. The dynamic range of this format-transformed image can be a second dynamic range, which can be greater than the first dynamic range of the first object image. The image content of the format-transformed image is the same as the image content of the first object image; that is, the visual effect of the format-transformed image is the same as the visual effect of the first object image.

[0115] Furthermore, server 200 can perform fusion processing on the content-enhanced image and the format-transformed image to obtain an optimized image of the target image. Server 200 can also send the optimized image to terminal device 1, so that terminal device 1 can display the optimized image of the target image to improve the visual effect of the user viewing the image.

[0116] The optimized image of the target image obtained by the method provided in this application can retain some of the original features of the target object in the target image by fusing the format transformation image, preventing the target object from being over-enhanced (because the human eye is very sensitive to the vision of the face, and even slight changes in skin color or content can be perceived). The content enhancement image obtained by fusing the image can also enhance the overall image content of the target image, thus improving the optimization effect of the target image and obtaining a superior optimized image of the target image.

[0117] Please see Figure 3 , Figure 3 This is a flowchart illustrating an image processing method provided in this application. The execution entity in the embodiments of this application can be an image processing device, which can be a single computer device or a cluster of multiple computer devices. The computer device can be a server, a terminal device, or other devices; there are no limitations on this. Figure 3 As shown, the method may include:

[0118] Step S101: Obtain a target image containing the target object; the target image is an image with a first dynamic range, and the target image contains a first object image, which refers to a local area image of the location of the target object in the target image.

[0119] Optionally, the image processing device can acquire a target image containing the target object, which is an image to be optimized (i.e., needs to be optimized). The target image can be an image with a first dynamic range, that is, the dynamic range of the target image can be referred to as the first dynamic range. For example, if the first dynamic range can be the dynamic range of SDR, then the target image can be an SDR image.

[0120] The target object can be any object that needs to be optimized. This optimization process can be an optimization process that preserves some of the original features of the target object in the target image (such as the original color or content) to prevent the target object from being over-optimized (such as over-enhanced) during the optimization process of the target image, which would result in problems such as insufficient aesthetics and poor visual effects of the optimized target object.

[0121] Depending on the specific optimization scenario, the type of the target object can also be different.

[0122] For example, because the human eye is very sensitive to changes in facial images (i.e., the human eye has a very strong perception of the visual effects of facial images), if a face is over-enhanced, it will become unclear and the skin color will become unrealistic. Therefore, in this scenario, the target object can be any face. By performing specific optimization processing on the face, the original skin color and facial content of the face in the target image can be preserved to a certain extent.

[0123] For example, the target object can be any item (such as a table, sofa, dye, cabinet, etc.). In this scenario, the target image can be a photo of the item (either a photo taken by the user of their own item or a photo taken by the merchant of their product). When optimizing this photo, in addition to enhancing the background of the item in the photo, the original characteristics of the item when it was actually taken can be preserved to a certain extent (such as the original color or original visual features of the item). This also ensures that the optimized item has a stronger sense of realism.

[0124] Similarly, the target object can also be the face of any animal, the entire body of any animal, etc. The specific target object can be determined according to the actual optimization scenario, and there are no restrictions on it.

[0125] Optionally, the target image may contain one or more target objects. Since the methods for performing specific optimization processing on each target object are independent and the same, it is understood that the following description is based on the process of performing specific optimization processing on a single target object.

[0126] The target image may contain a first object image, which may be a local region image of the location of the target object within the target image. As described below, the methods for obtaining the first object image may include, but are not limited to:

[0127] Image processing devices can perform object detection (also known as target detection, where the target is the target object) on a target image to obtain a detection box (bounding box) showing the location of the target object in the target image. This detection box can be called the object detection box.

[0128] For example, an image processing device can use a trained detection model to perform object detection on a target image to obtain a detection box indicating the location of the target object in the target image. This detection model can be any trained model that can be used to detect target objects (such as the location of the target object).

[0129] Furthermore, the image processing device can expand the object detection box on the target image to obtain an expanded object detection box, which can be called an expanded detection box.

[0130] The image processing device can use the image within the expanded detection box in the target image as the first object image. Therefore, it can be understood that the first object image not only contains the target object but may also contain a portion of the background image of the target object within the target image. Since the first object image can be an image of a local region in the target image, the size (e.g., dimensions) of the first object image can be smaller than the size (e.g., dimensions) of the target image.

[0131] This application expands the object detection box of the target object and obtains the first object image by expanding the detection box. This allows the target object to have some background image around it for optimization transition when performing specific optimization processing. This makes the optimization of the target object more natural and makes the optimized target object and the optimized background image fit better (i.e., the fusion effect is better). It also prevents the optimized target object from having obvious boundary sense. For details, please refer to the following description.

[0132] Optionally, the object detection box can be expanded to obtain an expanded detection box. The principle behind this expansion is shown in the following formula:

[0133] W R = (1+2α)×W B (1)

[0134] H R = (1+2α)×H B (2)

[0135] The object detection box can be a rectangle, W B W is the width of the object detection box. R To expand the width of the detection box, H B H is the height of the object detection box. R The height of the expanded detection box is α. α is an adjustable parameter (also called the expansion parameter). α adjusts the degree (e.g., ratio) of the expansion of the object detection box. A larger α results in a greater expansion of the object detection box, leading to a larger expanded detection box and consequently a larger first object image. A larger first object image results in a larger spatial transition region (e.g., the background region of the target object in the first object image) for subsequent optimization processing. Therefore, α can be understood as adjusting the size of the spatial transition region during specific optimization processing of the target object. Optionally, a suggested value for α is 1 / 5.

[0136] Please see Figure 4 , Figure 4 This is a schematic diagram of a scene for acquiring an image of a first object, as provided in this application. Figure 4 As shown, the target object can be a human face. The image processing device can perform face detection on the target image, which means that the object detection box of the target object in the target image can be detected.

[0137] Furthermore, the image processing device can expand the object detection box on the target image (i.e., enlarge it; the specific expansion ratio can be determined according to the actual application scenario, such as expanding outward from the center of the object detection box, or expanding outward from the center of the object detection box in all four directions), thus obtaining the expanded detection box of the object detection box.

[0138] Furthermore, the image processing device can use the image within the expanded detection box in the target image as the first object image, which includes the target object and a portion of the image surrounding the target object.

[0139] Step S102: Enhance the image content of the target image to obtain a content-enhanced image.

[0140] Optionally, the image processing device can enhance the entire image content (such as the entire image frame) of the target image (which can be referred to as content transformation of the target image) to obtain a content-enhanced image. This enhancement process can be understood as optimizing the image content of the target image.

[0141] For example, a content-enhanced image can be obtained by enhancing the contrast, brightness, and saturation of the target image (e.g., increasing contrast, brightness, and saturation). The enhancement of the target image's content can be achieved using appropriate image enhancement methods based on the specific application scenario, and there are no restrictions on this.

[0142] Image processing devices can select either traditional inverse tone mapping algorithms or AI-based inverse tone mapping algorithms (such as those based on trained enhancement models, which can be trained models for content enhancement of images) as needed to enhance the image content of the target image to obtain a content-enhanced image.

[0143] The dimensions of the content-enhanced image (i.e., the image dimension) are the same as those of the target image, and the number of pixels in the content-enhanced image is also the same as the number of pixels in the target image.

[0144] It is understandable that, since a content-enhanced image is obtained by enhancing the image content of a target image, the image content of a content-enhanced image is different from that of the target image, and the visual effect of a content-enhanced image is also different from that of the target image.

[0145] Furthermore, it should be noted that during the image content enhancement process of the target image, the image format of the target image can also be transformed simultaneously (e.g., including) to transform the target image from the first dynamic range to the second dynamic range, so that the resulting content-enhanced image can also be an image of the second dynamic range. The second dynamic range is greater than the first dynamic range. For example, if the second dynamic range can be the dynamic range of HDR, then the content-enhanced image can be an HDR image. The process of transforming the image format of the target image can refer to the process of transforming the image format of the first object image described in step S103 below, and the principles of the two processes can be the same.

[0146] In other words, the image format of the content-enhanced image can be the same as the image format of the optimized image of the target image in step S104 below.

[0147] Step S103: The image format of the first object image is transformed to obtain a format-transformed image; the format-transformed image is an image with a second dynamic range, the second dynamic range is greater than the first dynamic range, and the image content of the format-transformed image is the same as the image content of the first object image.

[0148] Optionally, the image processing device can perform image format transformation on the first object image (which can be referred to as performing format transformation on the first object image) to obtain a format-transformed image. Since the first object image is an image with a first dynamic range, the image format transformation on the first object image can be performed to transform the first object image from an image with a first dynamic range to an image with a second dynamic range. That is, the dynamic range of the format-transformed image can be the second dynamic range, which is greater than the first dynamic range.

[0149] It is understandable that since dynamic range represents the range of light intensity contained in an image (which can also be understood as the range of brightness), the larger the dynamic range, the larger the range of light intensity that the image can contain. Dark areas of the image can be darker and bright areas can be brighter with more detail and depth, making the visual effect of the image richer and thus the overall effect of the image better.

[0150] If it is necessary to optimize an SDR image (such as a target image) into an HDR image, then the second dynamic range can be the dynamic range of HDR. That is, transforming the image format of the first object image can transform the image format of the first object image from the SDR image format to the HDR image format, and the image format of the transformed image will be the HDR image format.

[0151] However, it should be noted that the format-converted image is different from the first object image only in that the image format has changed. The image content of the format-converted image remains the same as that of the first object image, so that the visual effect of the format-converted image is the same as that of the first object image in a standard display environment. The format-converted image can be obtained by representing the image format of the first object image as an HDR image.

[0152] Optionally, the process of transforming (i.e., converting) the image format of the first object image (which may include the image's color gamut format and the image's light format) to obtain a format-transformed image may include:

[0153] Assuming the first object image can be an RGB image (a color image) with BT.709 color gamut (a color gamut standard) and BT.709 gamma curve (a curve used to represent light), it is necessary to transform the first object image into an RGB image with BT.2020 color gamut (a color gamut standard) and PQ transfer curve (a curve used to convert the format of light to the format of HDR light) (the transformed image is the format-converted image).

[0154] First, the image processing device can perform EOTF conversion (i.e., electro-optical conversion) on the first object image obtained above. That is, the RGB channels of the first object image (including the R channel (red color channel), G channel (green color channel), and B channel (blue color channel)) are respectively applied to the EOTF conversion corresponding to the BT.709 gamma curve inverse transform, so as to convert the light on the RGB channels of the first object image into linear light. The image after performing EOTF conversion on the first object image can be called an electro-optical converted image.

[0155] Next, the image processing device can perform OOTF conversion (i.e., photo-to-photo conversion, such as converting linear scene light into display light) on the electro-optically converted image. This involves applying the OOTF conversion corresponding to the BT.709 gamma curve to the electro-optically converted image. The image resulting from this photo-to-photo conversion is called the photo-to-photo converted image. This step ensures that the visual effect of the subsequently converted format-transformed image is identical to that of the first object image.

[0156] Furthermore, image processing equipment can perform color gamut conversion on optically converted images, such as converting the optically converted image from the BT.709 color gamut to the BT.2020 color gamut. That is, the BT.709 color gamut of the optically converted image is represented as the BT.2020 color gamut. The image obtained after color gamut conversion of the optically converted image is called a color gamut converted image. The BT.2020 color gamut is larger than the BT.709 color gamut. By performing color gamut conversion on the optically converted image, the range of color gamut values ​​that the processed color gamut converted image can reach is wider.

[0157] Finally, the image processing device can perform OETF conversion (i.e., photoelectric conversion) on the color gamut converted image, that is, apply the color gamut converted image to the PQ OETF curve (a curve used to convert the light format to the HDR light format) to convert the light of the color gamut converted image into nonlinear light (the intensity of nonlinear light is generally linear with the subjective visual intensity of the human eye, that is, nonlinear light is closer to the human eye's perception of light intensity), thus obtaining the format-converted image (also called the format-converted image).

[0158] The color gamut of the first object image (e.g., BT.709 color gamut) and the color gamut of the format-converted image (e.g., BT.2020 color gamut) mentioned above are just examples. The color gamut of the first object image and the color gamut of the format-converted image can also be set to other color gamuts. If other color gamuts are used, the color gamuts exemplified in the above conversion process can be replaced with the other color gamuts for conversion.

[0159] Similarly, the light format of the first object image (such as the light format represented by the BT.709 gamma curve) and the light format of the format-converted image (such as the light format represented by the PQ transmission curve) are just examples. The light format of the first object image and the light format of the format-converted image can also be other formats. If they are other formats, the light format exemplified in the above conversion process can be replaced with the other format for conversion.

[0160] By transforming the color gamut and light format of the first object image, the dynamic range of the first object image can be transformed, that is, the first dynamic range of the first object image can be transformed into a second dynamic range. Both color gamut and light format conversions contribute to the transformation of dynamic range, but light format conversion plays a primary role.

[0161] Through the above process, the image format of the first object image is transformed, resulting in a format-transformed image. The format-transformed image has the same dimensions as the first object image, and the number of pixels (i.e., image pixels) in the format-transformed image is the same as the number of pixels in the first object image.

[0162] Step S104: The format-transformed image and the content-enhanced image are fused to obtain an optimized image of the target image; the optimized image is an image with the second dynamic range.

[0163] Optionally, the image processing device can perform fusion processing on the format-converted image and content-enhanced image obtained above to obtain an optimized image of the target image. The optimized image is also a second dynamic range image, such as an HDR image. As the name suggests, the optimized image is the image obtained after performing the above optimization processing on the target image. The dimension of the optimized image can be the same as the dimension of the target image.

[0164] The process by which an image processing device fuses a format-transformed image and a content-enhanced image to obtain an optimized image of the target image can be described as follows:

[0165] Image processing equipment can obtain the fusion weight of format-transformed images. The fusion weight of format-transformed images can be called the first fusion weight. As the name suggests, the first fusion weight is the weight used to fuse the format-transformed image when performing fusion processing on the format-transformed image and the content-enhanced image.

[0166] The first fusion weight can be obtained through the image color weight, image position weight, and / or image chromatic difference weight of the format-transformed image. Each of these weights can be represented as a corresponding matrix (which can be a two-dimensional matrix). In other words, each weight can be a weight matrix, and the dimension of this weight matrix can be the same as the dimension of the format-transformed image. For example, the number of elements in the weight matrix can be the same as the number of pixels in the format-transformed image, as described below.

[0167] Optionally, the image color weights can be obtained based on the difference (i.e., color difference) between the reference color of the target object and the image color of the first object image. In other words, the image color weights can be used to represent the color difference between the reference color of the target object and the image color of the first object image. The process of generating the image color weights can be found below. Figure 7 Specific descriptions in the corresponding embodiments.

[0168] In an image color weighting system, an element (i.e., a value) can be called a color weight. Each pixel in the format-transformed image can correspond to a color weight in the image color weighting system. That is, there is a one-to-one correspondence between each pixel in the format-transformed image and each element (i.e., each color weight) in the image color weighting system. For example, the color weight of the pixel in the first row and first column of the format-transformed image can be the color weight in the first row and first column of the image color weighting system (i.e., the element in the first row and first column); and the color weight of the pixel in the second row and second column of the format-transformed image can be the color weight in the second row and second column of the image color weighting system (i.e., the element in the second row and second column), and so on.

[0169] Optionally, image position weights can be obtained based on the different positions of each pixel in the format-transformed image; that is, image position weights can be used to represent the position of each pixel in the format-transformed image. The process of generating image position weights can be found below. Figure 9 Specific descriptions in the corresponding embodiments.

[0170] Similarly, an element (i.e., a value) in the image positional weights can be called a positional weight. Each pixel in the format-transformed image can correspond to a positional weight in the image positional weights. That is, there is a one-to-one correspondence between each pixel in the format-transformed image and each element (i.e., each positional weight) in the image positional weights. For example, the positional weight of any pixel in the format-transformed image can be the positional weight in the image positional weights that has the same position as that pixel. For instance, the positional weight of the pixel in the 1st row and 1st column of the format-transformed image can be the positional weight in the 1st row and 1st column of the image positional weights (i.e., the element in the 1st row and 1st column); and the positional weight of the pixel in the 2nd row and 2nd column of the format-transformed image can be the positional weight in the 2nd row and 2nd column of the image positional weights (i.e., the element in the 2nd row and 2nd column), and so on.

[0171] Optionally, the image color difference weight can be obtained based on the difference (i.e., pixel difference) between the pixel values ​​of pixels in the format-transformed image and the pixel values ​​of pixels in the second object image. Since the pixel value itself is used to represent the chromaticity of the pixel, the image color difference weight can be used to represent the color difference between pixels in the format-transformed image and pixels in the second object image. The second object image can refer to a local region image of the target object's location in the content-enhanced image. The dimension of the second object image can be the same as the dimension of the first object image, and the position of the first object image in the target image can also be the same as the position of the second object image in the content-enhanced image. Here, it can be understood that both the target image and the content-enhanced image can contain the target object, but the target object in the content-enhanced image can be an object after enhancement processing of the target object in the target image. The process of generating the image color difference weight can be referred to below. Figure 12 Specific descriptions in the corresponding embodiments.

[0172] Similarly, an element (i.e., a value) in the image color difference weighting can be called a color difference weight. Each pixel in the format-transformed image can correspond to a color difference weight in the image color difference weighting. That is, there is a one-to-one correspondence between each pixel in the format-transformed image and each element (i.e., each color difference weight) in the image color difference weighting. For example, the color difference weight of any pixel in the format-transformed image can be the color difference weight in the image color difference weighting that has the same position as that pixel. For example, the color difference weight of the pixel in the first row and first column of the format-transformed image can be the color difference weight in the first row and first column of the image color difference weighting (i.e., the element in the first row and first column); and the color difference weight of the pixel in the second row and second column of the format-transformed image can be the color difference weight in the second row and second column of the image color difference weighting (i.e., the element in the second row and second column), and so on.

[0173] Optionally, the first fusion weight can be obtained based on any one of the image color weight, image position weight, and image color difference weight mentioned above. It can be understood that the image color weight can preserve some of the original colors of the target object in the target image (e.g., if the target object is a face, then the original skin color of the face is preserved). Then, the image position weight and image color difference weight can improve the optimization effect on the target object while preserving some of its original colors through the image color weight. Therefore, if the first fusion weight is obtained based on any one of the image color weight, image position weight, and image color difference weight, then the image color weight can be preferred. That is, the image color weight can be the most important weight among the image color weight, image position weight, and image color difference weight.

[0174] Optionally, if the first fusion weight is obtained based on the image color weight, the image processing device can filter the image color weight (e.g., perform guided filtering) to obtain the first fusion weight. This avoids abrupt changes in the first fusion weight (e.g., a sudden appearance of a particularly large or small weight), making the subsequent fusion between the format-transformed image and the content-enhanced image smoother and more natural. If the first fusion weight is obtained based on either the image position weight or the image color difference weight, the principle for obtaining the first fusion weight based on either weight can be the same as the principle for obtaining the first fusion weight through the image color weight.

[0175] Alternatively, the first fusion weight can be obtained by fusing any two of the image color weight, image position weight, and image color difference weight (e.g., by performing a matrix multiplication operation). Similarly, since the image color weight is more important, if the first fusion weight is obtained by fusing any two of the image color weight, image position weight, and image color difference weight, then fusing the image color weight and image position weight (e.g., by performing a matrix multiplication operation) is preferable to obtain the first fusion weight. Alternatively, fusing the image color weight and image color difference weight (e.g., by performing a matrix multiplication operation) is also preferable to obtain the first fusion weight.

[0176] Optionally, after fusing any two of the image color weights, image position weights, and image color difference weights (e.g., by dot product), an initial fusion weight can be obtained. The image processing device can then filter this initial fusion weight (e.g., by guided filtering) to obtain the first fusion weight. Similarly, this avoids abrupt changes in the first fusion weight (e.g., a sudden appearance of a very large or very small weight), resulting in a smoother and more natural fusion between the format-transformed image and the content-enhanced image.

[0177] Alternatively, the first fusion weight can be obtained by fusing the image color weight, image position weight, and image color difference weight together (e.g., by matrix multiplication). Since fusing these three weights together yields a first fusion weight that considers the format-transformed image from multiple dimensions, this method is generally more accurate and is commonly used. However, to reduce computation or for other considerations, the above method (e.g., obtaining the first fusion weight using any one or two of the image color weight, image position weight, and image color difference weight) can be used.

[0178] Optionally, after fusing the three weights—image color weight, image position weight, and image color difference weight—together (e.g., by dot product), an initial fusion weight can be obtained. The image processing device can then filter this initial fusion weight (e.g., by guided filtering) to obtain the first fusion weight. Similarly, this avoids abrupt changes in the first fusion weight (such as a sudden appearance of a very large or very small weight), resulting in a smoother and more natural fusion between the format-transformed image and the content-enhanced image.

[0179] Since guided filtering is an edge-preserving filtering method, the edge information (i.e., weight information at the edges) of the first fused weight obtained after applying guided filtering to the aforementioned weights (such as any one of image color weights, image position weights, and image color difference weights, or the weight obtained by fusing at least two of these weights) will not be lost. This ensures the integrity of the first fused weight, making the dimension of the first fused weight obtained after filtering the same as the dimension of the format-transformed image. Optionally, the guided filtering described above can be replaced with other filters that achieve the same effect, without limitation.

[0180] The first fusion weight obtained above can also be represented as a matrix (such as a two-dimensional matrix). The dimension of the first fusion weight can be the same as the dimension of the format-transformed image. An element (i.e., a value) in the first fusion weight can be called a weight. There can be a one-to-one correspondence between each element (i.e., each weight) in the first fusion weight and each pixel in the format-transformed image. A pixel in the format-transformed image can have one of the weights in the first fusion weight. For example, the weight of any pixel in the format-transformed image in the first fusion weight can be the weight in the first fusion weight that has the same position as that pixel. For example, for the first fusion weight, the weight of the pixel in the first row and first column of the format-transformed image can be the weight in the first row and first column of the first fusion weight (i.e., the element in the first row and first column); and the weight of the pixel in the second row and second column of the format-transformed image can be the weight in the second row and second column of the first fusion weight (i.e., the element in the second row and second column), and so on.

[0181] Therefore, the image processing device can use the aforementioned first fusion weight to fuse the format-transformed image and the content-enhanced image to obtain an optimized image of the target image, as described below.

[0182] First, the image processing device can also obtain the fusion weight of the second object image through the first fusion weight, which can be referred to as the second fusion weight. The second object image is the local region image of the target object in the content-enhanced image. The size (e.g., image dimension) of the second object image is the same as the size of the format-transformed image. The first object image, the second object image, and the format-transformed image can all correspond to the same location region in the target image (e.g., the region of the expanded detection box mentioned above). That is, the first object image, the second object image, and the format-transformed image can all be local region images of the same size as the target object.

[0183] Optionally, the image processing device can multiply each target weight in the first fusion weight by a fusion coefficient (also called a fusion parameter) to obtain the target fusion weight (which can also be a weight matrix). For example, the element in the first row and first column of the target fusion weight is obtained by multiplying the element in the first row and first column of the first fusion weight by the fusion coefficient; and the element in the second row and second column of the target fusion weight is obtained by multiplying the element in the second row and second column of the first fusion weight by the fusion coefficient, and so on.

[0184] Optionally, the above-mentioned fusion coefficient is an adjustable parameter. The adjustable value range of the fusion coefficient can be [0, 1], which can be adjusted according to actual needs. By adjusting the fusion coefficient, the preservation strength (i.e. preservation degree) of the original creative intent of the target object (such as the original features of the target object in the target image (such as the original color features or content features)) can be adjusted. The larger the fusion coefficient, the greater the preservation strength of the original creative intent of the target object. Conversely, the smaller the fusion coefficient, the less the preservation strength of the original creative intent of the target object.

[0185] Therefore, there can be a one-to-one correspondence between each pixel in the format-transformed image and each element (which can be called a weight) in the target fusion weight. For example, the weight of any pixel in the format-transformed image in the target fusion weight can be the weight of the pixel at the same position in the target fusion weight. For instance, for the target fusion weight, the weight of the pixel in the first row and first column of the format-transformed image can be the weight of the pixel in the first row and first column of the target fusion weight (i.e., the element in the first row and first column); and the weight of the pixel in the second row and second column of the format-transformed image can be the weight of the pixel in the second row and second column of the target fusion weight (i.e., the element in the second row and second column).

[0186] Furthermore, the image processing device can use a constant 1 (which can be called the fusion weight adjustment parameter or fusion proportion adjustment parameter) to subtract each element (i.e., each weight) in the target fusion weight to obtain the second fusion weight. For example, the element in the first row and first column of the second fusion weight (which can also be called the weight) is obtained by subtracting the element in the first row and first column of the target fusion weight (i.e., subtracting the element value of the element in the first row and first column of the target fusion weight) from the constant 1; the element in the second row and second column of the second fusion weight is obtained by subtracting the element in the second row and second column of the target fusion weight from the constant 1, and so on.

[0187] Therefore, the second fusion weight can also be represented as a matrix (such as a two-dimensional matrix). The dimension of the second fusion weight can be the same as the dimension of the second object image. An element (i.e., a numerical value) in the second fusion weight can also be called a weight. There can also be a one-to-one correspondence between each element (i.e., each weight) in the second fusion weight and each pixel in the second object image. For example, the weight of any pixel in the second object image in the second fusion weight can be the weight in the second fusion weight that has the same position as that pixel. For instance, for the second fusion weight, the weight of the pixel in the first row and first column of the second object image can be the weight in the first row and first column of the second fusion weight (i.e., the element in the first row and first column); and the weight of the pixel in the second row and second column of the second object image can be the weight in the second row and second column of the second fusion weight (i.e., the element in the second row and second column), and so on.

[0188] Furthermore, the image processing device can perform fusion processing on the format-transformed image and the second object image using the aforementioned target fusion weights and second fusion weights to obtain a fused object image, including:

[0189] It is understandable that by subtracting the weights in the target fusion weights from the aforementioned constant 1 (i.e., the fusion weight adjustment parameter), a negative correlation can be established between the weights in the second fusion weights and the weights in the target fusion weights. The larger the weights in the target fusion weights, the smaller the weights in the second fusion weights. Therefore, the fusion ratio between the format-transformed image and the second object image can be adjusted through the second fusion weights and the target fusion weights. The larger the target fusion weight, the larger the fusion ratio for the format-transformed image and the smaller the fusion ratio for the second object image. Conversely, the larger the second fusion weight, the larger the fusion ratio for the second object image and the smaller the fusion ratio for the format-transformed image, as described below.

[0190] First, the image processing device can perform weighted processing on the format-transformed image based on the target fusion weight obtained through the first fusion weight, so as to obtain the weighted format-transformed image.

[0191] Image processing devices can multiply (i.e., weight) the pixel value of each pixel in the format-transformed image with its corresponding weight in the target fusion weights to obtain a weighted format-transformed image. For example, the pixel value of the pixel in the first row and first column of the weighted format-transformed image can be obtained by multiplying the pixel value of the pixel in the first row and first column of the format-transformed image with the element in the first row and first column of the target fusion weights (which can be called the weights).

[0192] Similarly, the image processing device can also perform weighted processing on the second object image based on the above-mentioned second fusion weight to obtain a weighted second object image.

[0193] For example, an image processing device can multiply (i.e., weight) the pixel value of each pixel in the second object image with the corresponding weight in the second fusion weight to obtain a weighted second object image. For instance, the pixel value of the pixel in the first row and first column of the weighted second object image can be obtained by multiplying the pixel value of the pixel in the first row and first column of the second object image with the element in the first row and first column of the second fusion weight.

[0194] The dimensions of the weighted format-transformed image obtained above are the same as the dimensions of the weighted second object image. The image processing device can perform summation processing on the weighted format-transformed image and the weighted second object image to obtain the fused object image.

[0195] For example, an image processing device can add the pixel value of each pixel in the weighted format-transformed image to the pixel value of the pixel at the same position in the weighted second object image to obtain the fused object image. The pixel value of the pixel in the first row and first column of the fused object image can be obtained by adding the pixel value of the pixel in the first row and first column of the weighted format-transformed image to the pixel value of the pixel in the first row and first column of the weighted second object image.

[0196] Furthermore, the image processing device can combine the images between the fused object image and the content enhancement image, excluding the second object image, to obtain an optimized image of the target image. That is, the optimized image is composed of the images between the fused object image and the content enhancement image, excluding the second object image. It can be understood that the optimized image is obtained by replacing the second object image in the content enhancement image with the fused object image.

[0197] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating a scene for image fusion processing provided in this application. For example... Figure 5 As shown, the target object may contain a first object image, and a format-transformed image can be obtained by performing format transformation on the first object image.

[0198] An image processing device enhances the image content of a target image to obtain a content-enhanced image. This content-enhanced image may contain a second object image, which can be understood as the second object image being an image after the image content of the first object image has been enhanced.

[0199] Furthermore, the image processing device can perform image fusion processing on the format-transformed image and the second object image using the aforementioned first fusion weight to obtain a fused object image. Next, the image processing device can replace the second object image in the content-enhanced image with the fused object image to obtain an optimized image of the target image.

[0200] By using the target fusion weight of the format-transformed image and the second fusion weight of the second object image, a balance can be achieved between the format-transformed image and the second object image when fusing them.

[0201] This can be understood as follows: the first fusion weight can be used to determine the fusion ratio (i.e. the degree of preservation of the format-transformed image) of the format-transformed image when it is fused with the second object image. The larger the first fusion weight, the greater the fusion ratio and preservation degree of the format-transformed image in the above-mentioned fused object image. Conversely, the smaller the first fusion weight, the smaller the fusion ratio and preservation degree of the format-transformed image in the above-mentioned fused object image.

[0202] Similarly, the second fusion weight can be used to determine the fusion ratio (i.e. the degree of preservation of the second object image) of the second object image when it is fused with the format-transformed image. The larger the second fusion weight, the greater the fusion ratio and the degree of preservation of the second object image in the above-mentioned fused object image. Conversely, the smaller the second fusion weight, the smaller the fusion ratio and the degree of preservation of the second object image in the above-mentioned fused object image.

[0203] Alternatively, in another feasible implementation, the second object image and the format-transformed image may not be fused separately, as described below.

[0204] Optionally, the dimensions of the format-transformed image can be padded with 0 pixels to the same dimension as the target image (i.e., the same dimension as the content-enhanced image) to obtain a padded format-transformed image. In this padded format-transformed image, all images except the format-transformed image obtained after format transformation of the first object image can be composed of 0 pixels.

[0205] Similarly, the size (i.e. dimension) of the target fusion weight can be filled with 0 elements (i.e., 0 weights) to be the same as the size of the format-transformed image after filling (i.e., the same as the size of the target image), so that the weights of the 0 pixels filled in the format-transformed image after filling are also 0 in the target fusion weight.

[0206] When both the dimensions of the format-transformed image and the target fusion weights are padded to match the dimensions of the target image, there is no need to obtain the second object image from the content-enhanced image, nor is there any need to perform additional fusion processing on the second object image and the format-transformed image. In this case, each element (i.e., each weight) in the padded target fusion weights can be directly subtracted from the constant 1 to obtain the fusion weights of the content-enhanced image (which can be referred to as the padded second fusion weights). The padded second fusion weights have the same dimensions as the content-enhanced image, and each pixel in the content-enhanced image corresponds one-to-one with the weight at the same position in the padded second fusion weights. In this case, the weights corresponding to the images in the content-enhanced image other than the second object image (i.e., the pixels in the images other than the second object image) in the padded second fusion weights are all 1.

[0207] Furthermore, the format-transformed image after padding can be directly multiplied at corresponding positions (i.e., the same positions) by the target fusion weights after padding (e.g., multiplying the pixel value of the first row and first column of the format-transformed image after padding with the element of the first row and first column of the target fusion weights after padding, and so on). This will yield a weighted image of the format-transformed image after padding (which can be called a weighted format-transformed image after padding). The dimension of the weighted format-transformed image after padding is the same as the dimension of the target image.

[0208] Similarly, the second fusion weight after padding in this case can be directly multiplied with the corresponding position (i.e., the same position) of the content-enhanced image (e.g., multiply the pixel value of the pixel in the first row and first column of the content-enhanced image with the element in the first row and first column of the second fusion weight after padding, and so on) to obtain the weighted content-enhanced image. The dimension of the weighted content-enhanced image is the same as the dimension of the target image.

[0209] Furthermore, the image processing device can directly add the weighted fill format transformation image and the weighted content enhancement image at corresponding positions (i.e., the same positions) (e.g., adding the pixel value of the pixel at the first row and first column of the weighted fill format transformation image to the pixel value of the pixel at the first row and first column of the weighted content enhancement image, and so on), thus directly obtaining the optimized image of the target image. Using this method to fuse the format transformation image and the content enhancement image improves the convenience and efficiency of fusing the format transformation image and the content enhancement image because no additional fusion processing is required for the second object image and the format transformation image. As shown in the following formula:

[0210] I HDR =W f ×k×HF +(1-W f ×k)×H C (3)

[0211] Among them, I HDR This represents image optimization, k represents the fusion coefficient, and W... f This can be represented as the first fusion weight being padded with zero weights to the same dimension as the target image (this can be called the padded first fusion weight), k×W f This refers to the target fusion weights after padding, H. F This represents the format-transformed image after filling. 1-W f ×k represents the second fusion weight obtained by subtracting each weight in the padded target fusion weight from the constant 1, where H is the fusion weight. C This refers to content-enhanced images.

[0212] It is understandable that the optimized image obtained by the two methods mentioned above (including the method with zero padding or without zero padding) is the same. The specific method used to fuse the format-transformed image and the content-enhanced image can be selected according to the actual application scenario, and there are no restrictions on this.

[0213] This application can also be applied to scenarios where videos are optimized: the target image can be a video frame containing the target object in the target video, the target video can be any video that needs to be optimized, and there can be one or more video frames containing the target object in the target video, that is, there can be one or more target images, and each target image can be optimized according to the process described above to obtain the optimized image of each target image.

[0214] For video frames in the target video that do not contain the target object, the image content of each video frame that does not contain the target object can be enhanced directly based on the above-mentioned method of enhancing the image content of the target image, so as to obtain the optimized image of each video frame that does not contain the target object. That is, for video frames that do not contain the target object, the image content of the video frame can be directly enhanced to obtain the optimized image of the video frame.

[0215] The above process yields the optimized image (which can be called the optimized video frame) of each video frame in the target video. The optimized video of the target video is composed of the optimized video frames of each video frame in the target video.

[0216] The image processing device can transmit the optimized video to a video client, enabling the video client to play the optimized video. In other words, using this application can further improve the optimization effect on the target video, thereby enhancing the viewing experience of the target video.

[0217] Please see Figure 6 , Figure 6 This is a schematic diagram of an architecture for image optimization provided in this application. For example... Figure 6 As shown, the SDR image here is the target image, and the target object can be a human face. The image processing device can perform face detection on the SDR image to obtain a first object image showing the location of the target object in the target image.

[0218] The image processing device can calculate the image color weight (i.e., the image color weight mentioned above) through the first object image, and can calculate the image position weight through the size of the first object image (such as width and height, or the size of the format-transformed image obtained from the first object image, the size of the first object image and the size of the format-transformed image are the same).

[0219] The image processing device can also perform format up-conversion (i.e. format up-conversion) on the first object image to obtain a format-converted image, and can perform content enhancement processing (i.e. content up-conversion, or content up-conversion) on the image content of the SDR image at this location to obtain a content-enhanced image.

[0220] Next, the image processing device can calculate the image color difference weights by converting the format image and enhancing the content image. Then, the image processing device can perform a fusion process (such as dot product operation) on the obtained image color weights, image position weights, and image color difference weights, followed by a filtering process (such as guided filtering) to obtain the first fusion weight of the format-converted image.

[0221] Then, the image processing device can use the first fusion weight to perform image fusion processing on the format-transformed image and the content-enhanced image to obtain the optimized image of the SDR image (i.e., the HDR image).

[0222] Based on the above description, this application proposes an inverse tone mapping method that preserves the original creative intent of the target object in the SDR image (such as the image preserved through format transformation) in the optimized HDR image (such as the optimized image of the target image). This method effectively maintains the original creative intent of the target object in the SDR image and performs spatial smoothing through three weights: image color weight (i.e., image color weight), image position weight, and image color difference weight. This effectively solves the problems of over-enhancement and visual distortion of the target object when optimizing an SDR image into an HDR image, and improves the optimization effect of optimizing an SDR image into an HDR image.

[0223] This application first acquires a target image containing a target object; the target image is an image with a first dynamic range, and the target image contains a first object image, which refers to a local region image of the target object's location in the target image; then, the image content of the target image can be enhanced to obtain a content-enhanced image; and the image format of the first object image can be transformed to obtain a format-transformed image; the format-transformed image is an image with a second dynamic range, the second dynamic range being greater than the first dynamic range, and the image content of the format-transformed image is the same as the image content of the first object image; furthermore, the format-transformed image and the content-enhanced image can be fused to obtain an optimized image of the target image; the optimized image is an image with a second dynamic range. Therefore, the method proposed in this application can perform overall content enhancement processing on the target image to obtain a content-enhanced image. It can also perform format transformation processing on the local region image (i.e., the first object image) where the target object is located in the target image, while keeping the image content of the first object image unchanged to obtain a format-transformed image. Furthermore, the content-enhanced image and the format-transformed image can be fused to obtain an optimized image of the target image. In the optimized image, the format-transformed image retains some of the original features of the target object in the target image, preventing the target object in the target image from being over-enhanced. Moreover, the content-enhanced image can also achieve overall enhancement of the image content of the target image. Therefore, a superior optimization processing of the target image is achieved, resulting in a superior optimized image of the target image.

[0224] Please see Figure 7 , Figure 7 This is a flowchart illustrating a method for generating image color weights provided in this application. Please refer to... Figure 7 The method may include:

[0225] Step S201: Obtain the reference pixel of the target object in the target image, and use the pixel color of the reference pixel as the reference color of the target object.

[0226] Optionally, the image processing device can obtain a reference pixel (i.e. a pixel that can characterize the color of the target object) in the target image, and can use the pixel color of the reference pixel (which may be the H component (i.e. hue) of the reference pixel) as the reference color of the target object.

[0227] For example, if the target object is a human face, the reference pixels can be pixels on the face in the target image. The number of reference pixels can be one or more. Each reference pixel has a reference color for the target object, and there is no restriction on this. The process of obtaining the reference pixels of the target object can include the following:

[0228] In the process of performing object detection on the target image to obtain the object detection box of the target object in the target image, the position of the key points of the target object in the target image can also be detected simultaneously (such as the detection model trained above can also be used to detect the position of the key points of the target object in the image). For example, the key points of the target object can include the facial key points of a person's face. These facial key points can include the key points of the left eye, right eye, left corner of the mouth, and right corner of the mouth. Then the position of the key points of the target object can include the position of the left eye key point, the right eye key point, the left corner of the mouth key point, and the right corner of the mouth key point.

[0229] Furthermore, the image processing device can obtain reference pixels of the target object by locating its key points. For example, the device can connect the left eye key point and the left corner of the mouth key point in the target image and take the pixel at the midpoint of this line as a reference pixel. This reference pixel is then taken from the left side of the face. Alternatively, by adding the coordinates of the left eye key point and the left corner of the mouth key point and then dividing by 2, the coordinates of the midpoint of the line connecting these two key points can be obtained, which is the coordinate of the reference pixel on the left side of the face.

[0230] Similarly, the image processing device can connect the key points of the right eye and the right corner of the mouth in the target image, and take the pixel at the midpoint of this line as a reference pixel for the target object. This reference pixel is then taken from the right side of the face. For example, by adding the coordinates of the right eye key point and the right corner of the mouth key point and then dividing by 2, the coordinates of the midpoint of the line connecting the two key points can be obtained, which is the coordinate of the reference pixel on the right side of the face.

[0231] Through the above process, a reference pixel is obtained on both the left and right sides of the face. This allows for consideration of the overall face color even if there are slight differences in color between the left and right sides. Alternatively, more reference pixels can be obtained from the target object (e.g., the forehead, chin) in the target image, as needed; there are no restrictions on this.

[0232] Step S202: Obtain the target color distance between the pixel color of each pixel in the first object image and the reference color of the target object.

[0233] Optionally, the image processing device can also obtain the pixel color of each pixel in the first object image, and the pixel color of each pixel in the first object image can be the H component (i.e., hue) of each pixel.

[0234] Optionally, the process by which the image processing device obtains the H component of each pixel in the first object image may include: firstly, the image processing device may perform filtering processing on the first object image (such as Gaussian filtering, or other filtering methods) to filter out image noise in the first object image and obtain a filtered object image.

[0235] Furthermore, the image processing device can convert the filtered image to the HSV color space (a color space) to obtain a color space converted image. In this HSV color space, each pixel in the color space converted image has an H component (hue component), an S component (saturation component), and a V component (luminance component). The image processing device can use the H component of each pixel in the color space converted image as the H component of each pixel at the same position in the first object image. For example, the H component of the pixel in the first row and first column of the color space converted image can be used as the H component of the pixel in the first row and first column of the first object image, and so on.

[0236] Optionally, it is understood that the reference pixel of the target object mentioned above also belongs to the pixel in the first object image, and the H component of the reference pixel of the target object can also be the H component of the reference pixel in the first object image obtained here (that is, the H component of the pixel at the same position in the color space conversion image corresponding to the reference pixel).

[0237] The image processing device can obtain the target color distance between the pixel color of each pixel in the first object image and the reference color of the target object. This target color distance can be used to reflect the difference between the pixel color of the pixel in the first object image and the reference color. The larger the target color distance, the greater the difference.

[0238] Any pixel in the first object image can be called a target pixel. The process of obtaining the target color distance between the pixel color of the target pixel and the reference color of the target object is described below.

[0239] Image processing equipment can obtain the color distance between the pixel color of the target pixel and the reference color of each reference pixel. The color distance between the target pixel and the reference color of a reference pixel can be used to take the absolute value of the difference between the hue value of the target pixel and the hue value of the reference pixel (so that the color distance is not negative).

[0240] Furthermore, the image processing device can take the smallest color distance (i.e., considering the smallest color difference) among the color distances between the pixel color of the target pixel and the reference color of each reference pixel as the target color distance between the pixel color of the target pixel and the reference color of the target object.

[0241] Based on the above principle, the image processing device can obtain the target color distance between the pixel color of each pixel in the first object image and the reference color of the target object. The target color distance between the pixel color of a pixel and the reference color of the target object can be called the target color distance corresponding to that pixel.

[0242] Step S203: Generate image color weights based on the target color distance between the pixel color of each pixel in the first object image and the reference color of the target object.

[0243] Optionally, the image processing device may obtain a color weight mapping function, which may be a pre-set, suitable function for generating color weights.

[0244] The image processing device can use this color weight mapping function to map the target color distance between each pixel in the first object image and the reference color of the target object, thereby generating the color weight of each pixel in the first object image.

[0245] It is understandable that when using this color weight mapping function to map the target color distance to the corresponding color weight, the smaller the target color distance, the larger the mapped color weight, and vice versa.

[0246] This is because the smaller the target color distance between a pixel's color and the reference color, the smaller the color difference between the pixel's color and the reference color. This increases the probability that the pixel is a target object pixel (such as a face pixel) in the first object image. A higher probability of it being a target object pixel allows for a larger color weight, enabling greater preservation of the pixel's color in subsequent processing. As shown in the following formula, the color weight mapping function can be (but is not limited to, and can also be other suitable functions) the following functions:

[0247] e (-γ×d) (4)

[0248] Where e is a natural constant (a transcendental number), γ represents an adjustable parameter; the larger γ is, the smaller the mapped color weight, and the weaker the fusion strength of the subsequent color weight on the format-transformed image. γ can be adjusted according to the actual application scenario; a suggested value is 2. d represents the position where the target color distance can be substituted. For example, the target color distance between the pixel color of the target pixel and the reference color can be substituted into d in the color weight mapping function to obtain the color weight of the target pixel.

[0249] The dimensions of the first object image and the format transformation image are the same. There is a one-to-one correspondence between pixels with the same pixel position in the first object image and the format transformation image. The color weight of any pixel in the first object image is the color weight of the corresponding pixel in the format transformation image.

[0250] For example, the color weight of the pixel in the first row and first column of the first object image can be the same as the color weight of the pixel in the first row and first column of the format-transformed image, and the color weight of the pixel in the second row and second column of the first object image can be the same as the color weight of the pixel in the second row and second column of the format-transformed image, and so on.

[0251] The image color weight can be obtained by transforming the color weight of each pixel in the image. This image color weight contains the color weight of each pixel in the transformed image.

[0252] Please see Figure 8 , Figure 8 This is a schematic diagram illustrating a scenario for generating image color weights, as provided in this application. Figure 8As shown, assume that the first object image contains pixels s1 to s9, where pixel s1 corresponds to a target color distance j1, pixel s2 corresponds to a target color distance j2, pixel s3 corresponds to a target color distance j3, pixel s4 corresponds to a target color distance j4, pixel s5 corresponds to a target color distance j5, pixel s6 corresponds to a target color distance j6, pixel s7 corresponds to a target color distance j7, pixel s8 corresponds to a target color distance j8, and pixel s9 corresponds to a target color distance j9.

[0253] The image processing device can use a color weight mapping function to map the target color distance j1 to obtain the color weight q1 of pixel s1, the target color distance j2 to obtain the color weight q2 of pixel s2, the target color distance j3 to obtain the color weight q3 of pixel s3, the target color distance j4 to obtain the color weight q4 of pixel s4, the target color distance j5 to obtain the color weight q5 of pixel s5, the target color distance j6 to obtain the color weight q6 of pixel s6, the target color distance j7 to obtain the color weight q7 of pixel s7, the target color distance j8 to obtain the color weight q8 of pixel s8, and the target color distance j9 to obtain the color weight q9 of pixel s9.

[0254] The image color weights can be obtained by using the color weights q1 to q9 mentioned above, and these image color weights can be represented as a weight matrix.

[0255] By using the image color weights obtained in this application, the original color features of the target object in the target image can be preserved to a greater extent in subsequent processing, thereby improving the optimization effect on the target object in the target image.

[0256] Please see Figure 9 , Figure 9 This is a flowchart illustrating a method for generating image position weights provided in this application. Please refer to... Figure 9 The method may include:

[0257] Step S301: Obtain the first image length of the format-transformed image in the first direction, and generate a first Gaussian distribution based on the first image length.

[0258] Optionally, the image processing device can obtain the image length of the format-transformed image in a first direction. This image length in the first direction can be referred to as the first image length. The first direction can be horizontal (e.g., the horizontal direction), and the first image length can be the width of the format-transformed image. The first image length can be equal to the number of pixels in the first direction of the format-transformed image, that is, it can be equal to the number of pixels in the horizontal direction of the format-transformed image (e.g., the total number of columns of pixels in the format-transformed image).

[0259] In a format-transformed image, each pixel can have a corresponding index value in the first direction. The index value of a pixel in the first direction is the column number minus 1, starting from column 1. For example, if a pixel is in the first column of the format-transformed image, its index value in the first direction can be 0; if a pixel is in the fifth column, its index value in the first direction can be 4. That is, the index values ​​of each pixel in the first direction can start from 0 (the index values ​​of each pixel are discrete). The index value of each pixel in the first direction reflects its position in the first direction of the format-transformed image.

[0260] In other words, the index value of the first pixel (or the first column of pixels) in the first direction of the format-transformed image is 0, the index value of the second pixel (or the second column of pixels) in the first direction of the format-transformed image is 1, and so on.

[0261] The image processing device can generate a Gaussian distribution for optimizing the format-transformed image from a first direction using the aforementioned first image length. This Gaussian distribution in the first direction can be referred to as the first Gaussian distribution. The process of generating the first Gaussian distribution may include:

[0262] First, the image processing device can generate a first initial Gaussian distribution based on the first image length. The mean (i.e., expected value) of this first initial Gaussian distribution can be called the first Gaussian mean. This first Gaussian mean can be half the length of the first image, that is, the first Gaussian mean can be equal to half the length of the first image (i.e., 1 / 2). This first initial Gaussian distribution can have independent and dependent variables. The independent variable of the first initial Gaussian distribution can be called the first independent variable, and the dependent variable of the first initial Gaussian distribution can be called the first dependent variable. The value range of the first independent variable can include the index value of each pixel in the first direction of the format-transformed image. For example, the value range of the first independent variable can be [0, W]. R -1],W RW represents the length of the first image, where 0 can represent the index of the first pixel in the first direction of the format-transformed image. R -1 can represent the index value of the last pixel in the first direction of the format-transformed image.

[0263] By setting the first Gaussian mean to half the length of the first image, the center of the first initial Gaussian distribution can be located in the middle of the width of the format-transformed image. This means that when the first independent variable takes the index value of each pixel in the first direction of the format-transformed image, the index value of each pixel in the first direction can be symmetrically distributed on both sides of the center of the first initial Gaussian distribution.

[0264] The first initial Gaussian distribution also has a standard deviation. Optionally, the standard deviation can be equal to half the length of the first image, or it can be adjusted to other values. The larger the standard deviation is set, the smoother (i.e. less intense) the function value of the first initial Gaussian distribution will change.

[0265] Furthermore, the image processing device can perform normalization processing (such as normalization) on the first initial Gaussian distribution to obtain the first Gaussian distribution. This process may include:

[0266] The image processing device can obtain a reference value for the first independent variable, which can be referred to as the first reference value. This first reference value can be used to set the spatial range for smooth transition and fusion of the format-transformed image in the first direction. This first reference value is not equal to the first Gaussian mean; it can be set according to the actual application scenario. For example, the first reference value could be βW. R W R The first image length is β, which is an adjustable parameter. The adjustable range of β can be [0, 1 / 2]. It is used to adjust the range of the spatial transition region in the first direction and the intensity of the transition. The larger β is, the larger the range of the spatial transition region and the less intense the transition, resulting in a smoother transition. For example, β can be 1 / 6.

[0267] Furthermore, the image processing device can obtain the value of the first dependent variable (which can be called the value of the first variable) when the first independent variable takes the value of the first reference value. That is, the first initial Gaussian distribution can be a one-dimensional Gaussian function, and the value of the first variable can be the function value obtained after taking the first dependent variable as the first reference value (i.e., the value of the first dependent variable).

[0268] The image processing device can normalize the value of the first dependent variable within a first range of values ​​in the first initial Gaussian distribution to a first constant value, which can be equal to 1. The image processing device can also normalize the value of the first dependent variable within a second range of values ​​in the first initial Gaussian distribution to a first target dependent variable, thereby obtaining the first Gaussian distribution.

[0269] The first value range is the range in which the value of the first dependent variable in the first initial Gaussian distribution is greater than or equal to the value of the first variable (i.e., the sharp point in the middle part of the first initial Gaussian distribution, i.e., the peak). The second value range is the range in which the value of the first dependent variable in the first initial Gaussian distribution is less than the value of the first variable (i.e., the curved parts on both sides of the first initial Gaussian distribution excluding the sharp point in the middle part).

[0270] Optionally, the first target dependent variable can be obtained by taking the values ​​of the first dependent variable and the first variable. For example, the first target dependent variable can be the ratio between the values ​​of the first dependent variable and the first variable (such as the dependent variable obtained by dividing the first dependent variable by the value of the first variable).

[0271] The first Gaussian distribution obtained through the above process can be such that the function value in the middle range (i.e., the first value range) is 1, and the function value in the two outer ranges (i.e., the second value range) is the value of the first target dependent variable. The function value in the two outer ranges is smaller as it gets closer to the boundary (i.e. the edge), and then gradually approaches 0.

[0272] Therefore, by using the first Gaussian distribution, the position weight of pixels in the middle of the format-transformed image in the first direction can be increased (e.g., equal to 1), while the position weight of pixels closer to the edges in the first direction can be decreased (e.g., smoothly transitioning from 1 to 0). This allows for smooth fusion of the format-transformed image in the first direction, preventing a noticeable boundary between the target object or fused object image and other parts of the optimized image in the first direction. This achieves a smooth transition of the local region image where the target object is located (i.e., transitionally optimizing the edge parts of the local region image where the target object is located (e.g., the format-transformed image)). As shown in the following formula:

[0273]

[0274] Where x is the first independent variable of the first initial Gaussian distribution, and G wn (x) is the first Gaussian distribution (and also has the first independent variable), G w (x) is the first initial Gaussian distribution, βW R As the first reference value, Gw (βW R That is, setting x to the value βW R The value of the first dependent variable (i.e., G) w The value of (x), which is the value of the first variable mentioned above, G w (βW R Let W be a point in the first initial Gaussian distribution. R G represents the length of the first image. w (x) <G w (βW R The range corresponding to ) is the second range of values ​​mentioned above, G w (x)≥G w (βW R The range corresponding to ) is the first value range mentioned above.

[0275] Please see Figure 10 , Figure 10 This is a schematic diagram of a first initial Gaussian distribution provided in this application. For example... Figure 10 As shown, when the first independent variable takes the value At that time, the first dependent variable of the first initial Gaussian distribution takes the maximum value (i.e., the peak value), which is located at the center of the first initial Gaussian distribution.

[0276] The value of the first independent variable is βW R ~(1-β)W R When the range of values ​​is defined as follows, the value of the first dependent variable is greater than or equal to the value of the first variable; that is, the range of values ​​for the first dependent variable is defined as the value of the first dependent variable. The range. For example, if β equals 1 / 6, then βW R For W R / 6,(1-β)W R 5W R / 6.

[0277] Furthermore, the first independent variable takes values ​​from 0 to βW. R and (1-β)W R ~W R When the value is within the range of -1, the value of the first dependent variable is less than the value of the first variable, that is, the second range is when the value of the first dependent variable is less than G. w (βW R ) range.

[0278] Step S302: Obtain the second image length of the format-transformed image in the second direction, and generate a second Gaussian distribution based on the second image length.

[0279] Optionally, the principle of generating the second Gaussian distribution is the same as that of generating the first Gaussian distribution, except that the direction of consideration is different. The second Gaussian distribution considers the optimization of the format-transformed image in the second direction.

[0280] Similarly, an image processing device can obtain the image length of a format-transformed image in a second direction. This image length in the second direction can be called the second image length. This second direction can be vertical (e.g., the vertical direction), and the second image length can be the height of the format-transformed image. This second image length can be equal to the number of pixels in the second direction of the format-transformed image, that is, it can be equal to the number of pixels in the vertical direction of the format-transformed image (e.g., the total number of rows of pixels in the format-transformed image).

[0281] In a format-transformed image, each pixel can also have a corresponding index value in the second direction. The index value of a pixel in the second direction is the row number it belongs to minus 1, starting from row 1. For example, if a pixel is in the first row of the format-transformed image, its index value in the second direction can be 0; if a pixel is in the fifth row, its index value in the second direction can be 4. That is, the index values ​​of each pixel in the second direction can start from 0 (the index values ​​of each pixel are discrete). The index value of each pixel in the second direction reflects its position in the second direction of the format-transformed image.

[0282] In other words, the index value of the first pixel (or the first row of pixels) in the second direction of the format-transformed image is 0 in the second direction, and the index value of the second pixel (or the second row of pixels) in the second direction of the format-transformed image is 1 in the first direction, and so on.

[0283] The image processing device can generate a Gaussian distribution for optimizing the format-transformed image from a second direction using the aforementioned second image length. This Gaussian distribution in the second direction can be referred to as the second Gaussian distribution. The process of generating the second Gaussian distribution may include:

[0284] First, the image processing device can generate a second initial Gaussian distribution based on the second image length. The mean (i.e., expected value) of this second initial Gaussian distribution can be called the second Gaussian mean. This second Gaussian mean can be half the length of the second image, that is, the second Gaussian mean can be equal to half the length of the second image (i.e., 1 / 2). This second initial Gaussian distribution can have independent and dependent variables. The independent variable of the second initial Gaussian distribution can be called the second independent variable, and the dependent variable of the second initial Gaussian distribution can be called the second dependent variable. The value range of the second independent variable can include the index values ​​of each pixel in the second direction of the format-transformed image. For example, the value range of the second independent variable can be [0, H]. R -1],H R H represents the second image length, 0 represents the index of the first pixel in the second direction of the format-transformed image, and H represents the second image length. R -1 represents the index of the last pixel in the second direction of the format-transformed image.

[0285] By setting the second Gaussian mean to half the length of the second image, the center of the second initial Gaussian distribution can be located at the middle of the height of the format-transformed image. This means that when the second independent variable takes the index value of each pixel in the second direction of the format-transformed image, the index value of each pixel in the second direction can be symmetrically distributed on both sides of the center of the second initial Gaussian distribution.

[0286] The second initial Gaussian distribution also has a standard deviation. Optionally, the standard deviation can be equal to half the length of the second image, or it can be adjusted to other values. The larger the standard deviation is set, the smoother (i.e. less intense) the function value of the second initial Gaussian distribution will change.

[0287] Furthermore, the image processing device can normalize the second initial Gaussian distribution (e.g., by normalization) to obtain the second Gaussian distribution. This process may include:

[0288] Image processing equipment can obtain a reference value for the second independent variable, which can be referred to as the second reference value. This second reference value can also be used to set the spatial range for smooth transition fusion of the format-transformed image in the second direction. This second reference value is not equal to the second Gaussian mean; it can be set according to the actual application scenario. For example, the second reference value could be βH. R H RThe second image length is β, which is the adjustable parameter mentioned above. The adjustable range of β can be [0, 1 / 2], which is used to adjust the range of the spatial transition region in the second direction and the intensity of the transition. The larger β is, the larger the range of the spatial transition region is, the less intense the transition is, and the smoother the transition is. For example, β can be 1 / 6.

[0289] Furthermore, the image processing device can obtain the value of the second dependent variable (which can be called the value of the second variable) when the second independent variable takes the value of the second reference value. That is, the second initial Gaussian distribution can also be a one-dimensional Gaussian function, and the value of the second variable can be the function value obtained after taking the second dependent variable as the second reference value (i.e., the value of the second dependent variable).

[0290] The image processing device can normalize the value of the second dependent variable within the third range of the second initial Gaussian distribution to a second constant value, which can be equal to 1. The image processing device can also normalize the value of the second dependent variable within the fourth range of the second initial Gaussian distribution to a second target dependent variable, thereby obtaining the second Gaussian distribution.

[0291] The third range of values ​​is the range in which the value of the second dependent variable in the second initial Gaussian distribution is greater than or equal to the value of the second variable (i.e., the peak of the middle part of the second initial Gaussian distribution). The fourth range of values ​​is the range in which the value of the second dependent variable in the second initial Gaussian distribution is less than the value of the second variable (i.e., the curves on both sides of the second initial Gaussian distribution excluding the peak of the middle part).

[0292] Optionally, the second target dependent variable can be obtained by taking the values ​​of the second dependent variable and the second variable. For example, the second target dependent variable can be the ratio between the values ​​of the second dependent variable and the second variable (such as the dependent variable obtained by dividing the second dependent variable by the value of the second variable).

[0293] The second Gaussian distribution obtained through the above process can also be that the function value in the middle range (i.e., the third range) is 1, and the function value in the two outer ranges (i.e., the fourth range) is the value of the second objective dependent variable. The function value in the two outer ranges is smaller as it gets closer to the boundary (i.e. the edge), and then gradually approaches 0.

[0294] Therefore, by using a second Gaussian distribution, the position weights of pixels in the middle of the format-transformed image in the second direction can be increased (e.g., equal to 1), while the position weights of pixels closer to the edges in the second direction can be decreased (e.g., smoothly transitioning from 1 to 0). This allows for smooth fusion of the format-transformed image in the second direction, preventing a noticeable boundary between the target object or fused object image and other parts of the optimized image in the second direction. This achieves a smooth transition of the local region image where the target object is located (i.e., transitional optimization of the edge parts of the local region image where the target object is located). As shown in the following formula:

[0295]

[0296] Where y is the second independent variable of the second initial Gaussian distribution, and G hn (y) is the second Gaussian distribution (and also has a second independent variable), G h (y) is the second initial Gaussian distribution, βH R As the second reference value, G h (βH R That is, setting y to the value βH R The value of the second dependent variable (i.e., G) h The value of (y), which is the value of the second variable mentioned above, G h (βH R H is a point in the second initial Gaussian distribution. R H represents the length of the second image. h (y) <G h (βH R The range corresponding to ) is the fourth value range mentioned above, G h (y)≥G h (βH R The range corresponding to ) is the third value range mentioned above.

[0297] Please see Figure 11 , Figure 11 This is a schematic diagram of a second initial Gaussian distribution provided in this application. Figure 11 As shown, when the second independent variable takes the value When the second dependent variable of the second initial Gaussian distribution reaches its maximum value (i.e., peak value), the peak value is located at the center of the second initial Gaussian distribution.

[0298] The value of the second independent variable is βH. R ~(1-β)H RWhen the value of the third dependent variable is within the range, the value of the second dependent variable is greater than or equal to the value of the first dependent variable; that is, the range of the third dependent variable is the value of the first dependent variable. The range. For example, if β equals 1 / 6, then βH R For H R / 6,(1-β)H R 5H R / 6.

[0299] Furthermore, the second independent variable takes values ​​from 0 to βH. R and (1-β)H R ~H R When the value is within the range of -1, the value of the second dependent variable is less than the value of the second variable, that is, the fourth value range is when the value of the second dependent variable is less than G. h (βH R ) range.

[0300] Step S303: Generate image position weights based on the first Gaussian distribution and the second Gaussian distribution.

[0301] Optionally, as can be seen from the above process, each pixel in the format-transformed image has an index value in a first direction and an index value in a second direction. The range of the first independent variable can include the index values ​​of each pixel in the first direction of the format-transformed image, and the range of the second independent variable can include the index values ​​of each pixel in the second direction of the format-transformed image. The following describes the process of generating image position weights using a first Gaussian distribution and a second Gaussian distribution:

[0302] The image processing device can multiply the first Gaussian distribution and the second Gaussian distribution to obtain the target Gaussian distribution, which can be a two-dimensional Gaussian distribution, as shown in the following formula:

[0303] W P (x,y)=G wn (x)×G hn (y) (7)

[0304] Among them, W P (x,y) represents the target Gaussian distribution, G wn (x) is a first Gaussian distribution, G hn (y) is a second Gaussian distribution.

[0305] Since the first Gaussian distribution has a first independent variable and the second Gaussian distribution has a second independent variable, the target Gaussian distribution can simultaneously have both a first and a second independent variable. Therefore, the dependent variable of the target Gaussian distribution can be the product of the dependent variables of the first and second Gaussian distributions.

[0306] The image processing device can take the first independent variable in the target Gaussian distribution as the index value of each pixel in the format-transformed image in the first direction, and take the second independent variable in the target Gaussian distribution as the index value of each pixel in the format-transformed image in the second direction, so as to obtain the position weight of each pixel in the format-transformed image.

[0307] For example, an image processing device can take the first independent variable of the target Gaussian distribution as the index value of any pixel in the format-transformed image in the first direction, and take the second independent variable of the target Gaussian distribution as the index value of that pixel in the second direction. The function value of the target Gaussian distribution obtained in this way (which can be understood as the value of the dependent variable of the target Gaussian distribution at this time) can be the position weight of that pixel.

[0308] Optionally, the first Gaussian distribution in the first direction and the second Gaussian distribution in the second direction of this application can also be designed as other distribution functions that can achieve the same effect, so that the designed distribution function in the first direction (similar to the first Gaussian distribution) can make the position weight of the pixel in the middle position in the first direction of the format-transformed image greater, and then the position weight of the pixel closer to the edge will decrease smoothly (such as the position weight of the outermost pixel will eventually decrease to approach 0), and make the designed distribution function in the second direction (similar to the second Gaussian distribution) can make the position weight of the pixel in the middle position in the second direction of the format-transformed image greater, and then the position weight of the pixel closer to the edge will also decrease smoothly (such as the position weight of the outermost pixel will eventually decrease to approach 0).

[0309] Therefore, the positional weights of each pixel in the format-transformed image constitute the image positional weights, which in turn include the positional weights of each pixel in the format-transformed image.

[0310] By using the image position weights generated in this application, the position weights of pixels located in the middle of the format-transformed image can be larger (e.g., equal to 1), and the position weights of pixels located at the edges of the format-transformed image can be gradually reduced to approach 0 (e.g., gradually decreasing from 1 to 0). This allows more image features (e.g., image color) located in the middle of the format-transformed image to be preserved, while fewer image features located at the edges are preserved. This enables the subsequent fusion of more image features from the corresponding edge positions in the second object image into the image features located at the edges of the format-transformed image (i.e., fusing more image features from the second object image at the edge positions). Consequently, the images at the edge positions of the fused object image in the optimized image can transition more naturally and smoothly with other images in the optimized image (e.g., images other than the second object image in the content-enhanced image), ensuring that the fused object image or target object does not exhibit a visually distinct boundary effect in the optimized image.

[0311] Please see Figure 12 , Figure 12 This is a flowchart illustrating a method for generating image color difference weights provided in this application. Please refer to... Figure 12 The method may include:

[0312] Step S401: Obtain the second object image in the content enhancement image; the second object image refers to the local region image of the location of the target object in the content enhancement image.

[0313] Optionally, the image processing device may acquire a second object image, which is a local region image of the target object's location in the content-enhanced image. The dimensions of the target image and the content-enhanced image may be the same, the dimensions of the first object image and the second object image may also be the same, and the location of the first object image in the target image and the location of the second object image in the content-enhanced image may be the same.

[0314] Step S402: Calculate the difference between the pixel values ​​of the same pixel positions in the format-transformed image and the second object image to obtain the target pixel difference between each pixel in the format-transformed image and the pixel at the same pixel position in the second object image.

[0315] Optionally, both the format-transformed image and the second object image can be RGB images. The pixels in the format-transformed image can have pixel values ​​in the three color channels: R, G, and B. The second object image can also have pixel values ​​in the three color channels: R, G, and B (i.e., the components in these three color channels).

[0316] An image processing device can subtract the pixel values ​​of a pixel in each color channel of the format-converted image from the pixel values ​​of a pixel at the same position in the second object image in each color channel (i.e., take the difference) to obtain the difference between the pixel values ​​of a pixel in each color channel of the format-converted image and the pixel values ​​of a pixel at the same position in the second object image in each color channel (which can be called the pixel difference, and this pixel difference can be used to represent the color difference between pixels).

[0317] The processing principle (i.e., difference processing) for pixel values ​​in each of the above color channels can be the same and performed independently. Each pixel in the format-transformed image has a pixel difference between it and the corresponding pixel at the same pixel position in the second object image in each color channel. Therefore, the maximum value of the three pixel differences between the pixel in the three color channels and the pixel at the same pixel position in the second image (i.e., considering the maximum color difference) can be used as the target pixel difference between the pixel in the three color channels and the pixel at the same pixel position in the second object image.

[0318] For example, the pixel value of pixel s1 in the first row and first column of the format-transformed image in the R channel can be subtracted from the pixel value of pixel s2 in the first row and first column of the second object image in the R channel, and the absolute value can be taken to obtain the pixel difference c1 between the pixel values ​​of pixel s1 and pixel s2 in the R channel. This pixel difference c1 can be used to represent the color difference between pixel s1 and pixel s2 in the R channel.

[0319] Similarly, the pixel value of pixel s1 in the first row and first column of the format-transformed image in the G channel can be subtracted from the pixel value of pixel s2 in the first row and first column of the second object image in the G channel, and the absolute value can be taken to obtain the pixel difference c2 between the pixel values ​​of pixel s1 and pixel s2 in the G channel. This pixel difference c2 can be used to represent the color difference between pixel s1 and pixel s2 in the G channel.

[0320] Similarly, the pixel value of pixel s1 in the first row and first column of the format-transformed image in the B channel can be subtracted from the pixel value of pixel s2 in the first row and first column of the second object image in the B channel, and the absolute value can be taken to obtain the pixel difference c3 between the pixel values ​​of pixel s1 and pixel s2 in the B channel. This pixel difference c3 can be used to represent the color difference between pixel s1 and pixel s2 in the B channel.

[0321] Therefore, the pixel difference with the largest value among the pixel differences c1, c2 and c3 can be taken as the target pixel difference between pixel s1 and pixel s2. This target pixel difference takes into account the maximum pixel color difference between pixel s1 and pixel s2. This target pixel difference can be called the target pixel difference corresponding to pixel s1 in the format-transformed image.

[0322] Step S403: Using the color difference weight mapping function, the target pixel difference value corresponding to each pixel in the format-transformed image is mapped to generate the color difference weight of each pixel in the format-transformed image.

[0323] Optionally, the image processing device may obtain a color difference weighting mapping function, which may be a pre-set appropriate function for generating color difference weights.

[0324] Image processing devices can use this color difference weighting mapping function to map the target pixel difference values ​​corresponding to each pixel in the format-transformed image, thereby generating the color difference weights of each pixel in the format-transformed image.

[0325] Understandably, when using this color difference weight mapping function to map the target pixel difference to the corresponding color difference weight, the smaller the target pixel difference, the smaller the color difference between the pixel point to which the target pixel difference belongs in the format-transformed image and the pixel point at the same position in the second object image. Therefore, the mapped color difference weight can be smaller. Conversely, the larger the target pixel difference, the larger the color difference between the pixel point to which the target pixel difference belongs in the format-transformed image and the pixel point at the same position in the second object image. Therefore, the mapped color difference weight can be larger.

[0326] This is because the smaller the color difference between two pixels, the smaller the color difference weight can be used to adjust the color difference between the two pixels. Conversely, the larger the color difference between two pixels, the larger the color difference weight needs to be used to adjust the color difference between the two pixels. This allows a balance to be achieved between the colors of the pixels in the subsequent format-transformed image (represented by the pixel values ​​in the color channels) and the colors of the pixels in the second object image (represented by the pixel values ​​in the color channels). This reduces the overall color difference between the format-transformed image and the second object image, ensuring overall consistency between the fused object image obtained by fusing the format-transformed image and the other images in the content-enhanced image, excluding the second object image. This makes the effect (such as visual effect) between the original features in the format-transformed image and the content-enhanced features in the second object image more balanced, thus ensuring that the optimized image of the final target image also has overall visual consistency.

[0327] As shown in the following formula, the color difference weighting mapping function f(z) can be (but is not limited to, and can also be other suitable functions) the following functions:

[0328]

[0329] The color difference weighting mapping function designed above is obtained by shifting the sigmoid function (a function of an S-shaped curve) downwards by 0.5 and then magnifying it by a factor of 2. In practical applications, other color difference weighting mapping functions that can achieve the same effect (such as sine functions or linear functions) can also be designed. e is a natural constant (belonging to a transcendental number), and z is the position where the target pixel difference can be substituted. For example, the color difference weight of any pixel in the format-transformed image can be obtained by substituting the target pixel difference into z in the color difference weighting mapping function.

[0330] The color difference weight of an image can be obtained by converting the color difference weight of each pixel in the format-converted image. This color difference weight includes the color difference weight of each pixel in the format-converted image.

[0331] Please see Figure 13 , Figure 13 This is a schematic diagram illustrating a scenario for generating image color difference weights, as provided in this application. Figure 13As shown, assuming the format-transformed image contains pixels s11 to s99, pixel s11 corresponds to the target pixel difference c1, pixel s22 corresponds to the target pixel difference c2, pixel s33 corresponds to the target pixel difference c3, pixel s44 corresponds to the target pixel difference c4, pixel s55 corresponds to the target pixel difference c5, pixel s66 corresponds to the target pixel difference c6, pixel s77 corresponds to the target pixel difference c7, pixel s88 corresponds to the target pixel difference c8, and pixel s99 corresponds to the target pixel difference c9.

[0332] Image processing equipment can use a color difference weighting mapping function to map the target pixel difference c1 to obtain the color difference weight q11 of pixel s11; use the same function to map the target pixel difference c2 to obtain the color difference weight q22 of pixel s22; use the same function to map the target pixel difference c3 to obtain the color difference weight q33 of pixel s33; and use the same function to map the target pixel difference c4 to obtain the color difference weight q44 of pixel s44. The color difference weighting mapping function can then be used to map the target pixel difference... The color difference weight q55 of pixel s55 is obtained by mapping c5. The color difference weight mapping function is then used to map the target pixel difference c6 to obtain the color difference weight q66 of pixel s66. The color difference weight mapping function is then used to map the target pixel difference c7 to obtain the color difference weight q77 of pixel s77. The color difference weight mapping function is then used to map the target pixel difference c8 to obtain the color difference weight q88 of pixel s88. The color difference weight mapping function is then used to map the target pixel difference c9 to obtain the color difference weight q99 of pixel s99.

[0333] The image color difference weights can be obtained by using the color difference weights q11 to q99, and these image color difference weights can be represented as a weight matrix.

[0334] By using the image color difference weights obtained in this application, the optimized image of the target image can be made visually consistent in the subsequent optimization, thereby improving the overall optimization effect and visual effect of the target image.

[0335] Please see Figure 14 , Figure 14 This is a schematic diagram of the structure of an image processing apparatus provided in this application. The image processing apparatus 140 may include: an acquisition module 1401, an enhancement module 1402, a transformation module 1403, and a fusion module 1404.

[0336] The acquisition module 1401 is used to acquire a target image containing a target object; the target image is an image with a first dynamic range, and the target image contains a first object image, which refers to a local area image of the location of the target object in the target image;

[0337] Enhancement module 1402 is used to enhance the image content of the target image to obtain a content-enhanced image;

[0338] The transformation module 1403 is used to transform the image format of the first object image to obtain a format-transformed image; the format-transformed image is an image with a second dynamic range, the second dynamic range is greater than the first dynamic range, and the image content of the format-transformed image is the same as the image content of the first object image.

[0339] The fusion module 1404 is used to fuse the format-transformed image and the content-enhanced image to obtain an optimized image of the target image; the optimized image is an image with a second dynamic range.

[0340] Optionally, the fusion module 1404 performs fusion processing on the format-transformed image and the content-enhanced image to obtain an optimized image of the target image in the following ways:

[0341] Obtain the first fusion weight of the format-transformed image;

[0342] Based on the first fusion weight, the format-transformed image and the content-enhanced image are fused to obtain an optimized image.

[0343] Optionally, the first fusion weight is obtained based on any of the following weights, or the first fusion weight is obtained by fusing at least two of the following weights:

[0344] Image color weighting for format-transformed images; image color weighting is obtained based on the difference between the reference color of the target object and the image color of the first object image;

[0345] Image position weights for format-transformed images; image position weights are obtained based on the position of each pixel in the format-transformed image.

[0346] Image color difference weighting for format-transformed images; image color difference weighting is obtained based on the pixel difference between the format-transformed image and the second object image, which refers to the local region image of the target object in the content-enhanced image.

[0347] Optionally, the image color of the first object image includes the pixel color of each pixel in the first object image; the process of generating image color weights by the fusion module 1404 includes:

[0348] Obtain the reference pixel of the target object in the target image, and use the pixel color of the reference pixel as the reference color of the target object;

[0349] Obtain the target color distance between the pixel color of each pixel in the first object image and the reference color of the target object;

[0350] Image color weights are generated based on the target color distance between the pixel color of each pixel in the first object image and the reference color of the target object.

[0351] Optionally, any pixel in the first object image is the target pixel, and there are one or more reference pixels, each of which has a reference color of the target object.

[0352] The fusion module 1404 obtains the target color distance between the pixel color of each pixel in the first object image and the reference color of the target object in the following ways:

[0353] Obtain the color distance between the pixel color of the target pixel and the reference color of each reference pixel;

[0354] The smallest color distance among the color distances between the target pixel's pixel color and the reference colors of one or more reference pixels is taken as the target color distance between the target pixel's pixel color and the reference color of the target object.

[0355] Optionally, the fusion module 1404 generates image color weights based on the target color distance between the pixel color of each pixel in the first object image and the reference color of the target object, including:

[0356] A color weight mapping function is used to map the target color distance between each pixel in the first object image and the reference color of the target object, thereby generating the color weight of each pixel in the first object image.

[0357] The format-transformed image has the same dimension as the first object image. There is a one-to-one correspondence between pixels in the first object image and the format-transformed image that have the same pixel position. The color weight of any pixel in the first object image is the color weight of the corresponding pixel in the format-transformed image. The image color weight includes the color weight of each pixel in the format-transformed image.

[0358] Optionally, the process of generating image location weights by the fusion module 1404 includes:

[0359] Obtain the first image length of the format-transformed image in the first direction, and generate a first Gaussian distribution based on the first image length;

[0360] Obtain the second image length of the format-transformed image in the second direction, and generate a second Gaussian distribution based on the second image length;

[0361] Image position weights are generated based on the first and second Gaussian distributions.

[0362] Optionally, the fusion module 1404 generates a first Gaussian distribution based on the first image length in the following ways:

[0363] A first initial Gaussian distribution is generated based on the first image length; the first initial Gaussian distribution has a first Gaussian mean, which is half the length of the first image.

[0364] Normalize the initial Gaussian distribution to obtain the first Gaussian distribution.

[0365] Optionally, the first initial Gaussian distribution has a first independent variable and a first dependent variable; the fusion module 1404 normalizes the first initial Gaussian distribution to obtain the first Gaussian distribution in the following ways:

[0366] Obtain the first reference value of the first independent variable, and obtain the first variable value of the first dependent variable when the first independent variable takes the first reference value. The first reference value is not equal to the first Gaussian mean.

[0367] The values ​​of the first dependent variable within the first range of the first initial Gaussian distribution are normalized to the first constant value, and the values ​​of the first dependent variable within the second range of the first initial Gaussian distribution are normalized to the first target dependent variable, thus obtaining the first Gaussian distribution;

[0368] The first value range is the range in which the value of the first dependent variable in the first initial Gaussian distribution is greater than or equal to the value of the first variable; the second value range is the range in which the value of the first dependent variable in the first initial Gaussian distribution is less than the value of the first variable; and the first target dependent variable is obtained based on the first dependent variable and the value of the first variable.

[0369] Optionally, the fusion module 1404 generates the second Gaussian distribution based on the second image length in the following ways:

[0370] A second initial Gaussian distribution is generated based on the second image length; the second initial Gaussian distribution has a second Gaussian mean, which is half the length of the second image.

[0371] The second initial Gaussian distribution is normalized to obtain the second Gaussian distribution.

[0372] Optionally, the second initial Gaussian distribution has a second independent variable and a second dependent variable; the fusion module 1404 normalizes the second initial Gaussian distribution to obtain the second Gaussian distribution in the following ways:

[0373] Obtain the second reference value of the second independent variable, and obtain the second variable value of the second dependent variable when the second independent variable takes the second reference value. The second reference value is not equal to the second Gaussian mean.

[0374] The values ​​of the second dependent variable within the third range of the second initial Gaussian distribution are normalized to the second constant value, and the values ​​of the second dependent variable within the fourth range of the second initial Gaussian distribution are normalized to the second target dependent variable, thus obtaining the second Gaussian distribution;

[0375] The third range of values ​​is the range in which the second dependent variable takes values ​​greater than or equal to the value of the second variable in the second initial Gaussian distribution. The fourth range of values ​​is the range in which the second dependent variable takes values ​​less than the value of the second variable in the second initial Gaussian distribution. The second target dependent variable is obtained based on the values ​​of the second dependent variable and the second variable.

[0376] Optionally, the first Gaussian distribution has a first independent variable, and the second Gaussian distribution has a second independent variable. The range of values ​​for the first independent variable includes the index value of each pixel in the format-transformed image in the first direction, and the range of values ​​for the second independent variable includes the index value of each pixel in the format-transformed image in the second direction. Each pixel in the format-transformed image has an index value in the first direction and an index value in the second direction. The fusion module generates image position weights based on the first and second Gaussian distributions in the following ways:

[0377] The target Gaussian distribution is obtained by multiplying the first Gaussian distribution and the second Gaussian distribution; the target Gaussian distribution has a first independent variable and a second independent variable.

[0378] The first independent variable of the target Gaussian distribution is taken as the index value of each pixel in the format-transformed image in the first direction, and the second independent variable of the target Gaussian distribution is taken as the index value of each pixel in the format-transformed image in the second direction, so as to obtain the position weight of each pixel in the format-transformed image.

[0379] The image position weights include the position weights of each pixel in the format-transformed image.

[0380] Optionally, the process of generating image color difference weights by the fusion module 1404 includes:

[0381] Obtain the second object image in the content-enhanced image; the second object image refers to the local region image of the location of the target object in the content-enhanced image;

[0382] The pixel values ​​at the same pixel position in the format-transformed image and the second object image are subtracted to obtain the target pixel difference between each pixel in the format-transformed image and the pixel at the same pixel position in the second object image.

[0383] A color difference weighting mapping function is used to map the target pixel difference value corresponding to each pixel in the format-transformed image, thereby generating the color difference weight of each pixel in the format-transformed image.

[0384] Among them, the image color difference weight includes the color difference weight of each pixel in the format-transformed image.

[0385] Optionally, the fusion module 1404 performs fusion processing on the format-transformed image and the content-enhanced image based on the first fusion weight to obtain an optimized image, including:

[0386] The second fusion weight of the second object image is determined based on the first fusion weight; the second object image refers to the local region image of the location of the target object in the content-enhanced image;

[0387] The format-transformed image is weighted based on the first fusion weight to obtain the weighted format-transformed image;

[0388] The second object image is weighted based on the second fusion weight to obtain the weighted second object image.

[0389] The weighted format-transformed image and the weighted second object image are summed to obtain the fused object image.

[0390] The images other than the second object image in the fused object image and the content enhancement image are combined to obtain the optimized image.

[0391] Optionally, the process of the above-mentioned device 140 acquiring the first object image includes:

[0392] Perform object detection on the target image to obtain the object detection bounding box of the target object in the target image;

[0393] The object detection box is expanded on the target image to obtain the expanded detection box;

[0394] The image within the expanded detection box in the target image is used as the first object image.

[0395] Optionally, the target image belongs to a video frame in the target video that contains the target object;

[0396] The aforementioned device 140 is also used for:

[0397] An optimized video is generated based on the optimized image, representing the target video.

[0398] The video will be optimized for transmission to the video client, enabling the video client to play the optimized video.

[0399] According to one embodiment of this application, Figure 3 The steps involved in the image processing method shown can be derived from... Figure 14 The image processing apparatus 140 shown is executed by various modules. For example, Figure 3 Step S101 shown can be performed by Figure 14 The acquisition module 1401 in the middle is used to execute, Figure 3 Step S102 shown can be performed by Figure 14 The enhancement module 1402 in the middle is used to execute; Figure 3 Step S103 shown can be performed by Figure 14 The transformation module 1403 in the middle is used to perform the operation. Figure 3 Step S104 shown can be performed by Figure 14 The fusion module 1404 in the middle is used to execute it.

[0400] This application first acquires a target image containing a target object; the target image is an image with a first dynamic range, and the target image contains a first object image, which refers to a local region image of the target object's location in the target image; then, the image content of the target image can be enhanced to obtain a content-enhanced image; and the image format of the first object image can be transformed to obtain a format-transformed image; the format-transformed image is an image with a second dynamic range, the second dynamic range being greater than the first dynamic range, and the image content of the format-transformed image is the same as the image content of the first object image; furthermore, the format-transformed image and the content-enhanced image can be fused to obtain an optimized image of the target image; the optimized image is an image with a second dynamic range. Therefore, the apparatus proposed in this application can perform overall content enhancement processing on the target image to obtain a content-enhanced image, and can additionally perform format transformation processing on the local region image (i.e., the first object image) where the target object is located in the target image, while keeping the image content of the first object image unchanged during the format transformation processing to obtain a format-transformed image. Furthermore, the content-enhanced image and the format-transformed image can be fused to obtain an optimized image of the target image. In the obtained optimized image, the format-transformed image retains some of the original features of the target object in the target image, preventing the target object in the target image from being over-enhanced. Moreover, the content-enhanced image can also achieve overall enhancement of the image content of the target image. Therefore, a superior optimization processing of the target image is achieved, resulting in a superior optimized image of the target image.

[0401] According to one embodiment of this application, Figure 14The modules in the image processing apparatus 140 shown can be individually or entirely combined into one or more units, or some of the units can be further divided into multiple functionally smaller sub-units to achieve the same operation without affecting the technical effects of the embodiments of this application. The above modules are based on logical function division. In practical applications, the function of one module can also be implemented by multiple units, or the function of multiple modules can be implemented by one unit. In other embodiments of this application, the image processing apparatus 140 may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0402] According to one embodiment of this application, a general-purpose computer device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), can perform operations such as... Figure 3 The computer program for each step involved in the corresponding method shown is used to construct, as... Figure 14 The image processing apparatus 140 shown herein, and the image processing method for implementing the embodiments of this application, are described. The aforementioned computer program may be recorded on, for example, a computer-readable recording medium, loaded onto the aforementioned computing device via the computer-readable recording medium, and run therein.

[0403] Please see Figure 15 , Figure 15 This is a schematic diagram of the structure of a computer device provided in this application. For example... Figure 15 As shown, the computer device 1000 may include a processor 1001, a network interface 1004, and a memory 1005. In some embodiments, the computer device 1000 may also include a user interface 1003 and at least one communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1005 may also be at least one storage device located remotely from the aforementioned processor 1001. Figure 15 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a device control application program.

[0404] exist Figure 15 In the computer device 1000 shown, the network interface 1004 provides network communication functionality; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0405] Obtain a target image containing the target object; the target image is an image with a first dynamic range, and the target image contains a first object image, which refers to a local region image of the location of the target object in the target image;

[0406] The image content of the target image is enhanced to obtain a content-enhanced image;

[0407] The image format of the first object image is transformed to obtain a format-transformed image; the format-transformed image is an image with a second dynamic range, which is greater than the first dynamic range, and the image content of the format-transformed image is the same as the image content of the first object image.

[0408] The format-transformed image and the content-enhanced image are fused to obtain an optimized image of the target image; the optimized image is an image with the second dynamic range.

[0409] It should be understood that the computer device 1000 described in the embodiments of this application can execute the foregoing text. Figure 3 The description of the image processing method in the corresponding embodiments can also be performed as described above. Figure 14 The description of the image processing apparatus 140 in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated here.

[0410] Furthermore, it should be noted that this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the aforementioned... Figure 3 The description of the image processing method in the corresponding embodiments is already provided and will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer storage medium embodiments related to this application, please refer to the description of the method embodiments of this application.

[0411] As an example, the aforementioned computer program can be deployed and executed on a single computer device, or deployed and executed on multiple computer devices located in one location, or executed on multiple computer devices distributed across multiple locations and interconnected via a communication network. These multiple computer devices distributed across multiple locations and interconnected via a communication network can form a blockchain network.

[0412] The aforementioned computer-readable storage medium can be an internal storage unit of the computer device, such as a hard drive or memory. It can also be an external storage device, such as a plug-in hard drive, smart media card (SMC), secure digital card (SD) card, or flash card. Furthermore, the computer-readable storage medium can include both internal and external storage units of the computer device. This computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. It can also be used to temporarily store data that has been output or will be output.

[0413] This application provides a computer program product comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the aforementioned... Figure 3 The image processing method described in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer-readable storage medium embodiments related to this application, please refer to the description of the method embodiments of this application.

[0414] The terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0415] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0416] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. An image processing method, characterized in that, The method includes: Obtain a target image containing a target object; the target image is an image with a first dynamic range, and the target image contains a first object image, which refers to a local region image of the location of the target object in the target image; The image content of the target image is enhanced to obtain a content-enhanced image; The image format of the first object image is transformed to obtain a format-transformed image; the format-transformed image is an image with a second dynamic range, the second dynamic range being greater than the first dynamic range, and the image content of the format-transformed image is the same as the image content of the first object image. The format-transformed image and the content-enhanced image are fused to obtain an optimized image of the target image; the optimized image is the image with the second dynamic range.

2. The method according to claim 1, characterized in that, The process of fusing the format-transformed image and the content-enhanced image to obtain the optimized image of the target image includes: Obtain the first fusion weight of the format-transformed image; Based on the first fusion weight, the format-transformed image and the content-enhanced image are fused to obtain the optimized image.

3. The method according to claim 2, characterized in that, The first fusion weight is obtained based on any one of the following weights, or the first fusion weight is obtained by fusing at least two of the following weights: The format-transformed image has image color weights; these image color weights are obtained based on the difference between the reference color of the target object and the image color of the first object image. The image position weights of the format-transformed image; the image position weights are obtained based on the position of each pixel in the format-transformed image; The image color difference weight of the format-transformed image; the image color difference weight is obtained based on the pixel difference between the format-transformed image and the second object image, the second object image refers to the local area image of the location of the target object in the content-enhanced image.

4. The method according to claim 3, characterized in that, The image color of the first object image includes the pixel color of each pixel in the first object image; The process of generating the image color weights includes: Obtain reference pixels of the target object in the target image, and use the pixel color of the reference pixels as the reference color of the target object; Obtain the target color distance between the pixel color of each pixel in the first object image and the reference color of the target object; The image color weights are generated based on the target color distance between the pixel color of each pixel in the first object image and the reference color of the target object.

5. The method according to claim 4, characterized in that, Any pixel in the first object image is a target pixel, and there are one or more reference pixels, each of which has a reference color of the target object; The step of obtaining the target color distance between the pixel color of each pixel in the first object image and the reference color of the target object includes: Obtain the color distance between the pixel color of the target pixel and the reference color of each reference pixel; The smallest color distance among the color distances between the target pixel color and the reference colors of one or more reference pixels is taken as the target color distance between the target pixel color and the reference color of the target object.

6. The method according to claim 4, characterized in that, The step of generating the image color weight based on the target color distance between the pixel color of each pixel in the first object image and the reference color of the target object includes: A color weight mapping function is used to map the target color distance between each pixel in the first object image and the reference color of the target object, thereby generating the color weight of each pixel in the first object image. Wherein, the dimension of the format transformation image is the same as the dimension of the first object image, there is a one-to-one correspondence between the pixels with the same pixel position in the first object image and the format transformation image, the color weight of any pixel in the first object image is the color weight of the corresponding pixel in the format transformation image, and the image color weight includes the color weight of each pixel in the format transformation image.

7. The method according to claim 3, characterized in that, The process of generating the image location weights includes: Obtain the first image length of the format-transformed image in a first direction, and generate a first Gaussian distribution based on the first image length; Obtain the second image length of the format-transformed image in the second direction, and generate a second Gaussian distribution based on the second image length; The image position weights are generated based on the first Gaussian distribution and the second Gaussian distribution.

8. The method according to claim 7, characterized in that, The step of generating a first Gaussian distribution based on the first image length includes: A first initial Gaussian distribution is generated based on the length of the first image; the first initial Gaussian distribution has a first Gaussian mean, which is half the length of the first image; The first initial Gaussian distribution is normalized to obtain the first Gaussian distribution.

9. The method according to claim 8, characterized in that, The first initial Gaussian distribution has a first independent variable and a first dependent variable; the normalization process of the first initial Gaussian distribution to obtain the first Gaussian distribution includes: Obtain the first reference value of the first independent variable, and obtain the first variable value of the first dependent variable when the first independent variable takes the first reference value, wherein the first reference value is not equal to the first Gaussian mean; The values ​​of the first dependent variable within the first range of the first initial Gaussian distribution are normalized to a first constant value, and the values ​​of the first dependent variable within the second range of the first initial Gaussian distribution are normalized to a first target dependent variable, thus obtaining the first Gaussian distribution; Wherein, the first value range is the range in which the value of the first dependent variable in the first initial Gaussian distribution is greater than or equal to the value of the first variable, the second value range is the range in which the value of the first dependent variable in the first initial Gaussian distribution is less than the value of the first variable, and the first target dependent variable is obtained based on the first dependent variable and the value of the first variable.

10. The method according to claim 7, characterized in that, The step of generating a second Gaussian distribution based on the second image length includes: A second initial Gaussian distribution is generated based on the second image length; the second initial Gaussian distribution has a second Gaussian mean, which is half the length of the second image. The second initial Gaussian distribution is normalized to obtain the second Gaussian distribution.

11. The method according to claim 10, characterized in that, The second initial Gaussian distribution has a second independent variable and a second dependent variable; the normalization process of the second initial Gaussian distribution to obtain the second Gaussian distribution includes: Obtain the second reference value of the second independent variable, and obtain the second variable value of the second dependent variable when the second independent variable takes the second reference value. The second reference value is not equal to the second Gaussian mean. The values ​​of the second dependent variable within the third range of the second initial Gaussian distribution are normalized to a second constant value, and the values ​​of the second dependent variable within the fourth range of the second initial Gaussian distribution are normalized to a second target dependent variable, thus obtaining the second Gaussian distribution; The third value range is the range in which the value of the second dependent variable in the second initial Gaussian distribution is greater than or equal to the value of the second variable; the fourth value range is the range in which the value of the second dependent variable in the second initial Gaussian distribution is less than the value of the second variable; and the second target dependent variable is obtained based on the second dependent variable and the value of the second variable.

12. The method according to claim 7, characterized in that, The first Gaussian distribution has a first independent variable, and the second Gaussian distribution has a second independent variable. The value range of the first independent variable includes the index value of each pixel in the format-transformed image in the first direction, and the value range of the second independent variable includes the index value of each pixel in the format-transformed image in the second direction. Each pixel in the format-transformed image has an index value in the first direction and an index value in the second direction. Generating the image position weights based on the first Gaussian distribution and the second Gaussian distribution includes: The first Gaussian distribution and the second Gaussian distribution are multiplied to obtain the target Gaussian distribution; the target Gaussian distribution has the first independent variable and the second independent variable. The first independent variable of the target Gaussian distribution is respectively set to the index value of each pixel in the format-transformed image in the first direction, and the second independent variable of the target Gaussian distribution is respectively set to the index value of each pixel in the format-transformed image in the second direction, so as to obtain the position weight of each pixel in the format-transformed image. The image position weights include the position weights of each pixel in the format-transformed image.

13. The method according to claim 3, characterized in that, The process for generating the image color difference weights includes: Obtain a second object image from the content-enhanced image; the second object image refers to a local region image of the location of the target object in the content-enhanced image; The pixel values ​​at the same pixel position in the format-transformed image and the second object image are subtracted to obtain the target pixel difference between each pixel in the format-transformed image and the pixel at the same pixel position in the second object image. A color difference weighting mapping function is used to map the target pixel difference value corresponding to each pixel in the format-transformed image, thereby generating the color difference weight of each pixel in the format-transformed image. The image color difference weight includes the color difference weight of each pixel in the format-transformed image.

14. The method according to claim 2, characterized in that, The step of fusing the format-transformed image and the content-enhanced image based on the first fusion weight to obtain the optimized image includes: A second fusion weight is determined based on the first fusion weight; the second object image refers to a local region image of the location of the target object in the content enhancement image; The format-transformed image is weighted based on the first fusion weight to obtain a weighted format-transformed image. The second object image is weighted based on the second fusion weight to obtain the weighted second object image; The weighted format-transformed image and the weighted second object image are summed to obtain a fused object image. The optimized image is obtained by combining the fused object image and the content-enhanced image, excluding the second object image.

15. The method according to claim 1, characterized in that, The process of obtaining the image of the first object includes: Perform object detection on the target image to obtain the object detection bounding box of the target object in the target image; The object detection box is expanded on the target image to obtain an expanded detection box; The image within the expanded detection box in the target image is taken as the first object image.

16. The method according to claim 1, characterized in that, The target image belongs to a video frame in the target video that contains the target object; The method further includes: Based on the optimized image, an optimized video of the target video is generated; The optimized video is transmitted to the video client, which then plays the optimized video.

17. An image processing apparatus, characterized in that, The device includes: An acquisition module is used to acquire a target image containing a target object; the target image is an image with a first dynamic range, and the target image contains a first object image, which refers to a local area image of the location of the target object in the target image; The enhancement module is used to enhance the image content of the target image to obtain a content-enhanced image; The transformation module is used to transform the image format of the first object image to obtain a format-transformed image; the format-transformed image is an image with a second dynamic range, the second dynamic range being greater than the first dynamic range, and the image content of the format-transformed image is the same as the image content of the first object image. The fusion module is used to fuse the format-transformed image and the content-enhanced image to obtain an optimized image of the target image; the optimized image is the image with the second dynamic range.

18. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-16.

19. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method according to any one of claims 1-16.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and executed as described in any one of claims 1-16.

Images