Image processing method and device, ISP, electronic equipment and computer storage medium

Abstract

The embodiment of the invention discloses an image processing method, and the method comprises the steps: recognizing a target region of an input image in response to an operation on the input image, determining a gain image of the target region, adjusting the gain image through the operation information of the operation, obtaining an adjusted gain image, and carrying out the adjustment of the gain image through the adjusted gain image. And processing the target area to obtain an output image. The embodiment of the invention further provides an image processing device, the ISP, electronic equipment and a computer storage medium.

Description

Technical Field

[0001] The present application relates to image processing technology, and in particular to an image processing method, device, ISP, electronic device and computer storage medium. Background Art

[0002] At present, tone mapping, as an important module in image signal processing (ISP), has aroused great research interest in recent years and has been widely used in many products.

[0003] Many mobile phone manufacturers will be equipped with self-developed tone mapping algorithms so that they can accurately restore the normal tone of the image in photo and video modes, giving users a better shooting experience.

[0004] However, the tone mapping algorithm is performed on the image itself, resulting in that the final image quality generated is difficult to satisfy users in some scenarios. Summary of the Invention

[0005] The embodiments of the present application provide an image processing method, apparatus, ISP, electronic device, and computer storage medium that can meet the individual needs of users.

[0006] The technical solution of this application is achieved as follows:

[0007] In a first aspect, an embodiment of the present application provides an image processing method, comprising:

[0008] In response to an operation on an input image, identifying a target area of the input image;

[0009] determining a gain image of the target area;

[0010] adjusting the gain image using the operation information of the operation to obtain an adjusted gain image;

[0011] The target area is processed using the adjusted gain image to obtain an output image.

[0012] In a second aspect, an embodiment of the present application provides an image processing device, comprising:

[0013] an identification module, configured to identify a target area of the input image in response to an operation on the input image;

[0014] A determination module, configured to determine a gain image of the target area;

[0015] an adjustment module, configured to adjust the gain image using the operation information of the operation to obtain an adjusted gain image;

[0016] A processing module is used to process the target area using the adjusted gain image to obtain an output image.

[0017] In a third aspect, an embodiment of the present application provides an ISP, comprising: a processor, configured to call and run a computer program from a memory, so that a device installed with the ISP executes the image processing method as described in one or more of the above embodiments; and a transceiver, configured to receive and send information during the process of sending and receiving information with the device or the ISP.

[0018] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: an ISP as described in one or more of the above embodiments, a processor, and a storage medium storing instructions executable by the processor; the storage medium relies on the processor to perform operations through a communication bus.

[0019] In a fifth aspect, an embodiment of the present application provides an electronic device comprising: a processor and a storage medium storing instructions executable by the processor; the storage medium relies on the processor to perform operations through a communication bus, and when the instructions are executed by the processor, the image processing method described in one or more of the above embodiments is executed.

[0020] In a sixth aspect, an embodiment of the present application provides a computer storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processors execute the image processing method described in one or more of the above embodiments.

[0021] Embodiments of the present application provide an image processing method, apparatus, ISP, electronic device, and computer storage medium, comprising: in response to an operation on an input image, identifying a target area of the input image, determining a gain image of the target area, adjusting the gain image using operation information of the operation to obtain an adjusted gain image, and processing the target area using the adjusted gain image to obtain an output image; that is, in embodiments of the present application, by responding to a user's operation on the input image, identifying the target area, and adjusting the gain image of the target area using the operation information, the output image obtained by processing the target area using the adjusted gain image is combined with the user's operation information, that is, combined with the user's personal needs, so that the obtained output image can meet the user's personal needs and improve the user's satisfaction with the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A flowchart of an optional image processing method provided in an embodiment of the present application;

[0023] Figure 2A flowchart of an example of an optional image processing method provided in an embodiment of the present application;

[0024] Figure 3 A schematic diagram of the architecture of an optional image segmentation network provided in an embodiment of the present application;

[0025] Figure 4 A schematic diagram of an optional user operation image provided in an embodiment of the present application;

[0026] Figure 5a A schematic diagram of an optional input image provided in an embodiment of the present application;

[0027] Figure 5b A schematic diagram of an optional output image provided in an embodiment of the present application;

[0028] Figure 6 A schematic structural diagram of an optional image processing device provided in an embodiment of the present application;

[0029] Figure 7 A schematic diagram of the structure of an optional ISP provided in an embodiment of the present application;

[0030] Figure 8 A schematic diagram of the structure of an optional electronic device provided in an embodiment of the present application Figure 1 ;

[0031] Figure 9 A schematic diagram of the structure of an optional electronic device provided in an embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0033] In the related art, solutions for tone mapping all have the problem of being unable to satisfy users' need to readjust the tone of a specific area with unsatisfactory tone.

[0034] To address the technical problem in related art that images processed by tone mapping cannot meet user needs, the present invention provides an image processing method. Figure 1 A flowchart of an optional image processing method provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the image processing method may include:

[0035] S101: In response to an operation on an input image, identifying a target area of the input image;

[0036] The image processing method provided in the embodiment of the present application can be applied to an ISP or a processor, and the embodiment of the present application does not make any specific limitations on this.

[0037] Among them, the above-mentioned input image can be an image taken by a camera, an image received from other electronic devices, or an image downloaded through the network. Here, the embodiment of the present application does not make specific limitations on this.

[0038] In addition, the above-mentioned input image can be an image in a video sequence, which can be a video captured by a camera, a video received from other electronic devices, or a video downloaded through the network. Here, the embodiment of the present application does not make specific limitations on this.

[0039] After receiving the input image, the input image is displayed on the display interface of the electronic device. At this time, the user issues an operation to the input image, so that the electronic device receives the operation on the input image and responds.

[0040] Specifically, the above operation can be a touch operation or a non-touch operation. Among them, the touch operation can be a click operation, a double-click operation, or a long press operation, etc., and the non-touch operation can be a gesture operation, a facial expression operation, etc. Here, the embodiment of the present application does not make specific limitations on this.

[0041] In S101, in order to implement an operation in response to the input image, the target area of the input image is identified. Specifically, in response to the operation, the operation position is determined, and the input image is instance segmented according to the operation position to obtain a segmented instance, and the segmented instance is determined as the target area.

[0042] That is to say, the operation information of the operation can be known through the user's operation, and the operation information may include: operation position, operation duration, operation direction, etc. Among them, for the operation position, if it is a touch operation, it is the position touched on the input image; if it is a non-touch operation, the operation position is the position corresponding to the operation on the input image.

[0043] After determining the operation location, instance segmentation is performed on the input image to obtain segmented instances, which are then used as target regions. Instance segmentation of the input image can be implemented using an image segmentation network. For example, the image segmentation network can include a basic convolution module and an upsampling module.

[0044] It should be noted that after the target area is identified in the input image, the image following the input image in the video sequence can also identify its corresponding target area through the above-mentioned image segmentation network based on the segmentation instance of the input image without receiving any operation.

[0045] In addition, after the target region is identified by the image segmentation network, the segmentation instance can be directly used as the target region, and the target region can also be updated by expanding the segmentation instance. Here, the embodiment of the present application does not make specific limitations on this.

[0046] S102: Determine a gain image of a target area;

[0047] After the target area of the input image is identified through the above S101, a gain image of the target area can be determined in S102. The gain image here can be a gain image for different color components, for example, it can be a gain image for brightness, it can also be a gain image for saturation, or it can also be a gain image for other color components. Here, the embodiment of the present application does not make specific limitations on this.

[0048] Among them, in the above-mentioned determination of the gain image of the target area, the gain image can be determined by using a statistical method for the target area, or the gain image of the target area can be determined by using an artificial intelligence (AI) model for the target area. Here, the embodiment of the present application does not make any specific limitations on this.

[0049] It should be noted that, in addition to determining the gain image of the target area based on the target area, the gain image of the target area can also be determined based on the target area and the input image. The gain image can be determined by statistically analyzing the target area and the input image, or by using an AI model to determine the gain image of the target area and the input image. This is not specifically limited in the embodiments of the present application.

[0050] S103: adjusting the gain image using the operation information of the operation to obtain an adjusted gain image;

[0051] After the gain image of the target area is determined in S102 , in S103 , the gain image is adjusted using the operation information to obtain an adjusted gain image.

[0052] Among them, the above-mentioned operation information may include operation duration, operation direction or operation method, etc. Here, the embodiment of the present application does not make specific limitations on this.

[0053] Here, the gain image may be adjusted according to one or more pieces of information in the operation information. The adjustment may be to proportionally reduce or increase the gain image. This is not specifically limited in the embodiments of the present application.

[0054] S104: Processing the target area using the adjusted gain image to obtain an output image.

[0055] After the adjusted gain image is obtained through the above S103, in S104, the target area can be processed using the adjusted gain image, so that the target area can be updated to obtain an output image.

[0056] The above-mentioned method can directly use the adjusted gain image to process the target area to obtain the output image; it can also first use the adjusted gain image to process the target area, and then use the target area and the target area after gain processing to update the target area to obtain the output image; it can also perform color space conversion on the adjusted gain image, and then use the converted gain image to process the target area to obtain the output image. Here, the embodiments of the present application do not specifically limit this.

[0057] After identifying the target area, in an optional embodiment, the method may further include:

[0058] Determine the area of the target area based on the preset pixel area size and taking the boundary of the target area as the boundary;

[0059] Update the target area based on the target area and field.

[0060] It can be understood that the electronic device is pre-set with a pixel area size, for example, an N×N pixel area size. Here, the boundary of the target area is used as the boundary to determine the area of the target area, so that the area is the area surrounding the target area on the input image, and each pixel on the boundary of the target area is used as the starting point to deviate from the target area in the direction of N-1 pixels as the area of the target area. Among them, if the distance from each pixel on the boundary to the boundary of the input image is less than N-1 pixels, then the pixel points from each pixel on the boundary to the boundary of the input image away from the target area in the direction of N-1 pixels can be used as the area of the target area.

[0061] After determining the domain of the target area, the target area can be updated based on the target area and the domain. The target area can be directly taken as the union of the target area and the domain. The target area and the domain can also be used to update a preset pixel area size to update the domain, thereby updating the target area. This embodiment of the present application does not specifically limit this.

[0062] In this way, by determining the neighborhood as described above and updating the target area using the target area and the domain, the target area can be expanded so that the updated target area can include as much similar information as possible around the selected segmentation instance, which helps to reserve space for the transition after tone mapping.

[0063] In order to update the target area, in an optional embodiment, updating the target area according to the target area and the domain may include:

[0064] Determine the average brightness of the target area and the average brightness of the field;

[0065] When the ratio of the average brightness of the domain to the average brightness of the target area falls within a preset range, updating the target area using the union of the target area and the domain;

[0066] In the case where the ratio of the average brightness of the domain to the average brightness of the target area does not fall within a preset range, the preset pixel area size is adjusted, and the domain of the target area is determined according to the preset pixel area size with the boundary of the target area as the boundary to update the domain, and the target area is updated using the union of the target area and the domain.

[0067] It can be understood that the average brightness of the target area is determined first, and the average brightness of the field is determined, and then the ratio of the average brightness of the field to the average brightness of the target area is calculated.

[0068] Determine whether the above ratio falls within a preset range. For example, the above preset range can be (-∞, 0.7) ∪ (1.5, +∞). If it falls within, it means that the range of the domain is appropriate, so the union of the target area and the domain is directly updated to the target area; if it does not fall within, it means that the range of the domain is not appropriate, so the preset pixel area size needs to be adjusted, for example, if the N×N pixel area size is adjusted to the 8N×8N pixel area size.

[0069] After adjusting the preset pixel area size, the boundary of the target area can be determined according to the preset pixel area size with the boundary of the target area as the boundary. In this way, the boundary is updated, and then the union of the target area and the boundary is updated as the target area.

[0070] In this way, the target region is updated by determining whether the ratio of the brightness of the region to the brightness of the instance falls within a preset range, so that a suitable region can be determined for the segmentation instance selected by the user, thereby obtaining a suitable target region.

[0071] In order to determine the gain image of the target area, in an optional embodiment, S102 may include:

[0072] Determine the target brightness mapping relationship according to the target area;

[0073] Using the target brightness mapping relationship, the target area is mapped to obtain the target area after brightness mapping;

[0074] A gain image is determined based on the target area after brightness processing.

[0075] It can be understood that after determining the target area, first, the target brightness mapping relationship of the target area can be determined. Here, the target brightness mapping relationship can be determined by statistical means or AI model means. Here, the embodiment of the present application does not make specific limitations on this.

[0076] Then, the target area is mapped using the target brightness mapping relationship, wherein the value of each color channel of the target area can be brightness mapped, so as to obtain the brightness mapped value of each color channel, that is, the target area after brightness mapping.

[0077] Finally, the gain image is determined based on the target area after brightness processing. The brightness value of the target area after brightness processing can be calculated first. The calculation formula for the brightness value of the RGB channel can be used to obtain the gain image. Then, the gain image is determined based on the brightness value of the target area after brightness processing. Here, the gain image can be obtained using a preset rule or a preset AI model. The embodiment of the present application does not specifically limit this.

[0078] In this way, the target area after brightness mapping is obtained through the above target brightness mapping relationship, and then the gain image is obtained, so that the obtained gain image is related to the target area after the target brightness mapping relationship is processed, thereby improving the accuracy of the gain image.

[0079] Furthermore, in order to obtain a gain image, in an optional embodiment, determining the gain image according to the target area after brightness mapping may include:

[0080] Determine the average brightness of the target area in the logarithmic domain after brightness mapping;

[0081] A gain image is determined according to the average brightness of the target area after brightness mapping in the logarithmic domain, the brightness of the target area after brightness mapping, the first value, and the second value.

[0082] It can be understood that the pixel values of the target area after brightness mapping can be converted to pixel values in the logarithmic domain first, and then the average brightness L of the pixel values in the logarithmic domain can be calculated. waver , the following formula can be used:

[0083]

[0084] Wherein, Lw is the brightness value of each pixel in the target area after brightness mapping, and the pixel size of the input image is m×n.

[0085] After obtaining the average brightness, a gain image can be calculated based on the average brightness, Lw, a first value, and a second value. The first value is the maximum of the maximum brightness of the target area after brightness mapping and the maximum brightness of the input image; the second value is the minimum of the minimum brightness of the target area after brightness mapping and the minimum brightness of the input image.

[0086] That is, the maximum value Max1 of the brightness of the target area after brightness mapping and the maximum value Max2 of the brightness of the input image are calculated, so that the first value Lw can be obtained by the following formula: max :

[0087] Lw max =max(Max1,Max2) (2)

[0088] After calculating the minimum brightness value Min1 of the target area after brightness mapping and the minimum brightness value Min2 of the input image, the second value Lw can be obtained by the following formula: min :

[0089] Lw min =max(Min1,Min2) (3)

[0090] When the average brightness of the target area after the brightness mapping in the logarithmic domain, the brightness of the target area after the brightness mapping, the first value, and the second value are known, the gain value can be calculated using the following formula to obtain the gain image gain:

[0091]

[0092] Where C is the gain coefficient.

[0093] Thus, by obtaining the gain image of the target area in the logarithmic domain, the obtained gain image is more consistent with the color distribution of the target area image, which helps to achieve the color tone processing of the target area to meet the user's personal needs.

[0094] In order to obtain the target brightness mapping relationship, in an optional embodiment, determining the target brightness mapping relationship according to the target area may include:

[0095] Counting the brightness values of the target area to obtain a first brightness mapping relationship;

[0096] Performing statistics on the brightness values of the input image to obtain a second brightness mapping relationship;

[0097] A target brightness mapping relationship is determined according to the first brightness mapping relationship and the second brightness mapping relationship.

[0098] It is understandable that after obtaining the target area, the brightness value of the target area and the brightness value of the input image can be calculated, and the brightness value of the target area and the brightness value of the input image can be statistically analyzed. The statistics here can include histogram statistics or cumulative histogram statistics. Here, the embodiments of the present application do not specifically limit this.

[0099] Through the above histogram statistics and / or cumulative histogram statistics, a first brightness mapping relationship for the target area and a second brightness mapping relationship for the input image can be obtained. The first brightness mapping relationship and the second brightness mapping relationship can be in the form of curves.

[0100] After obtaining the first brightness mapping relationship and the second brightness mapping relationship, the target brightness mapping relationship can be determined based on the first brightness mapping relationship and the second brightness mapping relationship. Here, the first brightness mapping relationship and the second brightness mapping relationship can be fused according to preset rules to obtain the target brightness mapping relationship. Alternatively, AI can be used to determine the target brightness mapping relationship based on the first brightness mapping relationship and the second brightness mapping relationship for fusion. Here, the embodiments of the present application do not make specific limitations on this.

[0101] In this way, the target brightness mapping relationship can be obtained by the above method, so that the target brightness mapping relationship is related to the statistical distribution of the brightness values of the target area and the statistical distribution of the brightness values of the input image, so that the target brightness mapping relationship is not only related to the brightness distribution of the target area, but also related to the brightness distribution of all areas of the input image, which improves the accuracy of the target brightness mapping relationship and helps to determine a gain image with higher accuracy.

[0102] Furthermore, in order to determine the target brightness mapping relationship, in an optional embodiment, determining the target brightness mapping relationship according to the first brightness mapping relationship and the second brightness mapping relationship may include:

[0103] Determining a weight of the first brightness mapping relationship and a weight of the second brightness mapping relationship;

[0104] Based on the weight of the first brightness mapping relationship and the weight of the second brightness mapping relationship, the first brightness mapping relationship and the second brightness mapping relationship are fused to determine a target brightness mapping relationship.

[0105] It is understandable that the weight of the first brightness mapping relationship and the weight of the second brightness mapping relationship can be determined, and then the first brightness mapping relationship and the second brightness mapping relationship are fused using the determined weights to obtain the target brightness mapping relationship.

[0106] The weight α of the first brightness mapping relationship can be obtained using the following formula:

[0107]

[0108] Wherein, R1 is the average brightness of the target area, and R2 is the average brightness of the input image. The weight of the above second brightness mapping relationship is 1-α.

[0109] In this way, the first brightness mapping relationship and the second brightness mapping relationship are fused in the above manner to obtain a target brightness mapping relationship, thereby improving the accuracy of the target brightness mapping relationship and facilitating determination of a gain image with higher accuracy.

[0110] In order to further improve the accuracy of the gain image, in an optional embodiment, the above method may further include:

[0111] The adjusted gain image is adjusted using the target brightness mapping relationship to obtain an adjusted gain image.

[0112] It is understandable that after obtaining the adjusted gain image, in addition to adjusting the adjusted gain image using the operation information, the adjusted gain image may also be adjusted using the target brightness mapping relationship obtained in the above manner to update the adjusted gain image.

[0113] Specifically, the value obtained by applying the target brightness mapping relationship to the same pixel value is multiplied by the corresponding value in the adjusted gain image, thereby updating and obtaining the adjusted gain value.

[0114] In this way, the adjusted gain image is adjusted using the target brightness mapping relationship to update the adjusted gain image, so that the adjusted gain image is more accurate, which helps to achieve a tone mapping effect on the target area.

[0115] In order to adjust the gain image based on the operation information, in an optional embodiment, S103 may include:

[0116] Determine the gain coefficient according to the operation information;

[0117] The gain image is adjusted using the gain coefficient to obtain an adjusted gain image.

[0118] It is understandable that, for a user's operation on an input image, the gain coefficient may be determined according to operation information of the operation, wherein the operation information may be one or more of operation duration, operation direction, or operation mode.

[0119] After obtaining the gain coefficient, the gain image is adjusted using the gain coefficient to obtain an adjusted gain image. Here, the gain coefficient can be directly multiplied by each gain in the gain image to obtain the adjusted gain image.

[0120] In this way, the gain coefficient is determined by the above-mentioned operation information, and the gain image is adjusted using the gain coefficient, so that the obtained adjusted gain image is related to the operation information issued by the user. In this way, the user can issue an operation according to his or her own needs to adjust the gain image according to his or her own needs, so that the obtained output image can meet the user's personal needs.

[0121] Furthermore, in order to determine the gain coefficient, in an optional embodiment, determining the gain coefficient according to the operation information may include:

[0122] The gain coefficient is determined according to the operation duration in the operation information.

[0123] It can be understood that the gain coefficient is determined by the operation duration in the operation information. Here, the relationship between the operation duration and the gain coefficient can be established in advance. Then, based on knowing the relationship, the user can perform operations on the input image according to his or her own needs and control the operation duration, thereby using the operation duration to determine the gain coefficient to adjust the gain image according to his or her own needs.

[0124] In this way, the gain coefficient is determined by the operation duration to adjust the gain image, so that the determined gain coefficient is related to the operation duration, which is beneficial for the user to adjust the gain image according to his or her own needs, thereby making the output image meet the user's needs.

[0125] Regarding the above operation duration, in an optional embodiment, the operation duration is positively correlated with the gain coefficient.

[0126] It can be understood that the operation time is positively correlated with the gain coefficient. For example, the gain coefficient C can be calculated using the following formula:

[0127] C=a×t (7)

[0128] Where a is a constant and t represents the operation duration.

[0129] In this way, by pre-establishing a positive correlation between the operation duration and the gain coefficient, the user can control the operation duration in a positive correlation manner, so that the gain coefficient of the target area can be adjusted according to their own needs, and the output image can meet the user's personal needs.

[0130] After obtaining the adjusted gain image, in order to utilize it to process the target area and obtain an output image, in an optional embodiment, S104 may include:

[0131] Processing the target area using the adjusted gain image to obtain a gain-processed target area;

[0132] The target region is updated according to the target region and the target region after gain processing to obtain an output image.

[0133] It is understood that after obtaining the adjusted gain image, the target area is processed using the adjusted gain image to obtain the target area after gain processing. The target area and the target area after gain processing are then used to determine the brightness value of the target area, thereby updating the target area and obtaining the output image.

[0134] It should be noted that the target area can be updated based on the target area and the target area after gain processing using a soft light mode for pixel-by-pixel blending. Alternatively, a lightweight neural network or a more complex tone mapping curve generation method can be used to update the target area. The soft light mode can also be other modes. For example, the other modes can be a variety of blending modes such as a spot light mode and a strong light mode.

[0135] In this way, the target area is updated by using the above target area and the target area after gain processing, so that the brightness value of the updated target area can meet the user's personal needs.

[0136] In addition, in addition to processing the brightness value of the target area in the above manner, in an optional embodiment, the above method may further include:

[0137] Determine a saturation mapping relationship based on the operation information of the operation;

[0138] The saturation of the target area is mapped using the saturation mapping relationship to obtain the saturation mapped target area to update the target area.

[0139] It is understandable that the saturation mapping relationship may be determined according to the operation information, and then the saturation of the target area may be mapped using the saturation mapping relationship, thereby obtaining the saturation mapped target area to update the target area.

[0140] That is, in addition to using the operation information to determine the gain coefficient, the operation information is also used to determine the saturation mapping relationship, thereby achieving saturation processing of the target area. Here, similar to brightness processing, the relationship between the operation information and the saturation mapping relationship is established in advance, so that the user knows the relationship between the operation information and the saturation mapping relationship. Then, the user can achieve the establishment of the saturation mapping relationship by controlling their own operations.

[0141] In this way, the saturation mapping relationship is determined through the above-mentioned operation information, so that the operation information can be related to the saturation mapping relationship, so that the user can realize the saturation mapping of the image through operation, and then the output image can meet the user's personal needs.

[0142] In order to determine the saturation mapping relationship using the operation information, in an optional embodiment, determining the saturation mapping relationship according to the operation direction in the operation information may include:

[0143] According to the operation direction in the operation information, a coefficient of the saturation mapping relationship is determined to obtain the saturation mapping relationship.

[0144] It can be understood that the relationship between the operation direction and the coefficient of the saturation mapping relationship is established in advance. In this way, after the operation direction is obtained, the coefficient corresponding to the operation direction can be obtained, and the coefficient can be determined as the coefficient of the saturation mapping relationship.

[0145] After obtaining the coefficient of the saturation mapping relationship, the coefficient is substituted into the saturation mapping relationship to obtain the saturation mapping relationship, which is used to map the saturation of the target area to update the target area.

[0146] In this way, the coefficient of the saturation mapping relationship is determined by the above-mentioned operation direction, so that the user's operation direction is related to the coefficient of the saturation mapping relationship, so that the user can realize the saturation mapping of the image through operation, and then the output image can meet the user's personal needs.

[0147] In addition, to obtain an output image, in an optional embodiment, processing the target area using the gain image to obtain the output image may include:

[0148] performing color space conversion on the adjusted gain image to obtain a converted gain image;

[0149] The converted gain image is used to process the target area after saturation mapping to update the target area and obtain an output image.

[0150] It can be understood that after obtaining the adjusted gain image, the adjusted gain image is converted in color space to obtain a converted gain image, and then the converted gain image is used to process the target area after saturation mapping to update the target area. That is, in addition to using the above-mentioned adjusted gain to process the brightness of the target area, the adjusted gain image is also used to obtain a gain image for saturation, and the gain image is used to process the target area after saturation mapping, so that the target area of the obtained output image is an image after brightness processing and saturation processing.

[0151] In this way, the saturation of the target area is processed in the above manner, so that the target area of the output image is processed not only in terms of brightness but also in terms of saturation, further meeting the needs of the user himself.

[0152] Furthermore, in order to improve the image quality of the output image while meeting the needs of the user, in an optional embodiment, the above method may further include:

[0153] determining an intensity image of the target area based on distances between pixels in the target area and endpoints of the input image;

[0154] The adjusted gain image is adjusted using the intensity image of the target region to update the adjusted gain image.

[0155] It can be understood that after obtaining the adjusted gain image, the intensity image of the target area can also be determined based on the distance between the pixels in the target area and the endpoints of the input image. Here, the distance between the pixels in the target area and the endpoints of the input image can be the sum of the distances between the pixels of the input image and the four endpoints of the input image. Of course, it can also be the distance between the pixels of the input image and any one of the endpoints of the input image, or the sum of the distances between the pixels of the input image and at least two endpoints of the input image. Here, the embodiments of the present application do not make specific limitations on this.

[0156] Here, a rule or AI model may be preset to obtain an intensity image of the target area, and then the intensity image of the target area may be used to adjust the adjusted gain image, thereby updating the adjusted gain image.

[0157] In this way, the intensity image of the target area is determined by the distance between the pixels in the target area and the endpoints of the input image, thereby adjusting the adjusted gain image, which helps to achieve different processing of the target area based on the distance and helps to improve the image quality of the target area.

[0158] The following examples are used to describe the image processing method described in one or more of the above embodiments.

[0159] Figure 2 A flowchart of an example of an optional image processing method provided in an embodiment of the present application is shown as follows: Figure 2 As shown, the image processing method may include:

[0160] S201: Acquire an input image and touch information of the input image;

[0161] S202: Image segmentation to identify touch area instances;

[0162] Specifically, it mainly includes the following steps:

[0163] Input the video sequence into ISP, and the current frame image is recorded as Frame i , the touch information includes the touch direction Z, time t, and touch area, recorded as Maski ;

[0164] Frame i Perform image instance segmentation to obtain the instance segmentation area touched by the user (equivalent to the target area mentioned above), as follows:

[0165] Figure 3 A schematic diagram of the architecture of an optional image segmentation network provided in an embodiment of the present application is shown as follows: Figure 3 As shown, image input 1 is Mask i , image input 2 is Frame i , the Mask obtained by image input 1 through the basic convolution module (each basic convolution contains a 3x3 convolution layer, a maximum pooling layer and a BN layer) i The multi-scale features of the image input 2 are spliced and fused through the basic convolution module to obtain the fusion of Mask i The multi-scale features after the multi-scale features are generated enter the next level of convolution module until all the basic convolutions in the network have completed feature extraction and the smallest scale feature map is obtained. The number of feature channels is doubled after each basic convolution.

[0166] The upsampling module in the second part can include a multi-layer network (each layer of the network includes 1 upsampling convolution layer, feature splicing and 2 3x3 convolution layers). The multi-layer network in the second part can be considered as a multi-layer upsampling network.

[0167] When processing the second part, the feature vectors of each downsampling network layer in the first part are first upsampled by each upsampling network layer in the second part to obtain the upsampled results. When concatenating features, feature vectors of the same size are first extracted from the feature vectors of the corresponding output layers in the first part as reference feature vectors for each upsampling network layer in the second part. This reference feature vector is then fused with the upsampling results of the second part to obtain the fused features of each upsampling network layer. This fusion can be called channel concatenation. The fused features of each upsampling network layer are further convolved to obtain the category of each pixel. After each upsampling convolution, the width and height of the feature map are doubled, and the number of channels is halved, which is then used for merging and concatenation with the shallow features of the first part.

[0168] Among them, the output result of the segmentation network is processed in the time domain as follows: after the image segmentation network, the output Seg i , according to Mask i All recorded image positions can determine the same type of segmentation instance X at the user touch position; in the next frame image Frame i+1 After output, Seg i+1 The image index to instance X can ensure that when Mask i+1When it does not exist and the image changes continuously, the segmentation network can continue to segment out parts of the same type.

[0169] S203: updating the segmentation instance and performing statistics on the segmentation instance to achieve mapping of the segmentation instance;

[0170] Here, we post-process the segmented instances X of the same type and perform statistical calculations on the image information. The details are as follows:

[0171] 1) First, based on the segmented instances X of the same type after segmentation, expand the range of the segmented instance X and perform an adaptive dilation operation on the range framed by the segmented instance X. Specifically, the following convolution kernel is defined:

[0172]

[0173] Record it as ONES 8x8 , used to determine the area L by segmenting the instance X boundary, and the average brightness Y of the area L outside and the average brightness Y within the segmentation instance X inside Do more than what is worth it:

[0174]

[0175] If N>1.5 or N<0.7, then the above convolution kernel is changed to ONES 8Nx8N , and then, redetermine the area L, where the purpose of changing the convolution kernel size is to make the final neighborhood L contain more similar information around it and reserve space for the transition after tone mapping.

[0176] In this way, the union of the area L and the segmentation instance X is obtained as the area L, and this area L corresponds to the above-mentioned target area.

[0177] 2) Statistical neighborhood L and full image frame i Histogram and cumulative histogram;

[0178] Assume that the grayscale of the image is divided into K levels at equal intervals. For an image with a pixel value of 10 bits, if K is set to 256, the step size of each grayscale level is That is, 0-15 is counted in Hist[0], 16-31 is counted in Hist[1], ..., 1008-1023 is counted in Hist

[255] .

[0179] Traverse the neighborhood L and the entire image Frame separately iThen, the brightness distribution histogram is subjected to contrast limited adaptive histogram equalization (CLAHE) and the histogram is smoothed. The cumulative distribution histogram curve of the image is calculated. After processing, the brightness cumulative distribution histogram curve CDF1 of the neighborhood L and the full image Frame are obtained. i The brightness cumulative distribution histogram curve CDF2.

[0180] After interpolating the curves CDF1 and CDF2 to obtain the 10-bit points of the image, the curves CDF1 and CDF2 are fused using the coefficient α, which can be calculated using the above formula (6).

[0181] The final cumulative distribution histogram curve is obtained (equivalent to the above target brightness mapping relationship):

[0182] CDF out =α·CDF1+(1-α)·CDF2 (9)

[0183] Using CDF out Process field L to obtain field L out .

[0184] S204: Generate a mapping curve to complete the domain L out Tone mapping;

[0185] Calculate the neighborhood L out The brightness channel, statistical neighborhood L out Average brightness and full image frame i The average brightness of the neighborhood L out The maximum brightness value Max1, the full picture Frame i The maximum brightness Max2, the neighborhood L out The minimum brightness value Min1, the full picture Frame i The minimum brightness value Min2.

[0186] The neighborhood L is calculated using the following formulas (1) to (5): out The gain (mapping curve, equivalent to the gain image above). The gain curve and the normalized CDF out Perform fusion to obtain the remapped gain final (equivalent to the above-mentioned adjusted gain image), and the following formula (10) is used to calculate the neighborhood L out Remap:

[0187] L out =L out ×gainfunal (10)

[0188] Finally, the newly obtained neighborhood L out The initial neighborhood L is mixed pixel by pixel using the soft light mode to obtain the final tone mapping result L final , taking an image with a pixel value of 10 bits as an example, the following formula can be used to obtain:

[0189]

[0190] S205: Adjust the gain using the touch information to adjust the tone mapping result.

[0191] Specifically, the mapping result is adjusted using the user's touch information, and the mapping result is re-optimized.

[0192] The user touch time t is proportional to the gain coefficient. When the touch time is longer, the gain value is larger, thereby making the mapping strength stronger.

[0193] The curve mapping saturation S is controlled by the user touch direction Z:

[0194] S=b·Z 3 (13)

[0195] Where b is the original saturation of the field L.

[0196] For example, when the touch direction is toward the first boundary of the image, Z=5; and when the touch direction is away from the first boundary of the image, Z=-5.

[0197] Among them, using L final Image, use the following formula to calculate the gain of UV map:

[0198] gain uv =gain×RGB2YUV 3x3 (14)

[0199] Among them, RGB2YUV 3x3 Represents the color transformation matrix from RGB to YUV, and applies the S curve for mapping to generate a saturation-enhanced L final Image, complete the control of image saturation and obtain the output image.

[0200] After mapping the curve, the intensity is attenuated according to the center point to make the transition more natural. The intensity image Decay(d) can be calculated using the following formula:

[0201]

[0202] Where d is the distance from the pixel to the endpoint of the output image, and σ controls the range of attenuation, where σ is a constant. The closer to the edge, the stronger the attenuation effect, thus reducing the intensity of the enhancement effect.

[0203] The obtained intensity image can be used to adjust the gain image. By adjusting the size of σ, the smoothness of the attenuation can be controlled. The larger the σ, the smoother the attenuation process.

[0204] Figure 4 A schematic diagram of an optional user operation image provided in an embodiment of the present application, such as Figure 4 As shown in FIG. 1 , the entire technical solution process is as follows: an input image is obtained, and a user clicks a point in a mountain area on the input image to obtain a segmentation instance X. Statistics are performed on the segmentation instance X to obtain statistical information, and a mapping curve is generated. The horizontal axis of the mapping curve is the original brightness value, and the vertical axis is the mapped brightness value. Here, the mapping curve is used to map the segmentation instance X to obtain an output image, that is, the output image is obtained using the above example.

[0205] In this example, it can be expanded as follows: the image segmentation network includes but is not limited to the segmentation network mentioned in the document, and can also be expanded to other two-input segmentation networks with better performance or higher accuracy; the image tone mapping curve formula includes but is not limited to the mapping curve calculation method mentioned in the document, as well as the soft light mixing mode, and can also be expanded to a lightweight neural network or a more complex tone mapping curve generation method, or a point light mode, a strong light mode and other mixing modes; the user-controlled interaction method is not limited to the touch area, touch time and touch direction, and the user interaction method can be more broadly defined, including but not limited to camera recognition of user posture, facial expressions, and more complex touch direction curves, etc., which can all be used as a way to adjust the tone mapping curve.

[0206] Figure 5a A schematic diagram of an optional input image provided in an embodiment of the present application, such as Figure 5a As shown, including part 5a1, Figure 5b A schematic diagram of an optional output image provided in an embodiment of the present application, such as Figure 5b As shown, including part 5b1, the color tone of part 5a1 is significantly improved in image quality compared with part 5b1.

[0207] This example proposes a new tone mapping method that combines user touch information and instance segmentation. A user touches a portion of a captured scene where the hue is unsatisfactory. Using a segmentation network, the user identifies similar portions of the scene with unsatisfactory hues. Furthermore, as the image continuously changes, the network continuously segments similar instances of the same type. The segmented portions of the same type are analyzed, their information combined with global image information is aggregated, and local and global tone mapping parameters are calculated to remap the hue of the similar portions. Furthermore, the user is given the ability to freely select adjustments to the mapping direction, intensity, and other adjustments based on the touch direction or duration, improving local tone mapping accuracy and ultimately achieving user-satisfactory image results. This method features high accuracy, strong real-time performance, and an interactive model.

[0208] This example allows the user to independently determine the image area to be adjusted and perform segmentation. By combining the segmentation network and the input of the mask image, it can accurately divide the user-selected segmentation instance and is more robust. It does not require a complex segmentation network and full-image block processing. The increase in input computational complexity through input dimensionality reduction is very small and can be fully supported by the mobile phone platform. It can combine the user's touch area, time, direction and user interaction to generate a tone mapping result for a specific area, and can be adaptively adjusted by the user.

[0209] An embodiment of the present application provides an image processing method, comprising: identifying a target area of the input image in response to an operation on an input image, determining a gain image of the target area, adjusting the gain image using operation information of the operation to obtain an adjusted gain image, and processing the target area using the adjusted gain image to obtain an output image; that is, in an embodiment of the present application, by identifying the target area in response to a user's operation on the input image, and adjusting the gain image of the target area using the operation information, the output image obtained by processing the target area using the adjusted gain image is combined with the user's operation information, that is, combined with the user's personal needs, so that the obtained output image can meet the user's personal needs and improve the user's satisfaction with the image.

[0210] Based on the same inventive concept as the above embodiments, the present embodiment provides an image processing device. Figure 6 A schematic diagram of the structure of an optional image processing device provided in an embodiment of the present application is shown as follows: Figure 6 As shown, the image processing device includes: a recognition module 61, a determination module 62, an adjustment module 63 and a processing module 64; wherein,

[0211] an identification module 61 for identifying a target area of the input image in response to an operation on the input image;

[0212] A determination module 62 is configured to determine a gain image of a target area;

[0213] An adjustment module 63 is configured to adjust the gain image using the operation information of the operation to obtain an adjusted gain image;

[0214] The processing module 64 is configured to process the target area using the adjusted gain image to obtain an output image.

[0215] In an optional embodiment, the adjustment module 63 is specifically configured to: determine a gain coefficient according to the operation information; and adjust the gain image using the gain coefficient to obtain an adjusted gain image.

[0216] In an optional embodiment, the adjustment module 63 determines the gain coefficient according to the operation information, including: determining the gain coefficient according to the operation duration in the operation information.

[0217] In an optional embodiment, the operation duration is positively correlated with the gain coefficient.

[0218] In an optional embodiment, the determination module 62 is specifically used to: determine a target brightness mapping relationship according to the target area; map the target area using the target brightness mapping relationship to obtain a target area after brightness mapping; and determine a gain image according to the target area after brightness processing.

[0219] In an optional embodiment, the determination module 62 determines the gain image based on the target area after brightness mapping, including: determining the average brightness of the target area after brightness mapping in the logarithmic domain; determining the gain image based on the average brightness of the target area after brightness mapping in the logarithmic domain, the brightness of the target area after brightness mapping, a first value, and a second value; wherein the first value is the maximum value of the brightness of the target area after brightness mapping and the maximum value of the brightness of the input image; and the second value is the minimum value of the brightness of the target area after brightness mapping and the minimum value of the brightness of the input image.

[0220] In an optional embodiment, the determination module 62 determines the target brightness mapping relationship based on the target area, including: performing statistics on the brightness values of the target area to obtain a first brightness mapping relationship; performing statistics on the brightness values of the input image to obtain a second brightness mapping relationship; and determining the target brightness mapping relationship based on the first brightness mapping relationship and the second brightness mapping relationship.

[0221] In an optional embodiment, the determination module 62 determines the target brightness mapping relationship based on the first brightness mapping relationship and the second brightness mapping relationship, including: determining the weight of the first brightness mapping relationship and the weight of the second brightness mapping relationship; based on the weight of the first brightness mapping relationship and the weight of the second brightness mapping relationship, fusing the first brightness mapping relationship and the second brightness mapping relationship to determine the target brightness mapping relationship.

[0222] In an optional embodiment, the device is further configured to: adjust the adjusted gain image by using the target brightness mapping relationship to update the adjusted gain image.

[0223] In an optional embodiment, the processing module 64 is specifically used to: use the adjusted gain image to process the target area to obtain the gain-processed target area; update the target area according to the target area and the gain-processed target area to obtain an output image.

[0224] In an optional embodiment, the device is further used to: determine a saturation mapping relationship based on operation information of the operation; use the saturation mapping relationship to map the saturation of the target area to obtain the target area after saturation mapping, so as to update the target area.

[0225] In an optional embodiment, the device determines the saturation mapping relationship according to the operation direction in the operation information, including: determining a coefficient of the saturation mapping relationship according to the operation direction in the operation information to obtain the saturation mapping relationship.

[0226] In an optional embodiment, the processing module 64 is specifically used to: convert the color space of the adjusted gain image to obtain a converted gain image; use the converted gain image to process the target area after saturation mapping to update the target area and obtain an output image.

[0227] In an optional embodiment, the device is further used to: determine the intensity image of the target area based on the distance between the pixels in the target area and the endpoints of the input image; and use the intensity image of the target area to adjust the adjusted gain image to update the adjusted gain image.

[0228] In an optional embodiment, the device is further configured to: determine the domain of the target area according to a preset pixel area size and with the boundary of the target area as the boundary; and update the target area according to the target area and the domain.

[0229] In an optional embodiment, the device updates the target area according to the target area and the field, including: determining the average brightness of the target area and the average brightness of the field; when the ratio of the average brightness of the field to the average brightness of the target area falls within a preset range, updating the target area using the union of the target area and the field; when the ratio of the average brightness of the field to the average brightness of the target area does not fall within the preset range, adjusting the preset pixel area size, determining the field of the target area according to the preset pixel area size and with the boundary of the target area as the boundary, to update the field, and updating the target area using the union of the target area and the field.

[0230] In practical applications, the above-mentioned recognition module 61, determination module 62, adjustment module 63 and processing module 64 can be implemented by a processor located on the image processing device, specifically a CPU, a microprocessor (Microprocessor Unit, MPU), a digital signal processor (Digital Signal Processing, DSP) or a field programmable gate array (Field Programmable Gate Array, FPGA) and the like.

[0231] The embodiment of the present application provides an ISP, Figure 7 A schematic diagram of an optional ISP structure provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, an embodiment of the present application provides an ISP 700, which includes:

[0232] The processor 71 is configured to call and execute a computer program from a memory so that a device equipped with the ISP 700 executes the method described in one or more of the above embodiments;

[0233] The transceiver 72 is used to receive and send information when sending and receiving information with the device or ISP 700.

[0234] Figure 8 A schematic diagram of the structure of an optional electronic device provided in an embodiment of the present application Figure 1 ,like Figure 8 As shown, an embodiment of the present application provides an electronic device 800, including:

[0235] ISP700, processor 81 and storage medium 82 storing instructions executable by the processor; the storage medium 82 relies on the processor 91 to perform operations through the communication bus 83,

[0236] Figure 9 A schematic diagram of the structure of an optional electronic device provided in an embodiment of the present application Figure 2 ,like Figure 9As shown, an embodiment of the present application provides an electronic device 900, including:

[0237] A processor 91 and a storage medium 92 storing instructions executable by the processor; the storage medium 92 relies on the processor 91 to perform operations through a communication bus 93, and when the instructions are executed by the processor, the image processing method executed by the processor side in one or more of the above embodiments is executed.

[0238] It should be noted that in actual application, the various components in the computer device are coupled together through the communication bus 93. It is understood that the communication bus 93 is used to realize the connection and communication between these components. In addition to the data bus, the communication bus 93 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 9 Various buses are labeled as communication buses 93.

[0239] An embodiment of the present application provides a computer storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processors execute the image processing method described in one or more of the above embodiments.

[0240] Among them, the computer-readable storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory (Flash Memory), a magnetic surface storage device, an optical disc, or a compact disc read-only memory (CD-ROM) and other memories.

[0241] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0242] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0243] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0244] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0245] The above description is merely a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application.

Claims

1. A method for processing an image, characterized in that: include: In response to an operation on an input image, identifying a target area of the input image; determining a gain image of the target area; adjusting the gain image using the operation information of the operation to obtain an adjusted gain image; The target area is processed using the adjusted gain image to obtain an output image.

2. The method according to claim 1, characterized in that The adjusting the gain image by using the operation information of the operation to obtain an adjusted gain image includes: determining a gain coefficient according to the operation information; The gain image is adjusted using the gain coefficient to obtain the adjusted gain image.

3. The method according to claim 2, characterized in that The step of determining a gain coefficient according to the operation information includes: The gain coefficient is determined according to the operation duration in the operation information.

4. The method according to claim 3, characterized in that The operation duration is positively correlated with the gain coefficient.

5. The method according to claim 1, wherein The determining the gain image of the target area includes: Determining a target brightness mapping relationship according to the target area; Mapping the target area using the target brightness mapping relationship to obtain the target area after brightness mapping; The gain image is determined according to the target area after brightness processing.

6. The method according to claim 5, characterized in that The step of determining the gain image according to the target area after brightness mapping includes: Determining the average brightness of the target area after the brightness mapping in the logarithmic domain; The gain image is determined based on the average brightness of the target area after brightness mapping in the logarithmic domain, the brightness of the target area after brightness mapping, a first value, and a second value; wherein the first value is the maximum value of the brightness of the target area after brightness mapping and the maximum value of the brightness of the input image; and the second value is the minimum value of the brightness of the target area after brightness mapping and the minimum value of the brightness of the input image.

7. The method according to claim 5, characterized in that The determining of the target brightness mapping relationship according to the target area includes: Counting the brightness values of the target area to obtain a first brightness mapping relationship; Performing statistics on the brightness values of the input image to obtain a second brightness mapping relationship; The target brightness mapping relationship is determined according to the first brightness mapping relationship and the second brightness mapping relationship.

8. The method according to claim 7, characterized in that The determining the target brightness mapping relationship according to the first brightness mapping relationship and the second brightness mapping relationship includes: Determining a weight of the first brightness mapping relationship and a weight of the second brightness mapping relationship; The first brightness mapping relationship and the second brightness mapping relationship are fused based on the weight of the first brightness mapping relationship and the weight of the second brightness mapping relationship to determine the target brightness mapping relationship.

9. The method according to claim 5, characterized in that The method further comprises: The adjusted gain image is adjusted using the target brightness mapping relationship to update the adjusted gain image.

10. The method according to claim 1, characterized in that The step of processing the target area using the adjusted gain image to obtain an output image includes: Processing the target area using the adjusted gain image to obtain a gain-processed target area; The target region is updated according to the target region and the target region after gain processing to obtain the output image.

11. The method according to claim 1, wherein The method further comprises: determining a saturation mapping relationship according to the operation information of the operation; The saturation of the target area is mapped using the saturation mapping relationship to obtain the saturation mapped target area, so as to update the target area.

12. The method according to claim 11, characterized in that The determining of the saturation mapping relationship according to the operation direction in the operation information includes: According to the operation direction in the operation information, a coefficient of the saturation mapping relationship is determined to obtain the saturation mapping relationship.

13. The method according to claim 11, characterized in that The step of processing the target area using the adjusted gain image to obtain an output image includes: Performing color space conversion on the adjusted gain image to obtain a converted gain image; The converted gain image is used to process the target area after the saturation mapping to update the target area and obtain the output image.

14. The method according to any one of claims 1 to 13, characterized in that The method further comprises: determining an intensity image of the target area based on distances between pixels in the target area and endpoints of the input image; The adjusted gain image is adjusted using the intensity image of the target area to update the adjusted gain image.

15. The method according to claim 1, wherein The method further comprises: Determine the area of the target area according to a preset pixel area size and taking the boundary of the target area as the boundary; The target area is updated according to the target area and the domain.

16. The method according to claim 15, characterized in that The updating of the target area according to the target area and the field includes: determining an average brightness of the target area and an average brightness of the field; When the ratio of the average brightness of the field to the average brightness of the target area falls within a preset range, updating the target area by using the union of the target area and the field; When the ratio of the average brightness of the field to the average brightness of the target area does not fall within a preset range, the preset pixel area size is adjusted, and the field of the target area is determined according to the preset pixel area size with the boundary of the target area as the boundary to update the field, and the target area is updated using the union of the target area and the field.

17. An image processing device, characterized in that: include: an identification module, configured to identify a target area of the input image in response to an operation on the input image; A determination module, configured to determine a gain image of the target area; an adjustment module, configured to adjust the gain image using the operation information of the operation to obtain an adjusted gain image; A processing module is used to process the target area using the adjusted gain image to obtain an output image.

18. An ISP, characterized in that: include: a processor, configured to call and run a computer program from a memory, so that a device equipped with the ISP executes the image processing method according to any one of claims 1 to 16; A transceiver is used to send and receive information between a device or an ISP.

19. An electronic device, characterized in that: include: The ISP, the processor, and the storage medium storing instructions executable by the processor according to claim 18; The storage medium relies on the processor to perform operations through a communication bus.

20. An electronic device, characterized in that: include: A processor and a storage medium storing instructions executable by the processor; The storage medium relies on the processor to perform operations through a communication bus, and when the instructions are executed by the processor, the image processing method described in any one of claims 1 to 16 is executed.

21. A computer storage medium, characterized in that Executable instructions are stored, and when the executable instructions are executed by one or more processors, the processors execute the image processing method according to any one of claims 1 to 16.

Images