Image processing method, intelligent device and storage medium

Abstract

An image processing method, an intelligent device and a storage medium, the method comprising the following steps: performing target type region segmentation processing on a target image to obtain first image data (S1); and performing first processing on the target image according to the first image data to obtain a processed image (S2). According to the method, the target type region and the non-target type region can be processed respectively, so that the processing results of the target type region and the non-target type region are more coordinated and natural, and the image processing effect can be improved.

Description

Image processing method, intelligent device and storage medium Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, an intelligent device, and a storage medium. Background Art

[0002] With the rapid development of terminal technology, the functions of mobile terminals such as mobile phones and tablet computers have been continuously improved, and have gradually become one of the commonly used tools in people's daily life and work.

[0003] During the process of conceiving and implementing this application, the inventors discovered that there are at least the following problems: in some implementations, when processing image colors, the entire image is processed identically, resulting in poor processing effects in target type areas and unnatural and uncoordinated processing results with non-target type areas.

[0004] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Technical Solutions

[0005] In response to the above technical problems, the present application provides an image processing method, an intelligent terminal and a storage medium, which can perform target type area segmentation processing on the target image, so that the processing results of the target type area and the non-target type area can be more coordinated and natural, thereby improving the image processing effect.

[0006] To solve the above technical problems, the present application provides an image processing method, comprising the following steps:

[0007] S1: performing target type region segmentation processing on the target image to obtain first image data;

[0008] S2: Performing a first processing on the target image according to the first image data to obtain a processed image.

[0009] Optionally, the step S1 includes:

[0010] S11: Obtain target type region segmentation model;

[0011] S12: performing target type region segmentation processing on the target image according to the target type region segmentation model to obtain second image data;

[0012] S13: Using the target image as a guide image, perform guided filtering processing on the second image data to obtain the first image data.

[0013] The S2 step comprises:

[0014] S21: Identify the target image according to the first image data to obtain a foreground image and a background image;

[0015] S22: performing color enhancement processing on the foreground image and the background image to obtain a foreground enhanced image and a background enhanced image;

[0016] S23: Performing color fusion on the foreground-enhanced image and the background-enhanced image according to the first image data to obtain a processed image.

[0017] Optionally, the target type area is a skin area, or other preset type areas (such as face, sky, moon, sun, landscape, building, car, television, etc.).

[0018] Optionally, the target type area is a skin area, and step S22 includes:

[0019] performing color enhancement processing on the background image according to a preset background color mapping table to obtain a background-enhanced image, or identifying the background type of the target image and performing color enhancement processing on the background image according to a color mapping table corresponding to the background type to obtain the background-enhanced image;

[0020] The foreground image is color enhanced according to a preset skin color mapping table to obtain a foreground enhanced image, or the skin color type of the target image is identified and the foreground image is color enhanced according to a color mapping table corresponding to the skin color type to obtain a foreground enhanced image.

[0021] Optionally, step S23 includes:

[0022] performing color fusion on the foreground-enhanced image and the background-enhanced image according to the first image data to obtain a fused image;

[0023] In response to the adjustment operation, the fused image is adjusted to obtain a processed image.

[0024] Optionally, the method further includes:

[0025] Determining whether the target image includes the target type area;

[0026] If the target type area is included, execute the step S1;

[0027] If the target type area is not included, a second process is performed on the target image according to a preset mapping table and / or adjustment operation.

[0028] This application also provides another image processing method, comprising the following steps:

[0029] S10: Determine whether the target image includes a target type area;

[0030] S20: If the target type area is included, performing a first processing on the target image according to the first image data to obtain a processed image, wherein the first image data is obtained by segmenting the target image corresponding to the target type area.

[0031] Optionally, before step S20, the method further includes:

[0032] Obtain target type region segmentation model;

[0033] performing target type region segmentation processing on the target image according to the target type region segmentation model to obtain second image data;

[0034] The target image is used as a guide image to perform guided filtering processing on the second image data to obtain the first image data.

[0035] Optionally, the target type region segmentation model is a hybrid distance network model, and obtaining the target type region segmentation model includes at least one of the following:

[0036] A lightweight deep neural network is used to extract features from the training dataset and training set labels to obtain feature images.

[0037] By combining the position distance and the high-dimensional feature distance, the relationship between the current pixel and the adjacent pixel points in the feature image is obtained to obtain a fused feature image;

[0038] High-dimensional features are supervised by high-dimensional feature distance loss, so that the high-dimensional features of each pixel in the fused feature image are moved closer to the corresponding feature center during the training process to obtain the target type region segmentation model.

[0039] Optionally, the step S20 includes:

[0040] Display the processed image;

[0041] In response to receiving an adjustment operation on the foreground image and / or the background image in the processed image, an adjusted effect image is displayed.

[0042] Optionally, the method further includes:

[0043] If the target type area is not included, a second process is performed on the target image according to a preset mapping table and / or adjustment operation.

[0044] The present application also provides a smart terminal, which includes: a memory and a processor, wherein the memory stores an image processing program, and when the processing program is executed by the processor, the steps of the image processing method described in any one of the above items are implemented.

[0045] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned image processing methods are implemented.

[0046] As described above, the image processing method, intelligent terminal, and storage medium of the present application include the following steps: performing target type region segmentation processing on the target image to obtain first image data; and performing first processing on the target image based on the first image data to obtain a processed image. The image processing method, intelligent device, and storage medium of the present application can perform target type region segmentation processing on the target image to obtain first image data, and process the target image based on the first image data. Therefore, the target type region and non-target type region can be processed separately, so that the processing results of the target type region and the non-target type region can be made more coordinated and natural, thereby improving the image processing effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for describing the embodiments. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without inventive work.

[0048] FIG1 is a schematic diagram of the hardware structure of a smart device for implementing various embodiments of the present application.

[0049] FIG2 is a diagram of a communication network system architecture provided in an embodiment of the present application.

[0050] FIG3 is a schematic diagram showing the hardware structure of the controller 140 according to an embodiment.

[0051] FIG4 is a schematic diagram showing the hardware structure of the network node 150 according to an embodiment.

[0052] FIG5 is a schematic diagram showing the hardware structure of the network node 160 according to an embodiment.

[0053] FIG6 is a schematic diagram showing the hardware structure of the controller 170 according to an embodiment.

[0054] FIG7 is a schematic diagram showing the hardware structure of the network node 180 according to an embodiment.

[0055] FIG8 is a schematic flow chart of an image processing method according to the first embodiment.

[0056] FIG. 9 is a schematic diagram showing a specific flow of step S1 in the image processing method shown in FIG. 8 according to an embodiment.

[0057] FIG. 10 is a schematic diagram showing a specific flow of step S2 in the image processing method shown in FIG. 8 according to an embodiment.

[0058] FIG. 11 is a schematic flow chart of an image processing method according to the second embodiment.

[0059] The objectives, features, and advantages of this application will be further illustrated with reference to the accompanying drawings and examples. The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and the accompanying text are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of this application to those skilled in the art by reference to specific embodiments.

[0060] Implementation Methods of the Application

[0061] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0062] It should be noted that, in this document, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined based on their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0063] It should be understood that although the terms "first," "second," "third," etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are used solely to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the term "if," as used herein, may be interpreted as "upon," "when," or "in response to a determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It should be further understood that the terms "comprising" and "including" indicate the presence of the recited features, steps, operations, elements, components, items, types, and / or groups, but do not preclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, types, and / or groups. The terms "or," "and / or," "including at least one of the following," etc., as used herein, may be interpreted as inclusive, meaning any one or any combination. For example, “comprising at least one of the following: A, B, C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”; and for another example, “A, B or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”. An exception to this definition will occur only when a combination of elements, functions, steps or operations are inherently mutually exclusive in some manner.

[0064] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence in the order indicated by the arrows. Unless clearly stated herein, the execution of these steps is not strictly limited in order, and they can be performed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0065] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0066] It should be noted that in this article, step codes such as S1, S2, S10, S20 are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the order. When implementing the step, those skilled in the art may execute S2 first and then S1, or may execute S20 first and then S10, etc., but these should all be within the scope of protection of this application.

[0067] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0068] In the subsequent description, the use of suffixes such as "module", "component" or "unit" to represent elements is only for the purpose of facilitating the description of the present application and has no specific meaning. Therefore, "module", "component" or "unit" can be used interchangeably.

[0069] Smart devices can be implemented in various forms. For example, the smart devices described in this application may include smart devices such as mobile phones, tablet computers, laptop computers, PDAs, portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, and other smart devices, as well as fixed terminals such as digital TVs and desktop computers.

[0070] The subsequent description will be made using a mobile terminal as an example. It will be understood by those skilled in the art that, in addition to components specifically for mobile purposes, the configuration according to the embodiments of the present application can also be applied to fixed-type terminals.

[0071] Please refer to Figure 1, which is a schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application. The mobile terminal 100 may include components such as an RF (Radio Frequency) unit 101, a WiFi module 102, an audio output unit 103, an A / V (Audio / Video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111. Those skilled in the art will appreciate that the mobile terminal structure shown in Figure 1 does not limit the mobile terminal. The mobile terminal may include more or fewer components than shown, or may combine certain components, or arrange the components differently.

[0072] The following is a detailed introduction to the various components of the mobile terminal in conjunction with Figure 1:

[0073] The RF unit 101 can be used to send and receive information or receive signals during calls. Specifically, it receives downlink information from the base station and transmits it to the processor 110 for processing. It also transmits uplink data to the base station. Typically, the RF unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, and more. Furthermore, the RF unit 101 can communicate with the network and other devices via wireless communication. The above-mentioned wireless communications may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing- Long Term Evolution), TDD-LTE (Time Division Duplexing- Long Term Evolution) and 5G, etc.

[0074] WiFi is a short-range wireless transmission technology. A mobile terminal, through WiFi module 102, enables users to send and receive emails, browse web pages, and access streaming media, providing wireless broadband Internet access. Although FIG1 illustrates WiFi module 102, it is understood that it is not a required component of the mobile terminal and can be omitted as needed without altering the essence of the invention.

[0075] The audio output unit 103 can convert audio data received by the RF unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output it as sound when the mobile terminal 100 is in a call signal reception mode, a talk mode, a recording mode, a voice recognition mode, a broadcast reception mode, or the like. Furthermore, the audio output unit 103 can also provide audio output related to a specific function performed by the mobile terminal 100 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, or the like.

[0076] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data from still images or videos captured by an image capture device (such as a camera) in video capture mode or image capture mode. The processed image frames may be displayed on the display unit 106. The image frames processed by the GPU 1041 may be stored in the memory 109 (or other storage medium) or transmitted via the RF unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) in various operating modes, such as phone call mode, recording mode, and voice recognition mode, and process such sound into audio data. In phone call mode, the processed audio (voice) data may be converted into a format that can be transmitted to a mobile communication base station via the RF unit 101. The microphone 1042 may implement various noise cancellation (or suppression) algorithms to eliminate (or suppress) noise or interference generated during the reception and transmission of audio signals.

[0077] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, or other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor can adjust the brightness of the display panel 1061 based on the brightness of the ambient light, and the proximity sensor can turn off the display panel 1061 and / or the backlight when the mobile terminal 100 is brought to your ear. An accelerometer, a type of motion sensor, can detect acceleration in all directions (typically three axes) and, when stationary, can detect the magnitude and direction of gravity. This can be used for applications that recognize the phone's posture (e.g., switching between landscape and portrait modes, related games, magnetometer posture calibration), vibration recognition-related functions (e.g., pedometer, tapping), and other functions. Other sensors that may be configured on a mobile phone, such as a fingerprint sensor, pressure sensor, iris sensor, molecular sensor, gyroscope, barometer, hygrometer, thermometer, infrared sensor, etc., are not described here.

[0078] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0079] The user input unit 107 can be used to receive input digital or character information and generate key signal input related to user settings and function control of the mobile terminal. Optionally, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also known as a touch screen, can detect user touch operations on or near it (for example, operations performed on or near the touch panel 1071 using a finger, stylus, or any other suitable object or accessory) and drive corresponding connected devices according to pre-set programs. The touch panel 1071 may include a touch detection device and a touch controller. Optionally, the touch detection device detects the user's touch position and detects signals generated by the touch operation, transmitting the signals to the touch controller. The touch controller receives the touch information from the touch detection device, converts it into touch point coordinates, and then sends it to the processor 110. The touch controller can also receive and execute commands from the processor 110. Furthermore, the touch panel 1071 can be implemented using various types of sensors, including resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may further include other input devices 1072. Optionally, the other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, power keys, etc.), a trackball, a mouse, a joystick, etc., and the specifics are not limited here.

[0080] Optionally, the touch panel 1071 may overlay the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. The processor 110 then provides a corresponding visual output on the display panel 1061 based on the type of touch event. Although in FIG1 , the touch panel 1071 and the display panel 1061 are shown as two separate components to implement the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 and the display panel 1061 may be integrated to implement the input and output functions of the mobile terminal, which is not limited to this specific embodiment.

[0081] The interface unit 108 serves as an interface through which at least one external device can be connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and an external device.

[0082] Memory 109 can be used to store software programs and various data. Memory 109 may primarily include a program storage area and a data storage area. Optionally, the program storage area may store an operating system and at least one application required for a function (such as sound playback or image playback); the data storage area may store data generated based on the use of the phone (such as audio data and a phone book). Memory 109 may also include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0083] Processor 110 is the control center of the mobile terminal, connecting all components of the mobile terminal using various interfaces and circuits. By running or executing software programs and / or modules stored in memory 109 and accessing data stored in memory 109, it executes various functions of the mobile terminal and processes data, thereby providing overall monitoring of the mobile terminal. Processor 110 may include one or more processing units; preferably, processor 110 may integrate an application processor and a modem processor. Optionally, the application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 110.

[0084] The mobile terminal 100 may further include a power supply 111 (such as a battery) for supplying power to various components. Preferably, the power supply 111 may be logically connected to the processor 110 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption.

[0085] Although not shown in FIG. 1 , the mobile terminal 100 may further include a Bluetooth module, etc., which will not be described in detail here.

[0086] To facilitate understanding of the embodiments of the present application, the communication network system on which the mobile terminal of the present application is based is described below.

[0087] Please refer to Figure 2, which is a communication network system architecture diagram provided in an embodiment of the present application. The communication network system is an LTE system of universal mobile communication technology. The LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203 and an operator's IP service 204, which are connected in sequence.

[0088] Optionally, UE201 may be the above-mentioned terminal 100, which will not be described in detail here.

[0089] E-UTRAN 202 includes eNodeB 2021 and other eNodeBs 2022 . Optionally, eNodeB 2021 may be connected to other eNodeBs 2022 via a backhaul (eg, an X2 interface). eNodeB 2021 is connected to EPC 203 , and eNodeB 2021 may provide UE 201 with access to EPC 203 .

[0090] The EPC 203 may include an MME (Mobility Management Entity) 2031, an HSS (Home Subscriber Server) 2032, other MMEs 2033, an SGW (Serving Gate Way) 2034, a PGW (PDN Gate Way) 2035, and a PCRF (Policy and Charging Rules Function) 2036. Optionally, the MME 2031 is a control node that processes signaling between the UE 201 and the EPC 203, providing bearer and connection management. The HSS 2032 provides registers for managing functions such as the Home Location Register (not shown) and stores user-specific information such as service features and data rates. All user data can be sent through SGW2034. PGW2035 can provide IP address allocation and other functions for UE 201. PCRF2036 is the policy and charging control policy decision point for service data flows and IP bearer resources. It selects and provides available policy and charging control decisions for the policy and charging execution function unit (not shown in the figure).

[0091] The IP service 204 may include the Internet, an intranet, an IMS (IP Multimedia Subsystem), or other IP services.

[0092] Although the above introduction takes the LTE system as an example, those skilled in the art should know that this application is not only applicable to the LTE system, but can also be applied to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA and future new network systems (such as 5G), etc., which are not limited here.

[0093] FIG3 is a schematic diagram of the hardware structure of a controller 140 provided in this application. The controller 140 includes a memory 1401 and a processor 1402. The memory 1401 is used to store program instructions, and the processor 1402 is used to call the program instructions in the memory 1401 to execute the steps performed by the controller in the first embodiment of the above method. The implementation principles and beneficial effects are similar and will not be repeated here.

[0094] Optionally, the controller further includes a communication interface 1403, which can be connected to the processor 1402 via a bus 1404. The processor 1402 can control the communication interface 1403 to implement the receiving and sending functions of the controller 140.

[0095] Figure 4 is a schematic diagram of the hardware structure of a network node 150 provided in this application. Network node 150 includes: a memory 1501 and a processor 1502. Memory 1501 is used to store program instructions, and processor 1502 is used to call the program instructions in memory 1501 to execute the steps performed by the first node in the first embodiment of the above method. The implementation principles and beneficial effects are similar and will not be repeated here.

[0096] Optionally, the controller further includes a communication interface 1503, which can be connected to the processor 1502 via a bus 1504. The processor 1502 can control the communication interface 1503 to implement the receiving and sending functions of the network node 150.

[0097] Figure 5 is a schematic diagram of the hardware structure of a network node 160 provided in this application. Network node 160 includes: a memory 1601 and a processor 1602. Memory 1601 is used to store program instructions, and processor 1602 is used to call the program instructions in memory 1601 to execute the steps performed by the intermediate node and the tail node in the first embodiment of the above method. The implementation principles and beneficial effects are similar and will not be repeated here.

[0098] Optionally, the controller further includes a communication interface 1603, which can be connected to the processor 1602 via a bus 1604. The processor 1602 can control the communication interface 1603 to implement the receiving and sending functions of the network node 160.

[0099] FIG6 is a schematic diagram of the hardware structure of a controller 170 provided in this application. The controller 170 includes a memory 1701 and a processor 1702. The memory 1701 is used to store program instructions, and the processor 1702 is used to call the program instructions in the memory 1701 to execute the steps performed by the controller in the second embodiment of the above method. The implementation principles and beneficial effects are similar and will not be repeated here.

[0100] Figure 7 is a schematic diagram of the hardware structure of a network node 180 provided in this application. Network node 180 includes: a memory 1801 and a processor 1802. Memory 1801 is used to store program instructions, and processor 1802 is used to call the program instructions in memory 1801 to execute the steps performed by the first node in the second embodiment of the above method. The implementation principles and beneficial effects are similar and will not be repeated here.

[0101] The integrated modules implemented in the form of software function modules can be stored in a computer-readable storage medium. The software function modules stored in a storage medium include a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute some of the steps of the methods of various embodiments of the present application.

[0102] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive solid state disk, SSD), etc.

[0103] Based on the above-mentioned mobile terminal hardware structure and communication network system, various embodiments of the present application are proposed.

[0104] First embodiment

[0105] FIG8 is a flow chart of an image processing method according to the first embodiment. As shown in FIG8 , the image processing method of the present application includes the following steps:

[0106] S1: performing target type region segmentation processing on the target image to obtain first image data;

[0107] S2: Performing a first processing on the target image according to the first image data to obtain a processed image.

[0108] Optionally, the target image may be a picture taken by a camera of the terminal device, or an image previously stored by the terminal device. Optionally, the target type area may be an area of ​​a preset type, or an area of ​​a type set by the user, such as a skin area.

[0109] Optionally, as shown in FIG9 , step S1: performing target type region segmentation processing on the target image to obtain first image data includes the following steps:

[0110] S11: Obtain target type region segmentation model;

[0111] S12: performing target type region segmentation processing on the target image according to the target type region segmentation model to obtain second image data.

[0112] Optionally, the target type region segmentation model is a Hybrid Distance Network (HDNet) model.

[0113] Optionally, step S11: obtaining a target type region segmentation model includes at least one of the following: extracting features from a training dataset and training set labels using a lightweight deep neural network (e.g., MobileNet V3) to obtain a feature image; obtaining the relationship between a current pixel and adjacent pixels in the feature image by combining position distance and high-dimensional feature distance to obtain a fused feature image; and supervising high-dimensional features using high-dimensional feature distance loss (HFD Loss) so that the high-dimensional features of each pixel in the fused feature image approach the corresponding feature center during training to obtain the target type region segmentation model. Optionally, a location-aware attention module (LAA) is used to obtain the relationship between the current pixel and adjacent pixels in the feature image by combining position distance and high-dimensional feature distance, which can enhance the perception of object size and enhance object-level continuity.

[0114] Optionally, in order to adapt to target images of different resolutions, the second image data (also known as the initial mask or the initial mask) needs to be converted to the resolution size of the target image, which will cause edge blurring and affect the subsequent first processing, such as color enhancement processing. Therefore, it is necessary to perform guided filtering on the second image data to perform edge over-processing on the second image data to obtain the first image data (also known as the processed mask or the processed mask), thereby reducing the color contrast after color enhancement. Among them, guided filtering (or guided filtering) is an image filtering technology that filters the initial image data through a guiding map so that the final output image is generally similar to the initial image data, but the texture part is similar to the guiding map, which can smooth the image and maintain the regional edge, and has the characteristics of fast processing speed. Optionally, as shown in Figure 9, step S1: performing target type region segmentation processing on the target image to obtain the first image data, also includes the following steps:

[0115] S13: Using the target image as a guide image, perform guided filtering processing on the second image data to obtain the first image data.

[0116] Optionally, as shown in FIG10 , the step S2 includes:

[0117] S21: Identify the target image according to the first image data to obtain a foreground image and a background image;

[0118] S22: performing color enhancement processing on the foreground image and the background image to obtain a foreground enhanced image and a background enhanced image;

[0119] S23: Performing color fusion on the foreground-enhanced image and the background-enhanced image according to the first image data to obtain a processed image.

[0120] Optionally, the first image data obtained through target type area segmentation processing is used to identify each pixel in the target image to generate a foreground image including only the foreground and a background image including only the background, and color enhancement processing is performed on the foreground image and the background image respectively to obtain a foreground enhanced image and a background enhanced image, and then the foreground enhanced image and the background enhanced image are color fused through the first image data to obtain a processed image, so that the processing result of each pixel point can be fused with the processing result corresponding to the target type area and the processing result corresponding to the non-target type area, so that the target type area and the non-target type area are more coordinated and natural, thereby improving the image processing effect.

[0121] Optionally, when the target type area is a skin area, step S22 includes:

[0122] performing color enhancement processing on the background image according to a preset background color mapping table to obtain a background-enhanced image, or identifying the background type of the target image and performing color enhancement processing on the background image according to a color mapping table corresponding to the background type to obtain the background-enhanced image;

[0123] The foreground image is color enhanced according to a preset skin color mapping table to obtain a foreground enhanced image, or the skin color type of the target image is identified and the foreground image is color enhanced according to a color mapping table corresponding to the skin color type to obtain a foreground enhanced image.

[0124] Optionally, a pre-trained scene classification model can be used to classify the target image to obtain the scene of the target image, such as beach, party, indoor, etc. The scene of the target image can correspond to the background type of the target image. Then, the corresponding color mapping table is selected according to the background type. The color mapping table contains color mapping relationships such as hue, contrast, saturation and brightness. It can be a mapping table configured by the system or a mapping table set and saved by the user. The color mapping table can be stored using a 3D look-up table (LUT). After determining the color mapping table, each pixel of the background image is processed as follows: trilinear interpolation is used in the color mapping table according to the RGB value of the pixel to obtain the corresponding converted color to obtain a background enhanced image. Optionally, the background image can also be color enhanced according to a preset background color mapping table to obtain a background enhanced image.

[0125] Alternatively, a pre-trained skin color classification model can be used to classify the target image and determine its skin color type. Next, a corresponding color mapping table is selected based on the skin color type. After determining the color mapping table, each pixel in the foreground image is processed as follows: trilinear interpolation is performed on the color mapping table based on the pixel's RGB value to obtain the corresponding converted color, thereby generating a foreground-enhanced image. Alternatively, color enhancement processing can be performed on the foreground image based on a preset skin color mapping table to obtain a foreground-enhanced image.

[0126] Optionally, step S23 includes:

[0127] performing color fusion on the foreground-enhanced image and the background-enhanced image according to the first image data to obtain a fused image;

[0128] In response to the adjustment operation, the fused image is adjusted to obtain a processed image.

[0129] Optionally, the color enhancement level of the fused image can be adjusted in response to a level adjustment operation on the displayed fused image, so that the color enhancement effect of the target type area and other areas in the processed image is more coordinated and natural, further improving the image processing effect.

[0130] Optionally, the method further includes:

[0131] Determining whether the target image includes the target type area;

[0132] If the target type area is included, execute the step S1;

[0133] If the target type area is not included, a second process is performed on the target image according to a preset mapping table and / or adjustment operation.

[0134] Optionally, for example, when the target type area is a portrait area, if the target image includes a portrait area, the target image is subjected to portrait area segmentation processing to obtain first image data, and the target image is subjected to first processing based on the first image data to obtain a processed image. For example, the foreground image and background image in the target image are obtained based on the first image data, and the background image and the foreground image are subjected to corresponding color enhancement processing respectively to obtain a foreground enhanced image and a background enhanced image, and the foreground enhanced image and the background enhanced image are subjected to fusion processing based on the first image data to obtain a processed image; if the target image does not include a portrait area, all areas of the target image are adjusted according to a preset mapping table. Therefore, when the target image includes a target type area, the target type area and the non-target type area can be processed separately, so that the processing results of the target type area and the non-target type area can be made more coordinated and natural. When the target image does not include a target type area, all areas of the target image are adjusted according to the preset mapping table, which can further improve the flexibility of image processing.

[0135] The image processing method of the present application includes the following steps: performing target-type region segmentation processing on a target image to obtain first image data; and performing first processing on the target image based on the first image data to obtain a processed image. After obtaining the first image data by using target-type region segmentation processing, the present application performs first processing on the target image based on the first image data. This can make the processing results of target-type regions and non-target-type regions more coordinated and natural, thereby improving the image processing effect, and is particularly suitable for scenes with portraits.

[0136] Second embodiment

[0137] FIG11 is a flow chart of an image processing method according to a second embodiment. As shown in FIG11 , the image processing method of the present application includes the following steps:

[0138] S10: Determine whether the target image includes a target type area;

[0139] S20: If the target type area is included, performing a first processing on the target image according to the first image data to obtain a processed image, wherein the first image data is obtained by segmenting the target image corresponding to the target type area.

[0140] Optionally, the target image may be a picture captured by a camera of the terminal device, and the step S10 may include converting the target image in NV21 format into a target image in RGB format. Optionally, the target type area may be an area of ​​a preset type or an area of ​​a type set by the user, such as a skin area.

[0141] Optionally, before the S20 step, it also includes: obtaining a target type region segmentation model; performing target type region segmentation processing on the target image according to the target type region segmentation model to obtain second image data; using the target image as a guide map, performing guided filtering processing on the second image data to obtain the first image data.

[0142] Optionally, the target type region segmentation model is a hybrid distance network model, and obtaining the target type region segmentation model includes at least one of the following: performing feature extraction on the training data set and the training set label through a lightweight deep neural network to obtain a feature image; obtaining the relationship between the current pixel point and the adjacent pixel points in the feature image by combining the position distance and the high-dimensional feature distance to obtain a fused feature image; supervising the high-dimensional features through the high-dimensional feature distance loss so that the high-dimensional features of each pixel point in the fused feature image approach the corresponding feature center during the training process to obtain the target type region segmentation model.

[0143] Optionally, to further enhance the flexibility of image processing and the coordination of the resulting image, the foreground image and / or background image in the processed image may be further adjusted based on an adjustment operation (e.g., a voice adjustment command, a gesture adjustment command, etc.). Following step S20, the process includes: displaying the processed image; and, in response to receiving an adjustment operation for the foreground image and / or background image in the processed image, displaying the adjusted resulting image.

[0144] Optionally, the method further includes: step S30: if the target type area is not included, performing a second processing on the target image according to a preset mapping table and / or adjustment operation.

[0145] Optionally, after step S20, the method includes converting the format of the processed image from RGB format to NV21 format and then outputting the converted image.

[0146] Optionally, a pre-trained scene classification model can be used to classify the target image to obtain the scene of the target image, such as outdoor scene, indoor scene, green plant scene, etc., and then the corresponding preset mapping table is obtained according to the scene of the target image, and the target image is subjected to a second processing, such as performing the same level of color enhancement processing on all areas of the target image.

[0147] The specific implementation process of the above steps is the same as the implementation process of the relevant steps in the first embodiment, and will not be repeated here. The embodiment of the present application also provides an intelligent device, which includes a memory and a processor, and the memory stores a processing program. When the processing program is executed by the processor, the steps of the image processing method in any of the above embodiments are implemented.

[0148] An embodiment of the present application further provides a computer-readable storage medium, on which a processing program is stored. When the processing program is executed by a processor, the steps of the image processing method in any of the above embodiments are implemented.

[0149] In the embodiments of the smart device and computer-readable storage medium provided in the embodiments of the present application, all technical features of any of the above-mentioned image processing method embodiments may be included. The expanded and explained content of the specification is basically the same as that of the various embodiments of the above-mentioned methods and will not be repeated here.

[0150] An embodiment of the present application further provides a computer program product, which includes computer program code. When the computer program code runs on a computer, the computer executes the methods in the various possible implementation modes described above.

[0151] An embodiment of the present application also provides a chip, including a memory and a processor, wherein the memory is used to store computer programs, and the processor is used to call and run the computer programs from the memory, so that a device equipped with the chip executes the methods in the various possible implementation modes as described above.

[0152] It is understood that the above scenarios are merely examples and do not limit the application scenarios of the technical solutions provided in the embodiments of this application. The technical solutions of this application can also be applied to other scenarios. For example, those skilled in the art will appreciate that with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application will also be applicable to similar technical problems.

[0153] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0154] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0155] The units in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0156] In this application, the same or similar terminology, technical solutions and / or application scenario descriptions are generally only described in detail the first time they appear. When they appear again later, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of this application, for the same or similar terminology, technical solutions and / or application scenario descriptions that are not described in detail later, you can refer to the previous relevant detailed descriptions.

[0157] In this application, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0158] The various technical features of the technical solution of this application can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0159] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as mentioned above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the method of each embodiment of the present application.

[0160] In the above embodiments, all or part of the embodiments can be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that integrates one or more available media. Available media can include magnetic media (e.g., floppy disks, storage disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0161] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image processing method, wherein: The following steps are involved: S1: performing target type region segmentation processing on the target image to obtain first image data; S2: Performing a first processing on the target image according to the first image data to obtain a processed image.

2. The method according to claim 1, wherein The S1 step includes: S11: Obtain target type region segmentation model; S12: performing target type region segmentation processing on the target image according to the target type region segmentation model to obtain second image data; S13: Using the target image as a guide image, perform guided filtering processing on the second image data to obtain the first image data.

3. The method according to any one of claims 1 to 2, wherein The S2 step comprises: S21: Identify the target image according to the first image data to obtain a foreground image and a background image; S22: performing color enhancement processing on the foreground image and the background image to obtain a foreground enhanced image and a background enhanced image; S23: Performing color fusion on the foreground-enhanced image and the background-enhanced image according to the first image data to obtain a processed image.

4. The method according to claim 3, wherein: The target type area is a skin area, and step S22 includes: performing color enhancement processing on the background image according to a preset background color mapping table to obtain a background-enhanced image, or identifying the background type of the target image and performing color enhancement processing on the background image according to a color mapping table corresponding to the background type to obtain the background-enhanced image; The foreground image is color enhanced according to a preset skin color mapping table to obtain a foreground enhanced image, or the skin color type of the target image is identified and the foreground image is color enhanced according to a color mapping table corresponding to the skin color type to obtain a foreground enhanced image.

5. The method according to claim 3, wherein: The step S23 includes: performing color fusion on the foreground-enhanced image and the background-enhanced image according to the first image data to obtain a fused image; In response to the adjustment operation, the fused image is adjusted to obtain a processed image.

6. An image processing method, wherein: The following steps are involved: S10: Determine whether the target image includes a target type area; S20: If the target type area is included, performing a first processing on the target image according to the first image data to obtain a processed image, wherein the first image data is obtained by segmenting the target image corresponding to the target type area.

7. The method according to claim 6, wherein: Before the step S20, the method further includes: Obtain target type region segmentation model; performing target type region segmentation processing on the target image according to the target type region segmentation model to obtain second image data; The target image is used as a guide image to perform guided filtering processing on the second image data to obtain the first image data.

8. The method according to claim 7, wherein: The target type region segmentation model is a hybrid distance network model. Obtaining the target type region segmentation model includes at least one of the following: A lightweight deep neural network is used to extract features from the training dataset and training set labels to obtain feature images. By combining the position distance and the high-dimensional feature distance, the relationship between the current pixel and the adjacent pixel points in the feature image is obtained to obtain a fused feature image; High-dimensional features are supervised by high-dimensional feature distance loss, so that the high-dimensional features of each pixel in the fused feature image are moved closer to the corresponding feature center during the training process to obtain the target type region segmentation model.

9. The method according to claim 6, wherein: The step S20 then includes: Display the processed image; In response to receiving an adjustment operation on the foreground image and / or the background image in the processed image, an adjusted effect image is displayed.

10. The method according to any one of claims 6 to 9, wherein The method further comprises: If the target type area is not included, a second process is performed on the target image according to a preset mapping table and / or adjustment operation.

11. A smart device, wherein: The smart device includes: a memory, a processor, and an image processing program stored in the memory and executable on the processor. When the image processing program is executed by the processor, the steps of the image processing method according to claim 1 are implemented.

12. A storage medium, wherein: The storage medium stores a computer program, which implements the steps of the image processing method according to claim 1 when executed by a processor.

Images