Image processing method, intelligent terminal and storage medium

Abstract

The invention provides an image processing method, an intelligent terminal and a storage medium, the image processing method is applied to a terminal device, and the method comprises the following steps: when a preset condition is satisfied, correcting at least a partial region of a shot picture according to an attitude parameter of the terminal device. Through the technical scheme of the invention, the optimization function of the shot picture can be realized, and the picture effect of the shot picture is optimized based on the attitude parameter of the terminal equipment, so that the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an image processing method, an intelligent terminal and a storage medium. Background Art

[0002] With the development of the times, smart terminals have also been widely used and popularized, and more and more users like to use smart terminals (such as mobile phones) to take pictures.

[0003] In one implementation, during the process of conceiving and implementing the present application, the inventors discovered that there are at least the following problems: when a smart terminal is shooting, the picture effect of the captured picture may be limited by the posture of the smart terminal during shooting and may cause distortion.

[0004] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the invention

[0005] In response to the above technical problems, the present application provides an image processing method, an intelligent terminal and a storage medium, which can optimize the picture effect of the captured picture based on the posture parameters of the terminal device, thereby improving the user experience.

[0006] The present application provides an image processing method which can be applied to a smart terminal, comprising: when a preset condition is met, correcting at least a partial area of ​​a captured image according to a posture parameter of a terminal device.

[0007] Optionally, the preset conditions are met, including at least one of the following: The current shooting mode is macro mode; The currently activated camera module is the wide-angle module; Identify the distorted area in the captured image; A first operation is received.

[0008] Optionally, at least part of the area includes at least one of the following: Distorted area of ​​the captured image; The document area of ​​the captured image; The picture area of ​​the captured image; The entire area of ​​the captured image.

[0009] Optionally, the posture parameter includes at least one of the following: The folding angle of the folding screen; The entire device's tilt angle.

[0010] Optionally, at least a partial area of ​​the captured image is corrected according to the posture parameters of the terminal device, including: calculating a distortion restoration ratio according to the posture parameters of the terminal device; expanding target geometric parameters of at least a partial area of ​​the captured image according to the distortion restoration ratio to obtain an intermediate image; and performing a size adjustment operation on the intermediate image.

[0011] Optionally, the distortion restoration ratio is calculated according to the posture parameter of the terminal device, including any of the following: Calculate the distortion restoration ratio based on the folding angle, the straight-line distance from the optical axis of the camera module to the folding axis, and the viewing angle of the camera module; The distortion restoration ratio is calculated based on the tilt angle of the entire machine, the straight-line distance from the optical axis of the camera module to the tilt fulcrum of the entire machine, and the viewing angle of the camera module.

[0012] Optionally, the target geometric parameter includes at least one of the length of the bottom edge of at least a portion of the captured image and the distance between two bottom edges.

[0013] Optionally, the resizing operation includes at least one of the following: Based on the first preset parameter, cropping the intermediate image; The cropped image is scaled based on the second preset parameter.

[0014] The present application also provides an intelligent terminal, comprising: a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the image processing method described in any one of the above items are implemented.

[0015] The present application also provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image processing method described in any one of the above items are implemented.

[0016] The present application provides an image processing method, a smart terminal and a storage medium. The image processing method is applied to a smart terminal, and includes the steps of: when a preset condition is met, correcting at least a part of the area of ​​the captured image according to the posture parameters of the terminal device. Through the technical solution of the present application, the optimization function of the captured image can be realized, so as to optimize the image effect of the captured image based on the posture parameters of the terminal device, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0018] Figure 1 A schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application.

[0019] Figure 2 A communication network system architecture diagram provided for an embodiment of the present application.

[0020] Figure 3 It is a flowchart of the image processing method shown in the embodiment of the present application.

[0021] Figure 4 It is a structural diagram of the folding screen mobile phone of the example of this application.

[0022] Figure 5 It is a schematic diagram of the calculation scenario of the distortion restoration ratio of the example of this application.

[0023] Figure 6 It is a schematic diagram of a scene for correcting a captured image in an example of the present application.

[0024] The realization of the purpose, functional features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. The above-mentioned drawings have shown clear embodiments of this application, which will be described in more detail later. These drawings and textual descriptions are not intended to limit the scope of the concept of this application in any way, but to illustrate the concept of this application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0025] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0026] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0027] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in this article, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination". Furthermore, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising" and "including" indicate that there are the described features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or", "and / or", "including at least one of the following" etc. used in this application can be interpreted as inclusive, or mean any one or any combination. For example, “comprising at least one of the following: A, B, C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”, and for another example, “A, B or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”. An exception to this definition will only occur when a combination of elements, functions, steps or operations are inherently mutually exclusive in some manner.

[0028] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are displayed in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and it can be performed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0029] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0030] It should be noted that in this article, step codes such as S1 are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the sequence.

[0031] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0032] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0033] The smart terminal may be implemented in various forms. For example, the smart terminal described in this application may include smart terminals such as mobile phones, tablet computers, laptop computers, PDAs, portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, etc., as well as fixed terminals such as digital TVs and desktop computers.

[0034] The following description will be made by taking a mobile terminal as an example, and those skilled in the art will understand that, in addition to components specifically used for mobile purposes, the construction according to the embodiments of the present application can also be applied to fixed-type terminals.

[0035] See also Figure 1, which is a schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application, the mobile terminal 100 may include: RF (Radio Frequency, radio frequency) unit 101, WiFi module 102, audio output unit 103, A / V (audio / video) input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, processor 110, and power supply 111 and other components. Those skilled in the art can understand that Figure 1 The structure of the mobile terminal shown in the figure does not constitute a limitation on the mobile terminal. The mobile terminal may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0036] Combine the following Figure 1 The following is a detailed introduction to the various components of the mobile terminal: The radio frequency unit 101 can be used for receiving and sending signals during information transmission or communication. Specifically, after receiving the downlink information of the base station, it is sent to the processor 110 for processing; in addition, the uplink data is sent to the base station. Generally, the radio frequency unit 101 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the radio frequency unit 101 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communications may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing- Long Term Evolution), TDD-LTE (Time Division Duplexing- Long Term Evolution), 5G and 6G, etc.

[0037] WiFi is a short-range wireless transmission technology. Mobile terminals can help users send and receive emails, browse web pages, and access streaming media through WiFi module 102. It provides users with wireless broadband Internet access. Figure 1 The WiFi module 102 is shown, but it is understandable that it is not an essential component of the mobile terminal and can be omitted as needed without changing the essence of the invention.

[0038] The audio output unit 103 can convert the audio data received by the RF unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output it as sound when the mobile terminal 100 is in a call signal reception mode, a talk mode, a recording mode, a voice recognition mode, a broadcast reception mode, etc. Moreover, the audio output unit 103 can also provide audio output related to a specific function performed by the mobile terminal 100 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0039] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processor (GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 106. The image frame processed by the graphics processor 1041 can be stored in the memory 109 (or other storage medium) or sent via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) via the microphone 1042 in a telephone call mode, a recording mode, a voice recognition mode, and the like, and can process such sound into audio data. The processed audio (voice) data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 101 in the case of a telephone call mode. The microphone 1042 can implement various types of noise elimination (or suppression) algorithms to eliminate (or suppress) noise or interference generated in the process of receiving and sending audio signals.

[0040] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 1061 and / or the backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can also be configured on the mobile phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0041] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0042] The user input unit 107 can be used to receive input digital or character information, and generate key signal input related to user settings and function control of the mobile terminal. Optionally, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also known as a touch screen, can collect user touch operations on or near it (such as operations performed by users using fingers, styluses, or any other suitable objects or accessories on or near the touch panel 1071), and drive corresponding connection devices according to a pre-set program. The touch panel 1071 may include a touch detection device and a touch controller. Optionally, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into touch point coordinates, and then sends it to the processor 110, and can receive and execute commands sent by the processor 110. In addition, the touch panel 1071 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 1071, the user input unit 107 may also include other input devices 1072. Optionally, the other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, a function key (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, etc., which are not specifically limited here.

[0043] Optionally, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. Then, the processor 110 provides corresponding visual output on the display panel 1061 according to the type of touch event. Figure 1 In the figure, the touch panel 1071 and the display panel 1061 are used as two independent components to implement the input and output functions of the mobile terminal. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to implement the input and output functions of the mobile terminal, which is not limited here.

[0044] The interface unit 108 serves as an interface through which at least one external device can be connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and an external device.

[0045] The memory 109 can be used to store software programs and various data. The memory 109 can mainly include a program storage area and a data storage area. Optionally, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 109 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0046] The processor 110 is the control center of the mobile terminal. It uses various interfaces and lines to connect various parts of the entire mobile terminal. It executes various functions of the mobile terminal and processes data by running or executing software programs and / or modules stored in the memory 109, and calling data stored in the memory 109, so as to monitor the mobile terminal as a whole. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor. Optionally, the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 110.

[0047] The mobile terminal 100 may further include a power supply 111 (such as a battery) for supplying power to various components. Preferably, the power supply 111 may be logically connected to the processor 110 via a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system.

[0048] although Figure 1 Not shown, the mobile terminal 100 may further include a Bluetooth module, etc., which will not be described in detail here.

[0049] In order to facilitate understanding of the embodiments of the present application, the communication network system on which the mobile terminal of the present application is based is described below.

[0050] See also Figure 2 , Figure 2 A communication network system architecture diagram is provided for an embodiment of the present application, wherein the communication network system is an LTE system of universal mobile communication technology, and the LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203 and an operator's IP service 204, which are connected in sequence for communication.

[0051] Optionally, UE201 may be the above-mentioned terminal 100, which will not be described in detail here.

[0052] E-UTRAN 202 includes eNodeB 2021 and other eNodeBs 2022 , etc. Optionally, eNodeB 2021 may be connected to other eNodeBs 2022 via a backhaul (eg, an X2 interface), and eNodeB 2021 is connected to EPC 203 , and eNodeB 2021 may provide UE 201 with access to EPC 203 .

[0053] EPC203 may include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving GateWay) 2034, PGW (PDN Gate Way) 2035, and PCRF (Policy and Charging Rules Function) 2036. Optionally, MME 2031 is a control node that processes signaling between UE 201 and EPC 203, and provides bearer and connection management. HSS 2032 is used to provide some registers to manage functions such as a home location register (not shown in the figure), and store some user-specific information such as service features and data rates. All user data can be sent through SGW2034. PGW2035 can provide IP address allocation and other functions for UE 201. PCRF2036 is the policy and charging control policy decision point for service data flow and IP bearer resources. It selects and provides available policy and charging control decisions for the policy and charging execution function unit (not shown in the figure).

[0054] The IP service 204 may include the Internet, an intranet, an IMS (IP Multimedia Subsystem) or other IP services.

[0055] Although the above introduction takes the LTE system as an example, those skilled in the art should know that the present application is not only applicable to the LTE system, but also to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, 5G and future new network systems (such as 6G), etc., without limitation here.

[0056] Based on the above-mentioned mobile terminal hardware structure and communication network system, various embodiments of the present application are proposed.

[0057] Reference Figure 3 , Figure 3 : is a flowchart of an image processing method shown in an embodiment of the present application. The image processing method in the embodiment of the present application can be applied to a smart terminal (such as a mobile phone), and includes the following steps (for example, step S1): Step S1, when a preset condition is met, correct at least a partial area of ​​the captured image according to the posture parameters of the terminal device.

[0058] Optionally, the preset conditions may represent various functional scenarios that may cause a risk of distortion in the captured image, and / or received user operations.

[0059] Optionally, the preset conditions are met, including but not limited to at least one of the following: The current shooting mode is macro mode; The currently activated camera module is the wide-angle module; Identify the distorted area in the captured image; A first operation is received.

[0060] Optionally, macro mode is a shooting function that allows the camera to shoot small objects such as insects, flowers, coins, etc. at a very close distance without losing clarity. The following are some key features of macro mode: (1) Close-up shooting: Macro mode usually allows the camera to shoot at a distance of several centimeters to tens of centimeters, which is much closer than the minimum focusing distance of normal mode; (2) High magnification: Macro photography can provide high magnification, which means that the object photographed appears larger in the photo than it actually is.

[0061] Optionally, in macro mode, since the shooting module is usually close to the object being photographed, it is easily affected by the posture of the terminal device, resulting in the risk of distortion in the captured image. Therefore, when the current shooting mode is macro mode, it can be directly determined that the preset conditions are met, so that at least part of the captured image is corrected according to the posture parameters of the terminal device to optimize the image effect of the captured image; or, when the current shooting mode is macro mode, it can be combined with other distortion risk determination factors (such as the currently activated camera module, the distortion recognition result of the captured image, at least one of the received user operations, etc.) to jointly determine whether the preset conditions are met, and when it is determined that the preset conditions are met, at least part of the captured image is corrected according to the posture parameters of the terminal device to optimize the image effect of the captured image. In this way, the technical solution of this embodiment can accurately predict the correction needs or timing for the captured image based on multiple distortion risk determination factors.

[0062] Optionally, a wide-angle module is a type of camera compact module (CCM) that uses a wide-angle lens, allowing the camera module to capture a wider field of view (FOV) than a standard lens. The focal length of a wide-angle lens is shorter, which can be between 6-35 mm, and the viewing angle is greater than that of a standard lens, approximately above 53° up to 220°.

[0063] Optionally, the wide-angle module has at least the following features: (1) Wide viewing angle: It can cover a larger scene and is suitable for landscape, architectural and large group photography that require a wide field of view; (2) Short focal length: Wide-angle modules have a shorter focal length, which allows them to capture more image content at the same shooting distance; (3) Long depth of field: Wide-angle lenses usually have a large depth of field, which allows both the foreground and background to maintain good clarity; (4) High dynamic range: Some wide-angle modules can provide a wider dynamic range and capture more light and dark details.

[0064] Optionally, due to the wide viewing angle of the wide-angle module, the picture taken by the wide-angle module may have perspective distortion, especially at the edge of the picture, and when perspective distortion occurs, the degree of distortion is easily affected by the posture of the terminal device. Therefore, when the currently activated camera module is a wide-angle module, it can be directly determined that the preset conditions are met, so that at least part of the area of ​​the captured picture is corrected according to the posture parameters of the terminal device to optimize the picture effect of the captured picture; or, when the currently activated camera module is a wide-angle module, it can be combined with other distortion risk determination factors (such as the current shooting mode, the distortion recognition result of the captured picture, at least one of the received user operations, etc.) to jointly determine whether the preset conditions are met, and when it is determined that the preset conditions are met, at least part of the area of ​​the captured picture is corrected according to the posture parameters of the terminal device to optimize the picture effect of the captured picture. In this way, the technical solution of this embodiment can accurately predict the correction needs or timing of the captured picture based on multiple distortion risk determination factors.

[0065] Optionally, the wide-angle module in this embodiment can support macro mode. In this way, this embodiment can turn on the macro mode after activating the wide-angle module to capture more background picture content and focused detail picture content, thereby improving the shooting performance of the macro mode.

[0066] Optionally, after the wide-angle module is activated, when it is detected that the focus distance meets the close-range focus condition, the macro mode is automatically turned on, and it is determined that the preset conditions are met, so as to perform the optimization function of the captured image, so as to correct at least part of the captured image according to the posture parameters of the terminal device to optimize the image effect of the captured image. In this way, the technical solution of this embodiment can improve the image effect of the captured image obtained by the wide-angle module supporting the macro mode, thereby improving the user experience.

[0067] Optionally, after activating the wide-angle module and turning on the macro mode, it is more susceptible to the posture of the terminal device, resulting in the risk of distortion in the captured image. Therefore, the technical solution of this embodiment can determine that the preset condition is met when the wide-angle module is activated and the macro mode is turned on.

[0068] Optionally, the method of identifying the distorted area in the captured image may be to use AI recognition technology.

[0069] Optionally, the technical solution of this embodiment can directly determine whether the preset conditions are met when it is identified that there are distorted areas in the captured image, so as to correct at least part of the captured image according to the posture parameters of the terminal device to optimize the picture effect of the captured image; or, when it is identified that there are distorted areas in the captured image, it can be combined with other distortion risk determination factors (such as the current shooting mode, the currently activated camera module, at least one of the received user operations, etc.) to jointly determine whether the preset conditions are met, and when it is determined that the preset conditions are met, at least part of the captured image can be corrected according to the posture parameters of the terminal device to optimize the picture effect of the captured image. In this way, the technical solution of this embodiment can accurately predict the need or timing for correction of the captured image based on multiple distortion risk determination factors.

[0070] Optionally, the first operation can be associated with the optimization function of the captured image. Optionally, the technical solution of this embodiment can identify the active needs of the user when receiving the first operation, and can directly determine whether the preset conditions are met, so as to correct at least part of the captured image according to the posture parameters of the terminal device to optimize the image effect of the captured image; or, when receiving the first operation, it can be combined with other distortion risk determination factors (such as the current shooting mode, the currently activated camera module, at least one of the distortion recognition results of the captured image, etc.) to jointly determine whether the preset conditions are met, and when it is determined that the preset conditions are met, at least part of the captured image is corrected according to the posture parameters of the terminal device to optimize the image effect of the captured image. In this way, the technical solution of this embodiment can avoid users from accidentally touching the optimization function of the captured image based on multiple distortion risk determination factors.

[0071] Optionally, at least a partial area of ​​the captured image may represent the entire or partial image area that affects the user's perception.

[0072] Optionally, at least part of the area includes but is not limited to at least one of the following: Distorted area of ​​the captured image; The document area of ​​the captured image; The picture area of ​​the captured image; The entire area of ​​the captured image.

[0073] The technical solution of this embodiment can achieve overall optimization of the captured image or targeted optimization of a specific object.

[0074] Optionally, the posture parameter may represent various physical parameters (such as angle parameters) of the smart terminal during shooting, and is used to determine relevant parameters of the shooting viewing angle.

[0075] Optionally, the relevant parameters of the shooting viewing angle may include the size of the viewing angle area, the object distance, etc.

[0076] Optionally, the posture parameters include but are not limited to at least one of the following: The folding angle of the folding screen; The entire device's tilt angle.

[0077] The technical solution of this embodiment can optimize the picture effect of the captured image (for example, optimize the perspective distortion of the captured image) based on the folding angle of the folding screen and / or the overall tilt angle of the terminal device.

[0078] Optionally, correcting at least a portion of the captured image according to the posture parameters of the terminal device may include: calculating a distortion restoration ratio according to the posture parameters of the terminal device; expanding target geometric parameters of at least a portion of the captured image according to the distortion restoration ratio to obtain an intermediate image; and performing a size adjustment operation on the intermediate image.

[0079] Optionally, the distortion restoration ratio is calculated according to the posture parameters of the terminal device, and the distortion restoration ratio can be calculated according to the posture parameters of the terminal device and the shooting principle of small far and large near. Optionally, the shooting principle of small far and large near is the main reason for the distorted area in the shooting picture, so the distortion restoration ratio can be a manifestation of the adjustment rule of large far and small near.

[0080] Optionally, taking the captured image as a rectangular image as an example, based on the distortion restoration ratio, the farthest bottom edge of the rectangular image can be expanded mainly from the starting point of the viewing angle, so that the rectangular image is expanded into a trapezoidal image (i.e., the intermediate image), thereby improving the distorted area in the captured image, and then the trapezoidal image can be cropped and scaled according to the target display size to obtain a rectangular image with improved distortion, and used as the target captured image.

[0081] Optionally, the target geometric parameter may include at least one of the length of the bottom edge of at least a portion of the captured image, the distance between two opposite bottom edges of at least a portion of the captured image, and the like.

[0082] Optionally, the resizing operation includes at least one of the following: Based on the first preset parameter, cropping the intermediate image; The cropped image is scaled based on the second preset parameter.

[0083] Optionally, the first preset parameter may correspond to a target geometry of at least a portion of the captured image. Optionally, the target geometry may represent a cropping target.

[0084] Optionally, the second preset parameter may correspond to a target display ratio or a target display size of at least a partial area of ​​the captured image.

[0085] Optionally, the distortion restoration ratio is calculated according to the posture parameters of the terminal device, including but not limited to any of the following: Calculate the distortion restoration ratio based on the folding angle, the straight-line distance from the optical axis of the camera module to the folding axis, and the viewing angle of the camera module; The distortion restoration ratio is calculated based on the tilt angle of the entire machine, the straight-line distance from the optical axis of the camera module to the tilt fulcrum of the entire machine, and the viewing angle of the camera module.

[0086] Optionally, the folding angle can be detected by a sensor in the smart terminal. Optionally, the folding angle can be detected by a sensor at the folding axis position. For example, a magnet is configured at the folding axis (or rotating axis) position of a folding screen mobile phone, and the magnetic sensor is located in the lower half of the screen. When the folding screen is opened or closed, the position of the magnet will change accordingly. The magnetic sensor can detect the change in magnetic field strength, thereby calculating the folding angle.

[0087] Optionally, the tilt angle of the entire device may be detected by a sensor in the smart terminal.

[0088] Optionally, the smart terminal may have built-in sensors such as accelerometers, gyroscopes, and magnetometers, which can cooperate with each other to determine the tilt angle and / or folding angle of the whole device, etc. These sensors can measure the elevation angle, depression angle, or tilt angle of the smart terminal relative to the direction of gravity.

[0089] Optionally, when the whole machine tilt of the smart terminal is achieved by relying on a fixed bracket, the whole machine tilt angle may be detected by a sensor of the fixed bracket and fed back to the smart terminal.

[0090] Optionally, the straight-line distance from the optical axis of the camera module to the folding axis and the straight-line distance from the optical axis of the camera module to the tilt fulcrum of the entire machine can be pre-calibrated or detected according to a preset detection method.

[0091] Optionally, the viewing angle of the camera module may be a characteristic attribute of the camera module itself.

[0092] The image processing method provided in this embodiment is applied to a smart terminal, comprising: step S1, when a preset condition is met, correcting at least a part of the area of ​​the captured image according to the posture parameter of the terminal device. Through the technical solution of this embodiment, the optimization function of the captured image can be realized, so as to optimize the image effect of the captured image based on the posture parameter of the terminal device, thereby improving the user experience.

[0093] Based on the same inventive concept as the above embodiments, the above embodiments are exemplarily described below through a specific application scenario of a folding screen mobile phone: See also Figure 4 The foldable phone has a rear-mounted wide-angle camera module on the back, and the wide-angle camera module has a macro shooting mode (<10cm). In addition, the foldable phone can adjust the folding angle based on the hinge.

[0094] See also Figure 5 , the foldable screen mobile phone is placed on the desktop (support object) in a folded state, and the rear wide-angle camera module is used to shoot the desktop ant specimen (subject). When the motor position sensor code value of the wide-angle camera module is used to calculate the current focus distance, after confirming that it is within 10cm (the feedback lens is in the close-range focus position), the macro shooting mode is turned on, and the icon of the perspective distortion correction function (i.e., the shooting picture optimization function) in the operation interface of the foldable screen mobile phone lights up, and the user can click to use or turn it off; the correction process under the perspective distortion correction function is as follows: 1. After reading the angle of the foldable screen mobile phone in the folded state, the straight-line distance from the optical axis of the wide-angle camera module to the folding screen rotation axis and the viewing angle of the long side or short side of the wide-angle camera module are matched to calculate the distortion restoration ratio, which can characterize the viewing angle and the trapezoidal picture shape projected by the supporting surface; 2. The first rectangular picture originally shot is restored to a trapezoidal picture; 3. The trapezoidal picture is cropped to be corrected into a second rectangular picture; 4. The rectangular ratio and pixels of the second rectangular picture are adjusted to be consistent with the ratio and pixels of the first rectangular picture, and the correction is completed.

[0095] Alternatively, see Figure 5 After reading the angle of the folded folding screen mobile phone, the distortion restoration ratio is calculated by matching the straight-line distance from the optical axis of the wide-angle camera module to the folding screen rotation axis with the viewing angle in the long or short side direction of the wide-angle camera module. The process can be: 1. Determine the known parameters: folding angle θ, straight-line distance L1 from the optical axis of the wide-angle camera module to the folding screen rotation axis, and viewing angle α.

[0096] 2. Calculate other parameters based on known parameters: 2.1 According to the folding angle θ, determine the inclination angle ∠ii=180°-θ of the folding screen.

[0097] 2.2 Based on the viewing angle α, determine the internal supplementary angle ∠i=90°-α / 2.

[0098] 2.3 According to the straight-line distance L1 from the optical axis of the wide-angle camera module to the rotation axis of the folding screen, the inclination angle ∠ii, and the internal supplementary angle ∠i, calculate the vertical line distance L2 from the first intersection point of the viewing angle and the bearing surface to the folding screen: L2 / X=tan(i);(1) L2 / (L1-X)=tan(i); (2) Wherein, X represents the straight-line distance from the vertical point of the folding screen to the optical axis of the wide-angle camera module, which is a set unknown number, and L1-X represents the straight-line distance from the vertical point of the folding screen to the rotation axis of the folding screen; Combining equations (1) and (2), we can solve for L2.

[0099] 2.4 According to the straight-line distance L2 from the first intersection point of the viewing angle and the bearing surface to the folding screen and the inclination angle ∠ii, calculate the straight-line distance L5 from the first intersection point to the rotation axis: Based on L2 / L5=sin(ii), the conversion is L5=L2 / sin(ii).

[0100] 2.5 According to the inclination angle ∠ii, the internal supplementary angle ∠i, and the viewing angle α, calculate the first angle between the field of view and the bearing surface ∠iii=180°-∠i-∠ii-α.

[0101] 2.6 Based on the inclination angle ∠ii and the internal supplementary angle ∠i, calculate the solution angle of the first right-angled side of the right triangle ∠iv=90°-∠iii-∠ii.

[0102] 2.7 Based on the solved angle ∠iv and the straight-line distance L1 from the optical axis of the wide-angle camera module to the rotation axis of the folding screen, calculate the first right-angled side y of the right triangle: Based on y / L1=cos(iv), the conversion is y= L1*cos(iv).

[0103] 2.8 According to the straight-line distance L5 from the first intersection point to the rotation axis, the first right-angled side y of the right triangle, and the first included angle ∠iii between the field of view and the bearing surface, calculate the length L4 of the bearing surface covered by the viewing angle (i.e., the length from the second intersection point of the viewing angle and the bearing surface to the first intersection point): Based on y / (L4+L5) =sin(iii), the conversion is L4 =y / sin(iii)-L5.

[0104] 2.9 Based on the length L4 of the bearing surface covered by the viewing angle, the straight-line distance L5 from the first intersection point to the rotation axis, and the vertical line distance L2 from the first intersection point of the viewing angle and the bearing surface to the folding screen, calculate the vertical line length L3 of the second viewing angle parallel line corresponding to the first intersection point to the second intersection point: Based on b / a=L4 / L5=L3 / L2, the conversion is L3=L4*L2 / L5.

[0105] 2.10 According to the length L4 of the bearing surface covered by the viewing angle, the straight-line distance L5 from the first intersection point to the rotation axis, the vertical line distance L2 from the first intersection point of the viewing angle and the bearing surface to the folding screen, and the vertical line length L3 of the second viewing angle parallel line corresponding to the first intersection point to the second intersection point, calculate the length b of the second viewing angle parallel line corresponding to the second intersection point and the length a of the first viewing angle parallel line corresponding to the first intersection point: b / a=L4 / L5; (3) b / a=L3 / L2; (4) Formulas (3) and (4) are combined to obtain the length b of the second perspective parallel line corresponding to the second intersection point and the length a of the first perspective parallel line corresponding to the first intersection point.

[0106] 2.11 According to the length b of the second viewing angle parallel line corresponding to the second intersection point, the length a of the first viewing angle parallel line corresponding to the first intersection point, and the ratio L*a of the originally shot first rectangular image, the distortion restoration ratio is calculated as L / a*b. Alternatively, see Figure 6 In this example, based on the distortion restoration ratio L / a*b and the length L4 of the bearing surface covered by the viewing angle, the first rectangular image originally photographed can be restored to a trapezoidal image; the trapezoidal image can be cropped to be corrected to a second rectangular image; and the rectangular ratio and pixels of the second rectangular image can be adjusted to be consistent with the ratio and pixels of the first rectangular image. The process can be: 1. Based on the distortion restoration ratio L / a*b and the length L4 of the bearing surface covered by the viewing angle, the proportion L*a of the original first rectangular image is restored to a trapezoidal image with a short base of L, a long base of L / a*b, and a height of L4.

[0107] 2. Based on the short bottom side, cut the trapezoidal image to correct it into a second rectangular image. The ratio of the second rectangular image is L*L4.

[0108] 3. Scale the second rectangular image from the proportion L*L4 to the proportion L*a.

[0109] In this way, in this example, when the wide-angle camera module turns on the macro shooting mode, the perspective distortion correction function can be turned on, so that a more realistic macro image of the subject can be captured, thereby improving the picture effect.

[0110] An embodiment of the present application also provides an intelligent terminal, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the steps of the image processing method in any of the above embodiments are implemented.

[0111] An embodiment of the present application further provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the image processing method in any of the above embodiments are implemented.

[0112] In the embodiments of the smart terminal and storage medium provided in the present application, all technical features of any of the above-mentioned image processing method embodiments may be included, and the expanded and explained contents of the specification are basically the same as those of the embodiments of the above-mentioned methods, and will not be repeated here.

[0113] The embodiment of the present application further provides a computer program product, which includes a computer program code. When the computer program code runs on a computer, the computer executes the methods in the above various possible implementation modes.

[0114] An embodiment of the present application also provides a chip, including a memory and a processor, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that a device equipped with the chip executes the methods in various possible implementation modes as described above.

[0115] It is understood that the above scenarios are only examples and do not constitute a limitation on the application scenarios of the technical solutions provided in the embodiments of the present application. The technical solutions of the present application can also be applied to other scenarios. For example, it is known to those skilled in the art that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0116] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0117] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0118] The units in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0119] In the present application, the same or similar terminology concepts, technical solutions and / or application scenario descriptions are generally described in detail only the first time they appear. When they appear again later, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of the present application, for the same or similar terminology concepts, technical solutions and / or application scenario descriptions that are not described in detail later, reference can be made to the previous related detailed descriptions.

[0120] In the present application, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0121] The various technical features of the technical solution of the present application can be arbitrarily combined. In order to make the description concise, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present application.

[0122] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, a controlled terminal, or a network device, etc.) to execute the method of each embodiment of the present application.

[0123] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a storage medium, or transmitted from one storage medium to another storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The storage medium can be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a storage disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state storage disk Solid State Disk (SSD)), etc.

[0124] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image processing method, characterized in that: include: When the preset conditions are met, at least a partial area of ​​the captured image is corrected according to the posture parameters of the terminal device.

2. The image processing method according to claim 1, characterized in that: The preset conditions are met, including at least one of the following: The current shooting mode is macro mode; The currently activated camera module is the wide-angle module; Identifying a distorted area in the captured image; A first operation is received.

3. The image processing method according to claim 2, characterized in that: The at least partial area includes at least one of the following: The distorted area of ​​the captured image; The document area of ​​the shooting picture; The image area of ​​the shooting picture; The entire area of ​​the captured image.

4. The image processing method according to any one of claims 1 to 3, characterized in that: The posture parameter includes at least one of the following: The folding angle of the folding screen; The entire device's tilt angle.

5. The image processing method according to claim 4, characterized in that: The correcting at least a part of the captured image according to the posture parameter of the terminal device includes: Calculate the distortion restoration ratio according to the posture parameters of the terminal device; Expanding target geometric parameters of at least a portion of the captured image according to the distortion restoration ratio to obtain an intermediate image; A resizing operation is performed on the intermediate image.

6. The image processing method according to claim 5, characterized in that: The calculating of the distortion restoration ratio according to the posture parameter of the terminal device includes any of the following: Calculating the distortion restoration ratio based on the folding angle, the straight-line distance from the optical axis of the camera module to the folding axis, and the viewing angle of the camera module; The distortion restoration ratio is calculated based on the tilt angle of the whole machine, the straight-line distance from the optical axis of the camera module to the tilt fulcrum of the whole machine, and the viewing angle of the camera module.

7. The image processing method according to claim 5, characterized in that: The target geometric parameter includes at least one of the length of the bottom edge of at least a portion of the captured image and the distance between the two bottom edges.

8. The image processing method according to claim 5, characterized in that: The resizing operation includes at least one of the following: Based on a first preset parameter, cropping the intermediate image; The cropped image is scaled based on the second preset parameter.

9. An intelligent terminal, characterized in that: include: A memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the image processing method according to any one of claims 1 to 8 are implemented.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 8 are implemented.

Images