Photographing method, electronic device, and storage medium

Abstract

The application provides a photographing method, an electronic device and a storage medium. The method comprises the following steps: in response to an operation of a user opening a camera, a preview picture is acquired; it is detected whether a display picture of a display screen is contained in the preview picture, the display picture of the display screen being a picture displayed by a photographed display screen; if it is detected that the display picture of the display screen is contained in the preview picture, it is detected whether banding and / or moire is present in a display screen area in the preview picture, the display screen area being an area in which the display picture of the display screen is displayed. The method provided by the application is helpful to improve the accuracy of banding and moire detection.

Description

Technical Field

[0001] This application relates to the field of image capture, and more particularly to a capture method, electronic device, and storage medium. Background Technology

[0002] In daily life, people often use the camera function of electronic devices such as mobile phones or tablets to take pictures. When the picture includes a screen display, if the picture is taken using the normal exposure method of current technology, banding and moiré patterns may occur. Banding can manifest as a certain number of bright and dark stripes or bright and dark rolling stripes, while moiré patterns can manifest as various colored stripes and uncertain optical textures.

[0003] It is understandable that banding and moiré patterns can negatively impact image quality. Therefore, the detection and removal of banding and moiré patterns have become important research topics. Inaccurate banding and moiré detection will inevitably affect their removal, thus impacting the final image quality. For example, inaccurate banding detection may cause banding to appear in areas where it was not originally present due to incorrect exposure settings; conversely, inaccurate moiré detection, such as misclassifying non-moiré textures as moiré, will lead to overprocessing and reduced image sharpness. Therefore, a new imaging method is urgently needed to improve the accuracy of banding and moiré detection. Summary of the Invention

[0004] This application provides an imaging method, electronic device, and storage medium that help improve the accuracy of banding and moiré detection.

[0005] In a first aspect, embodiments of this application provide a shooting method, the method comprising: in response to a user opening a camera, acquiring a preview image; detecting whether the preview image contains a display screen, wherein the display screen is the image displayed on the screen being photographed; if the preview image contains the display screen, detecting whether banding and / or moiré patterns exist in the display screen area of ​​the preview image, wherein the display screen area is the area displaying the display screen.

[0006] In this embodiment, when a user takes a picture using a camera in an electronic device, it detects whether the preview screen contains the screen being photographed. If the preview screen contains the screen being photographed, the detection area for banding and / or moiré patterns can be located in the area where the screen being photographed is located in the preview screen. This eliminates the need for global detection of the entire preview screen as described in the prior art, and only requires local detection of the area where the screen being photographed is located in the preview screen, thereby improving the accuracy of banding and moiré pattern detection.

[0007] In one possible implementation, if banding and / or moiré patterns are detected in the display area of ​​the preview image, the banding and / or moiré patterns are eliminated.

[0008] In this embodiment of the application, when banding and / or moiré patterns are detected in a local area (e.g., the area where the screen being photographed in the preview image is located), banding and / or moiré patterns can be eliminated only in that local area, without having to eliminate banding and / or moiré patterns in the entire preview image, thereby improving the quality and efficiency of elimination.

[0009] In one possible implementation, banding removal includes: determining a target exposure duration, the target exposure duration being M times the display screen flicker period, the display screen flicker period being the flicker period of the display screen being photographed; and taking a picture based on the target exposure duration to remove banding, where M is a positive integer greater than or equal to 1.

[0010] In this embodiment, by adjusting the exposure time to match the flicker period of the screen being photographed, the elimination quality can be improved.

[0011] In one possible implementation, banding removal includes: determining a first exposure time and a second exposure time, wherein the first exposure time is determined by the display screen flicker cycle, the display screen flicker cycle being the flicker cycle of the display screen being photographed, and the second exposure time is determined by the overall brightness of the preview image; taking a picture using the first exposure time to obtain a first image, and taking a picture using the second exposure time to obtain a second image; covering the display screen area in the second image with the display screen area in the first image to obtain a third image, wherein the third image is the image after banding removal.

[0012] In this embodiment, two images are obtained by taking pictures with different exposure times. One image may have banding eliminated but may have overexposure, while the other image may not have overexposure but has banding in the area where the screen is located. By fusing the two images, both banding and overexposure can be overcome, thus improving image quality.

[0013] In one possible implementation, the elimination of the moiré pattern includes: determining a target focal length, which is a focal length that is out of focus; taking a picture based on the target focal length to eliminate the moiré pattern; or taking a picture using an ultra-wide-angle camera to eliminate the moiré pattern; or downsampling the captured image to eliminate the moiré pattern.

[0014] In this embodiment, low-pass filtering can be achieved by adjusting focus, changing the camera, or downsampling, thereby effectively eliminating moiré patterns.

[0015] In one possible implementation, detecting whether banding exists in the display area of ​​the preview screen includes: acquiring continuous multi-frame image data; detecting dark stripes in the display area of ​​each frame of the continuous multi-frame image data; and determining whether banding exists in the display area based on the frequency of the dark stripes appearing in the continuous multi-frame image data.

[0016] In this embodiment of the application, banding in the display screen area is detected by statistically analyzing the frequency of dark stripes appearing in multiple consecutive frames of images. This method can effectively detect banding.

[0017] In one possible implementation, if the preview image contains the display screen, detecting whether banding and / or moiré patterns exist in the display screen area of ​​the preview image includes:

[0018] If the preview image contains the display screen, the system automatically detects whether banding and / or moiré patterns exist in the display screen area of ​​the preview image.

[0019] In this embodiment, after detecting that the preview screen contains the display screen, the system automatically detects whether there is banding and / or moiré patterns in the display screen area, which can save user operations and improve user experience.

[0020] In one possible implementation, if the preview image contains the display screen, detecting whether banding and / or moiré patterns exist in the display screen area of ​​the preview image includes: if the preview image contains the display screen, prompting the user whether to perform banding and / or moiré pattern detection; and in response to the user's confirmation operation, detecting whether banding and / or moiré patterns exist in the display screen area of ​​the preview image.

[0021] In this embodiment of the application, determining whether to perform banding and / or moiré detection through interaction with the user can better match the user's personalized needs.

[0022] Secondly, embodiments of this application provide a shooting device, including one or more functional modules, which are used to implement the shooting method as described in the first aspect.

[0023] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory, wherein the memory is used to store a program; and the processor is used to run the program to implement the shooting method as described in the first aspect.

[0024] Fourthly, embodiments of this application provide a readable storage medium storing a program that, when run on an electronic device, causes the electronic device to implement the shooting method described in the first aspect.

[0025] Fifthly, embodiments of this application provide a program that, when run on a processor of an electronic device, causes the electronic device to perform the shooting method as described in the first aspect.

[0026] In one possible design, the program in the fifth aspect can be stored wholly or partially on a storage medium packaged with the processor, or it can be stored wholly or partially on a memory not packaged with the processor. Attached Figure Description

[0027] Figure 1 A schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application;

[0028] Figure 2 A flowchart illustrating an embodiment of the shooting method provided in this application;

[0029] Figure 3 This is a schematic diagram of the preview screen provided in the embodiments of this application;

[0030] Figure 4 A flowchart illustrating another embodiment of the shooting method provided in this application;

[0031] Figure 5 This is a schematic diagram illustrating the effect of image fusion provided in an embodiment of this application;

[0032] Figure 6 This is a schematic diagram of the imaging device provided in an embodiment of this application. Detailed Implementation

[0033] In this embodiment of the application, unless otherwise stated, the character " / " indicates that the preceding and following objects are in an OR relationship. For example, A / B can represent A or B. "AND / OR" describes the relationship between the associated objects, indicating that three relationships can exist. For example, A AND / OR B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0034] It should be noted that the terms "first" and "second" used in the embodiments of this application are used only for distinguishing descriptive purposes and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated, nor should they be construed as indicating or implying order.

[0035] In the embodiments of this application, "at least one" refers to one or more items, and "more than one" refers to two or more items. Furthermore, "at least one of the following" or similar expressions refer to any combination of these items, which may include any combination of a single item or a plurality of items. For example, at least one of A, B, or C can represent: A, B, C, A and B, A and C, B and C, or A, B, and C. Each of A, B, and C can be an element itself or a set containing one or more elements.

[0036] In this application, terms such as "exemplary," "in some embodiments," and "in another embodiment" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the term "exemplary" is intended to present the concept in a concrete manner.

[0037] In the embodiments of this application, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction, their meanings are consistent. Similarly, in the embodiments of this application, "communication" and "transmission" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction, their meanings are consistent. For example, transmission can include sending and / or receiving, and can be a noun or a verb.

[0038] In the embodiments of this application, the term "equal to" can be used in conjunction with "greater than" to apply to technical solutions employing the condition of "greater than", and can also be used in conjunction with "less than" to apply to technical solutions employing the condition of "less than". It should be noted that when "equal to" is used with "greater than", it cannot be used with "less than"; and when "equal to" is used with "less than", it cannot be used with "greater than".

[0039] In daily life, people often use the camera function of electronic devices such as mobile phones or tablets to take pictures. When the picture includes a screen display, if the picture is taken using the normal exposure method of current technology, banding and moiré patterns may occur. Banding can manifest as a certain number of bright and dark stripes or bright and dark rolling stripes, while moiré patterns can manifest as various colored stripes and uncertain optical textures.

[0040] It is understandable that banding and moiré patterns can negatively impact image quality. Therefore, the detection and removal of banding and moiré patterns have become important research topics. Inaccurate banding and moiré detection will inevitably affect their removal, thus impacting the final image quality. For example, inaccurate banding detection may cause banding to appear in areas where it was not originally present due to incorrect exposure settings; conversely, inaccurate moiré detection, such as misclassifying non-moiré textures as moiré, will lead to overprocessing and reduced image sharpness. Therefore, a new imaging method is urgently needed to improve the accuracy of banding and moiré detection.

[0041] To address the aforementioned issues, this application proposes a shooting method. The electronic device can also be a mobile device, which may also be referred to as User Equipment (UE), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile terminal, user terminal, terminal, wireless communication equipment, user agent, or user device. Mobile devices can be stations (STAION, ST) in a WLAN, cellular phones, cordless phones, Session Initiation Protocol (SIP) phones, Wireless Local Loop (WLL) stations, Personal Digital Assistant (PDA) devices, handheld devices with wireless communication capabilities, computing devices or other processing devices connected to a wireless modem, in-vehicle devices, vehicle-to-everything (V2X) terminals, computers, laptops, handheld communication devices, handheld computing devices, satellite wireless devices, wireless modem cards, set-top boxes (STBs), customer premises equipment (CPEs), and / or other devices used for communication over wireless systems, as well as next-generation communication systems, such as mobile devices in 5G networks or mobile devices in future evolved Public Land Mobile Network (PLMN) networks. These mobile devices can also be wearable devices. Wearable devices, also known as wearable smart devices, are a general term for devices that utilize wearable technology to intelligently design and develop everyday wearables, such as glasses and watches. Wearable devices are portable devices that are worn directly on the body or integrated into a user's clothing or accessories. Wearable devices are not just hardware devices, but also achieve powerful functions through software support, data interaction, and cloud interaction.

[0042] Figure 1 First, the hardware structure of the electronic device 100 is shown as an example.

[0043] Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0044] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0045] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0046] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0047] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0048] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0049] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge electronic device 100, and can also be used for data transfer between electronic device 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0050] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0051] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0052] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0053] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0054] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0055] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0056] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0057] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0058] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0059] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0060] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0061] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0062] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0063] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0064] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0065] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0066] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0067] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0068] Internal memory 121 can be used to store computer executable program code, which includes instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of electronic device 100 by running instructions stored in internal memory 121 and / or instructions stored in memory located in the processor.

[0069] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0070] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0071] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or make hands-free calls through the speaker 170A.

[0072] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a telephone call or voice message, the receiver 170B can be brought close to the ear to listen to the voice.

[0073] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Electronic device 100 may have at least one microphone 170C. In some embodiments, electronic device 100 may have two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic device 100 may also have three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0074] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0075] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0076] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations performed on different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0077] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0078] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with and separate from the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to realize functions such as calls and data communication. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0079] Next, combined Figure 2 The imaging method provided in the embodiments of this application will be described by way of example.

[0080] like Figure 2 The diagram shown is a flowchart of an embodiment of the shooting method provided in this application, which specifically includes the following steps:

[0081] Step 201: In response to the user opening the camera, a preview image is obtained.

[0082] Specifically, users can open the camera through the camera application of an electronic device, or users can open the camera by calling the camera application through a third-party application, or users can open the camera in other ways. This application embodiment does not make any special limitations on this.

[0083] In response to a user opening the camera, the electronic device can obtain a preview image through the camera.

[0084] Understandably, the preview screen contains the content the user wants to capture. For example, the user can take a picture of the content displayed on the screen, and when the camera is pointed at the screen, the preview screen will display the content displayed on the screen.

[0085] The content displayed on the screen may include, but is not limited to, text, images, and other content.

[0086] Step 202: Check whether the preview screen contains the display screen.

[0087] Specifically, after the electronic device acquires the preview image, it can detect whether the preview image contains the display screen's image, which is the image displayed on the captured display screen.

[0088] The display screen can be a display screen on a mobile terminal (e.g., a mobile phone, tablet, smartwatch, vehicle terminal, etc.) or a display screen on a fixed terminal (e.g., a television, desktop computer, large screen, etc.). This application embodiment does not impose any special limitation on the type of display screen.

[0089] If the display screen is on and showing relevant content, the display screen has a certain brightness, meaning that the brightness of the display screen is different from that of the surrounding environment.

[0090] It is understandable that the specific methods for detecting the presence of a display screen in the preview image can be found in the descriptions in the relevant technologies, and will not be repeated here.

[0091] If the preview screen contains a display screen, proceed to step 2031. Alternatively,

[0092] If no display screen is detected in the preview, proceed to step 2032.

[0093] Figure 3 An example image showing the preview screen is provided.

[0094] refer to Figure 3 Screen 300 is a preview image captured by the user through the camera of an electronic device. Screen 300 is the image displayed on the screen of the electronic device. In screen 300, area 301 displays the image displayed on the screen, which is the image displayed on the screen of another device being photographed (e.g., a television, tablet, large screen).

[0095] Step 2031: Detect whether there is banding and / or moiré patterns in the display area of ​​the preview screen.

[0096] Specifically, this step only detects banding and / or moiré patterns on the display screen area in the preview image, that is, it performs local detection in the preview image, which helps to improve the accuracy of banding and / or moiré pattern detection.

[0097] The display screen area is the area where the screen displays the image. For example, the display screen area can be like this: Figure 3 Region 301 in the middle.

[0098] In some optional embodiments, if the preview screen contains a display screen, the electronic device can automatically detect whether there is banding and / or moiré patterns in the display screen area of ​​the preview screen.

[0099] In some optional embodiments, if the preview screen contains the display screen, the electronic device may prompt the user whether to perform banding and / or moiré detection.

[0100] Understandably, if an electronic device detects banding and / or moiré patterns, it will attempt to eliminate them. Taking banding as an example, when an electronic device detects banding, it can eliminate it by adjusting the exposure time. However, adjusting the exposure time may result in overexposure in some areas, leading to a poor user experience.

[0101] For example, an electronic device may pop up a prompt box containing relevant text and controls. The text may be used to prompt the user that the device is capturing content on the display screen and whether local banding and / or moiré detection is required to achieve better shooting results. If the user clicks the "OK" control, the electronic device may perform local banding and / or moiré detection; or, if the user clicks the "Reject" control, the electronic device may not perform local banding and / or moiré detection. In this case, the electronic device may also choose not to perform global banding and / or moiré detection of the preview screen, or the electronic device may further choose to perform global banding and / or moiré detection of the preview screen. This application embodiment does not impose any special limitations on this.

[0102] In some alternative embodiments, the user may need to take multiple photos consecutively, for example, when photographing a slideshow. In this case, when the user takes the first photo, the electronic device can prompt the user whether local banding and / or moiré detection is required. If the user selects local banding and / or moiré detection when taking the first photo, the electronic device can automatically perform local banding and / or moiré detection when taking subsequent photos (e.g., the second, third, etc.) without prompting the user again.

[0103] Understandably, the shooting time interval between two adjacent photos (e.g., the second and the first, the third and the second, etc.) can be preset. If the shooting time interval between the current photo and the previous photo is within the preset time interval, the electronic device can automatically perform local banding and / or moiré detection on the current photo. The preset time interval can be an empirical value.

[0104] For example, taking the current photo as the second photo and the previous photo as the first photo, if the time interval between the second photo and the first photo is within a preset time interval, the electronic device can automatically perform local banding and / or moiré detection on the second photo without prompting the user whether local banding and / or moiré detection is needed when taking the second photo.

[0105] In some optional embodiments, when the electronic device has performed local detection of banding and / or moiré patterns and takes another photo within a preset time interval, the threshold for detecting whether the preview image contains the display screen's image can be lowered. It is understood that in scenarios where the user intends to take multiple photos, if the threshold for detecting whether the preview image contains the display screen's image remains unchanged after the user takes the first photo, the electronic device may fail to detect the display screen's image in the preview image due to user or environmental factors. This would result in not triggering banding and / or moiré pattern detection, thereby reducing image quality and the user experience. Therefore, after the user takes the first photo, the threshold for detecting whether the preview image contains the display screen's image can be lowered during subsequent photos to compensate for errors caused by user or environmental factors. This effectively triggers banding and / or moiré pattern detection, improving image quality and the user experience.

[0106] Next, the detection method of banding in the display area will be described by way of example using specific embodiments.

[0107] In some optional embodiments, banding detection may include: acquiring multiple consecutive frames of image data, and detecting and segmenting each frame of the acquired image data. Detection of each frame can be used to detect display areas within that frame. Segmentation of each frame can be used to extract display areas from each frame. Next, dark stripes can be detected within each frame to identify them, and the frequency of dark stripe occurrences can be calculated based on the temporal variation of each frame.

[0108] When comparing dark stripes between two adjacent frames of image data, the display screen areas of the two adjacent frames of image data can be registered so that the same or similar dark stripes can be found between the two adjacent frames of image data in the display screen area.

[0109] Understandably, if the frequency of dark stripes appearing is stable, it can be assumed that banding is more likely to occur in the display area.

[0110] For example, suppose N consecutive frames of image data are acquired, where N is an integer greater than 1, and the (m-1)th frame and the mth frame are two consecutive frames, and the mth frame and the (m+1)th frame are two consecutive frames, where m is an integer. <m<N。

[0111] It is understandable that the specific methods for detecting moiré patterns in the display screen area can be found in the descriptions in relevant technologies, and will not be repeated here.

[0112] Step 2032: Detect whether banding and / or moiré patterns exist in the preview screen.

[0113] It is understood that this step involves detecting banding and / or moiré patterns across the entire preview screen, i.e., performing a global detection within the preview screen. For details on the method of this global detection, please refer to the descriptions in relevant technologies, which will not be elaborated here.

[0114] Figure 4 This is a flowchart illustrating another embodiment of the shooting method provided in this application. After step 2031, the following steps may be included:

[0115] Step 2041: Eliminate banding and / or moiré patterns in the display area of ​​the preview screen.

[0116] In some optional embodiments, eliminating banding in the display area of ​​the preview screen may include: adjusting the exposure time and taking a picture based on the adjusted exposure time, wherein the adjusted exposure time can be M times the flicker period of the display screen in the preview screen, where M is a positive integer greater than or equal to 1, and the flicker period of the display screen is the flicker period of the display screen being photographed, that is, the flicker period of the display screen is the flicker period of the display screen corresponding to the display screen in the preview screen. The flicker period of the display screen can be calculated by the flicker frequency of the display screen, for example, the flicker period of the display screen = 1 / the flicker frequency of the display screen.

[0117] For example, assuming the screen flicker frequency in the preview image is 50Hz, then the screen flicker period is 0.02s, and the exposure time can be M times 0.02s, such as 0.02s, 0.04s, 0.06s, etc.

[0118] Understandably, although adjusting the exposure time may result in overexposure in some areas of the photo if the exposure time exceeds the maximum exposure time limit, due to the limitation of the maximum exposure time.

[0119] In some optional embodiments, eliminating banding in the display area of ​​the preview screen may include: determining a first exposure time based on the display screen flicker frequency, taking a picture at the first exposure time to obtain a first image; determining a second exposure time based on the overall brightness of the preview screen, taking a picture at the second exposure time to obtain a second image.

[0120] Understandably, the first image may experience localized overexposure due to the maximum exposure time limitation; however, the display screen area in the first image has banding eliminated. The second image may show banding in the display screen area, but it does not exhibit overexposure. In this case, the display screen area in the first image can be cropped, and this cropped area can replace the display screen area in the second image, resulting in a merged image. In this merged image, the display screen area has no banding, and the background area other than the display screen area is not overexposed.

[0121] Now combined Figure 5 An example illustration of image fusion is provided.

[0122] Image 500 is a first image obtained by taking a first exposure time. In image 500, region 501 is the display screen area. Because the first exposure time was used for exposure in region 501, there is no banding. Region 502 is the background area other than the display screen area. Because the first exposure time was used for exposure in region 502, local overexposure will occur, for example, region 503. It is understood that region 503 is only an illustrative representation of the location of the overexposure phenomenon and does not constitute a limitation on the embodiments of this application. In some embodiments, the overexposure phenomenon may occur in multiple locations or in other places.

[0123] Image 510 is a second image obtained by taking a photo with a second exposure time. In image 510, region 511 is the display screen area. Because the first exposure time was used to take the photo, banding will occur in region 511. Region 512 is the background area excluding the display screen area. Because the second exposure time was used to take the photo, overexposure will not occur in region 512.

[0124] Next, region 501 from image 500 is overlaid on region 511 from image 510, thereby obtaining a merged image 520. Image 520 may include regions 521 and 522, where region 521 displays the content of region 501, and region 522 displays the content of region 512. Since the content of region 501 has no banding and the content of region 512 has no overexposure, image 520 has neither banding nor overexposure.

[0125] In some optional embodiments, eliminating moiré patterns in the display area of ​​the preview image may include adjusting the focal length. Adjusting the focal length can be either increasing or decreasing the focal length. It is understood that by adjusting the focal length, high-frequency components in the image can be filtered out, thereby achieving a low-pass filtering effect and effectively reducing moiré patterns in the captured image.

[0126] In some optional embodiments, eliminating moiré patterns in the display area of ​​the preview image may include using a low-resolution camera for shooting. For example, assuming shooting at 1x magnification, the camera could be switched to an ultra-wide-angle camera. Because ultra-wide-angle cameras have a wide angle range, but their resolution is lower than other cameras, they can act as a low-pass filter, effectively reducing moiré patterns in the captured image.

[0127] In some optional embodiments, eliminating moiré patterns in the display area of ​​the preview image may include downsampling the image data. It is understood that during post-processing of image data, downsampling can achieve a low-pass filtering effect, thereby effectively reducing moiré patterns in the captured image.

[0128] In some alternative embodiments, after step 2032, the following steps may be included:

[0129] Step 2042: Remove banding and / or moiré patterns from the preview screen.

[0130] Understandably, the specific methods for eliminating banding and / or moiré patterns in the preview image can be found in the descriptions of relevant technologies, and will not be elaborated upon here.

[0131] Figure 6 This is a schematic diagram of the structure of one embodiment of the imaging device of this application, as shown below. Figure 6 As shown, the aforementioned imaging device 60 may include: an acquisition module 61 and a detection module 62; wherein,

[0132] Module 61 is used to acquire a preview image in response to the user's action of opening the camera;

[0133] The detection module 62 is used to detect whether the preview image contains the display screen, wherein the display screen is the image displayed on the captured display screen; if the preview image contains the display screen, it detects whether there is banding and / or moiré patterns in the display screen area of ​​the preview image, wherein the display screen area is the area where the display screen is displayed.

[0134] In one possible implementation, the shooting device 60 further includes:

[0135] The elimination module 63 is used to eliminate banding and / or moiré patterns if banding and / or moiré patterns are detected in the display area of ​​the preview screen.

[0136] In one possible implementation, the elimination module 63 is specifically used to determine the target exposure duration, which is M times the display screen flicker period, and the display screen flicker period is the flicker period of the display screen being photographed.

[0137] Shooting is performed based on the target exposure time to eliminate the banding, where M is a positive integer greater than or equal to 1.

[0138] In one possible implementation, the elimination module 63 is specifically used to determine a first exposure duration and a second exposure duration. The first exposure duration is determined by the display screen flicker cycle, which is the flicker cycle of the display screen being photographed. The second exposure duration is determined by the overall brightness of the preview screen.

[0139] A first image is obtained by taking a picture using the first exposure time, and a second image is obtained by taking a picture using the second exposure time;

[0140] The display area in the first image is overlaid on the display area in the second image to obtain a third image, which is the image after banding removal.

[0141] In one possible implementation, the elimination module 63 is specifically used to determine the target focal length, which is the out-of-focus focal length;

[0142] Shoot based on the target focal length to eliminate the moiré pattern;

[0143] or,

[0144] The moiré pattern was eliminated by using an ultra-wide-angle camera.

[0145] or,

[0146] The captured image is downsampled to eliminate the moiré pattern.

[0147] In one possible implementation, the detection module 62 is further configured to acquire continuous multi-frame image data;

[0148] Detect dark stripes in the display screen area of ​​each frame of image data in the continuous multi-frame image data;

[0149] The presence of banding in the display screen area is determined based on the frequency of the dark stripes appearing in the continuous multi-frame image data.

[0150] In one possible implementation, the detection module 62 is further configured to automatically detect whether banding and / or moiré patterns exist in the display area of ​​the preview screen if the preview screen contains the display screen.

[0151] In one possible implementation, the detection module 62 is further configured to prompt the user whether to perform banding and / or moiré detection if the preview image contains the display screen.

[0152] In response to the user's confirmation, the system detects whether banding and / or moiré patterns exist in the display area of ​​the preview screen.

[0153] Figure 6 The imaging device 60 provided in the illustrated embodiment can be used to execute the technical solution of the method embodiment shown in this application, and its implementation principle and technical effect can be further referred to the relevant description in the method embodiment.

[0154] The above should be understood Figure 6 The division of the various modules in the illustrated imaging device 60 is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented in software via processing element calls, while others are implemented in hardware. For example, the detection module can be a separate processing element or integrated into a chip in the electronic device. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. During implementation, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0155] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, these modules can be integrated together as a System-On-a-Chip (SOC).

[0156] In the above embodiments, the processor may include, for example, a CPU, DSP, microcontroller, or digital signal processor, and may also include a GPU, embedded neural network processing unit (NPU), and image signal processor (ISP). The processor may also include necessary hardware accelerators or logic processing hardware circuits, such as an ASIC, or one or more integrated circuits for controlling the execution of the program in this application. Furthermore, the processor may have the function of operating one or more software programs, which may be stored in a storage medium.

[0157] This application also provides a readable storage medium storing a program that, when run on an electronic device, causes the electronic device to execute the method provided in the embodiments shown in this application.

[0158] This application also provides a program product including a program that, when run on an electronic device, causes the electronic device to perform the method provided in the embodiments shown in this application.

[0159] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, A and B simultaneously, or B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0160] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented using electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0161] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0162] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a readable storage medium. Based on this understanding, the technical solution of this application, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0163] The above description is merely a specific embodiment of this application. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application. The protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A shooting method, characterized in that, The method includes: In response to the user opening the camera, a preview image is captured; Detect whether the preview screen contains the display screen's display, where the display screen's display is the image shown on the captured screen; If the preview image contains the display image of the display screen, detect whether there is banding and / or moiré patterns in the display screen area of ​​the preview image, where the display screen area is the area where the display image of the display screen is displayed.

2. The method according to claim 1, characterized in that, The method further includes: If banding and / or moiré patterns are detected in the display area of ​​the preview screen, the banding and / or moiré patterns are eliminated.

3. The method according to claim 2, characterized in that, Eliminating the banding includes: Determine the target exposure duration, which is M times the display screen flicker period, where the display screen flicker period is the flicker period of the display screen being photographed; Shooting is performed based on the target exposure time to eliminate the banding, where M is a positive integer greater than or equal to 1.

4. The method according to claim 2, characterized in that, Eliminating the banding includes: The first exposure time and the second exposure time are determined. The first exposure time is determined by the flicker period of the display screen, which is the flicker period of the display screen being photographed. The second exposure time is determined by the overall brightness of the preview screen. A first image is obtained by taking a picture using the first exposure time, and a second image is obtained by taking a picture using the second exposure time; The display area in the first image is overlaid on the display area in the second image to obtain a third image, which is the image after banding removal.

5. The method according to claim 2, characterized in that, The elimination of the moiré pattern includes: Determine the target focal length, which is the focal length that is out of focus; Shoot based on the target focal length to eliminate the moiré pattern; or, The moiré pattern was eliminated by using an ultra-wide-angle camera. or, The captured image is downsampled to eliminate the moiré pattern.

6. The method according to any one of claims 1-5, characterized in that, The detection of whether banding exists in the display area of ​​the preview screen includes: Acquire continuous multi-frame image data; Detect dark stripes in the display screen area of ​​each frame of image data in the continuous multi-frame image data; The presence of banding in the display screen area is determined based on the frequency of the dark stripes appearing in the continuous multi-frame image data.

7. The method according to claim 1, characterized in that, If the preview image contains the display screen, detecting whether banding and / or moiré patterns exist in the display screen area of ​​the preview image includes: If the preview image contains the display screen, the system automatically detects whether banding and / or moiré patterns exist in the display screen area of ​​the preview image.

8. The method according to claim 1, characterized in that, If the preview image contains the display screen, detecting whether banding and / or moiré patterns exist in the display screen area of ​​the preview image includes: If the preview screen contains the display screen, the user is prompted whether to perform banding and / or moiré pattern detection. In response to the user's confirmation, the system detects whether banding and / or moiré patterns exist in the display area of ​​the preview screen.

9. An electronic device, characterized in that, include: A processor and a memory, the memory being used to store a program; the processor being used to run the program to implement the shooting method as described in any one of claims 1-8.

10. A readable storage medium, characterized in that, The readable storage medium stores a program that, when run on an electronic device, implements the shooting method as described in any one of claims 1-8.

Images