Image processing method, electronic equipment and readable storage medium

Abstract

The invention discloses an image processing method, electronic equipment and a readable storage medium, and belongs to the technical field of terminals. The method is applied to electronic equipment, and comprises the following steps: in a process of acquiring an image through a camera, performing celestial body detection on a first preview image acquired by the camera; under the condition that the celestial body exists in the first preview image, determining first coordinate information of the celestial body identified in the first preview image; under the condition that the first coordinate information and the second coordinate information are the same, the first preview image is updated to be a second preview image, and the second coordinate information is coordinate information, determined based on multiple sensors in the electronic equipment, of a celestial body in the first preview image. According to the method, the recognized celestial body position in the preview image is compared with the theoretical display position of the celestial body in the preview image in actual life, so that whether the celestial body exists in the preview image or not can be further determined, and the accuracy of image processing is improved.

Description

Image processing methods, electronic devices, and readable storage media

[0001] This application claims priority to Chinese patent application filed on August 9, 2024, with application number 202411101245.3 and entitled "Image Processing Method, Electronic Device and Readable Storage Medium", the entire contents of which are incorporated herein by reference.

[0002] This application relates to the field of terminal technology, and in particular to an image processing method, an electronic device, and a readable storage medium.

[0003] With the development of terminal technology, the camera functions of electronic devices are becoming increasingly sophisticated. For example, electronic devices can include multiple camera modes, such as moon-viewing mode, portrait mode, and sunrise / sunset mode. In moon-viewing mode or sunrise / sunset mode, electronic devices can capture clear images of celestial bodies through their cameras.

[0004] However, when electronic devices detect the presence of celestial bodies in a captured image, they may perform image sharpness processing. However, electronic devices may also misdetect celestial bodies, leading to errors in image processing.

[0005] This application provides an image processing method, an electronic device, and a readable storage medium, which can reduce the possibility of image processing errors caused by celestial body identification errors in related technologies. The technical solution is as follows:

[0006] Firstly, an image processing method is provided for use in electronic devices, the method comprising:

[0007] During the process of acquiring images through a camera, celestial body detection is performed on the first preview image acquired by the camera; if a celestial body is present in the first preview image, the first coordinate information of the celestial body identified in the first preview image is determined; if the first coordinate information and the second coordinate information are the same, the first preview image is updated to a second preview image, the image quality of the second preview image is better than that of the first preview image, and the second coordinate information is the coordinate information of the celestial body in the first preview image determined by multiple sensors in the electronic device.

[0008] Thus, by comparing the celestial positions identified in the preview image with the theoretical positions in the preview image in real life, it is possible to further determine whether there are celestial bodies in the preview image, thereby improving the accuracy of image processing.

[0009] As an example of this application, if the first coordinate information and the second coordinate information are the same, the electronic device can also determine the second coordinate information before updating the first preview image to the second preview image.

[0010] That is, during the process of acquiring images through the camera, sensor data collected by each of the multiple sensors in the electronic device is obtained; based on the sensor data collected by each sensor, the azimuth and elevation angle differences between the camera and the celestial body are determined; and based on the azimuth and elevation angle differences and the field of view angle of the camera, the second coordinate information is determined.

[0011] In this way, the second coordinate information is determined by sensor data collected from multiple sensors, making the scheme more systematic and improving the accuracy of determining the coordinates of celestial bodies in the preview image.

[0012] In some embodiments, the electronic device can determine the second coordinate information upon recognizing the presence of a celestial body in the first preview image, or it can determine the second coordinate information during image acquisition after the camera is activated. Of course, in some embodiments, to reduce the power consumption of the electronic device, it can determine the second coordinate information upon recognizing the presence of a celestial body in the first preview image.

[0013] As an example of this application, the multiple sensors include a magnetometer, a positioning device, and a gravity sensor.

[0014] The operation of the electronic device to determine the azimuth and elevation differences between the camera and the celestial body based on the sensor data collected by each sensor includes:

[0015] Based on the first magnetic data collected by the magnetic sensor and the gravity data collected by the gravity sensor, the azimuth and elevation angles of the camera are determined, resulting in the first azimuth and the first elevation angle. The coordinates of the celestial body are determined, resulting in the third coordinate, and the coordinates of the electronic device are determined, resulting in the fourth coordinate. Based on the third and fourth coordinates, the elevation and azimuth angles of the celestial body relative to the electronic device are determined, resulting in the second azimuth and the second elevation angle. The azimuth angle difference between the first and second azimuth angles is determined, resulting in the azimuth angle difference between the camera and the celestial body. The elevation angle difference between the first and second elevation angles is also determined, resulting in the elevation angle difference between the camera and the celestial body.

[0016] Thus, by determining the elevation and azimuth differences between the camera and the celestial body, the position of the celestial body can be accurately determined.

[0017] As an example of this application, the operation of an electronic device determining the azimuth and elevation angles of a camera based on first magnetic data collected by a magnetic sensor and gravity data detected by a gravity sensor, and obtaining the first azimuth and elevation angles respectively, includes:

[0018] Based on the first magnetic force data and gravity data, a rotation matrix is ​​determined, which is used to switch the coordinate system; the first vector of the camera in the electronic device coordinate system is obtained; based on the rotation matrix and the first vector, the first elevation angle and the first azimuth angle are determined.

[0019] Thus, by determining the rotation matrix using the first magnetic force data and gravity data, the rotation matrix of the object is made more accurate.

[0020] As an example of this application, the operation of the electronic device to determine the first elevation angle and the first azimuth angle based on the rotation matrix and the first vector includes:

[0021] Multiply the rotation matrix by the first vector to obtain the second vector of the camera in the Earth coordinate system; determine the first elevation angle based on the Z-axis component of the second vector in the Earth coordinate system; determine the first azimuth angle based on the X-axis and Y-axis components of the second vector in the Earth coordinate system.

[0022] Thus, by using a rotation matrix, coordinate system transformation can be performed quickly and accurately, thereby enabling the rapid determination of the first and second azimuth angles.

[0023] As an example of this application, the operation of the electronic device to determine the second coordinate information based on the azimuth difference, elevation difference, and the field of view angle of the camera includes:

[0024] The azimuth and elevation differences are converted to the Cartesian coordinate system where the first preview image is located; based on the field of view angle, the angular coordinates in the Cartesian coordinate system are converted to proportional coordinates to obtain the second coordinate information.

[0025] In this way, by using the camera's field of view angle, the azimuth and elevation differences between the camera and the celestial body can be converted into the first preview image, thereby accurately determining the theoretical position of the celestial body in the preview image if the camera were to take a picture of it.

[0026] As an example of this application, the electronic device includes a magnetometer and a gyroscope sensor;

[0027] The electronic device determines the azimuth and elevation angles of the camera based on the first magnetic data collected by the magnetic sensor and the gravity data collected by the gravity sensor. Before obtaining the first azimuth and elevation angles, frame stabilization can also be performed.

[0028] In other words, the electronic device continuously captures multiple frames of preview images through a camera; during the capture of each frame of preview image, the gyroscope sensor collects gyroscope data corresponding to each frame of preview image, and the magnetometer collects second magnetic data corresponding to each frame of preview image; based on the gyroscope data corresponding to each frame of preview image, the magnetic weight queue is updated, and the magnetic weight queue includes the weights corresponding to the second magnetic data corresponding to each frame of preview image; based on the second magnetic data corresponding to each frame of preview image and the weights corresponding to each frame of preview image in the magnetic weight queue, the average magnetic data is determined, and the first magnetic data is obtained.

[0029] Thus, since the magnetic data collected by the magnetic sensor may fluctuate significantly, in order to obtain accurate magnetic data, the electronic device performs frame stabilization operation using gyroscope data collected by the gyroscope sensor, thereby reducing the impact of electronic device jitter on the collected magnetic data.

[0030] As an example of this application, the operation of updating the magnetic weight team of the electronic device based on the gyroscope data corresponding to each frame of preview image includes:

[0031] If the acquired preview image is the first frame preview image, the weight corresponding to the first frame preview image is determined to be 1; if the acquired preview image is not the first frame preview image, the weight corresponding to the currently acquired preview image is determined based on the gyroscope data corresponding to the currently acquired preview image and the weight corresponding to the previously acquired preview image; the weight corresponding to the currently acquired preview image and the magnetic data are stored in the magnetic weight queue.

[0032] As an example of this application, the electronic device can also adjust the size of the prediction box.

[0033] That is, the electronic device acquires the second magnetic data collected by the magnetic sensor; obtains the current location of the electronic device through the positioning device; determines the geomagnetic magnitude of the current location of the electronic device through the world magnetic model based on the current location of the electronic device, and obtains the third magnetic data; determines the deviation value between the second magnetic data and the third magnetic data; and determines the display size of the prediction box based on the deviation value.

[0034] It should be noted that the display position of the prediction box is the position of the celestial body in the preview image.

[0035] Therefore, since magnetic sensors are highly susceptible to interference when acquiring magnetic data, the confidence level of the acquired magnetic data can be determined by comparing it with theoretical magnetic data. Simultaneously, the size of the prediction frame is adjusted based on the deviation value of the magnetic data, thereby improving display richness and reliability.

[0036] Secondly, an image processing apparatus is provided, which has the function of implementing the image processing method described in the first aspect. The image processing apparatus includes at least one module for implementing the image processing method provided in the first aspect. This apparatus is used in an electronic device and includes:

[0037] The detection module performs celestial object detection on the first preview image captured by the camera during the image acquisition process.

[0038] The first determining module is used to determine the first coordinate information of the celestial body identified in the first preview image when there is a celestial body in the first preview image;

[0039] The first update module is used to update the first preview image to a second preview image when the first coordinate information and the second coordinate information are the same. The image quality of the second preview image is better than that of the first preview image. The second coordinate information is the coordinate information of the celestial body in the first preview image determined by multiple sensors in the electronic device.

[0040] As an example of this application, the device also includes:

[0041] The acquisition module is used to acquire sensor data collected by each of the multiple sensors in the electronic device during the process of acquiring images through the camera;

[0042] The second determining module is used to determine the azimuth and elevation differences between the camera and the celestial body based on the sensor data collected by each sensor.

[0043] The third determining module is used to determine the second coordinate information based on the azimuth difference, elevation difference, and the field of view angle of the camera.

[0044] As an example of this application, the multiple sensors include a magnetometer, a positioning device, and a gravity sensor.

[0045] The second determining module is used for:

[0046] Based on the first magnetic data collected by the magnetic sensor and the gravity data collected by the gravity sensor, the azimuth and elevation angles of the camera are determined, resulting in the first azimuth and the first elevation angle. The coordinates of the celestial body are determined, resulting in the third coordinate, and the coordinates of the electronic device are determined, resulting in the fourth coordinate. Based on the third and fourth coordinates, the elevation and azimuth angles of the celestial body relative to the electronic device are determined, resulting in the second azimuth and the second elevation angle. The azimuth angle difference between the first and second azimuth angles is determined, resulting in the azimuth angle difference between the camera and the celestial body. The elevation angle difference between the first and second elevation angles is also determined, resulting in the elevation angle difference between the camera and the celestial body.

[0047] As an example of this application, the second determining module is used for:

[0048] Based on the first magnetic force data and gravity data, a rotation matrix is ​​determined, which is used to switch the coordinate system; the first vector of the camera in the electronic device coordinate system is obtained; based on the rotation matrix and the first vector, the first elevation angle and the first azimuth angle are determined.

[0049] As an example of this application, the second determining module is used for:

[0050] Multiply the rotation matrix by the first vector to obtain the second vector of the camera in the Earth coordinate system; determine the first elevation angle based on the Z-axis component of the second vector in the Earth coordinate system; determine the first azimuth angle based on the X-axis and Y-axis components of the second vector in the Earth coordinate system.

[0051] As an example of this application, the third determining module is used for:

[0052] The azimuth and elevation differences are converted to the Cartesian coordinate system where the first preview image is located; based on the field of view angle, the angular coordinates in the Cartesian coordinate system are converted to proportional coordinates to obtain the second coordinate information.

[0053] As an example of this application, the electronic device includes a magnetometer and a gyroscope sensor;

[0054] The device also includes:

[0055] The first acquisition module is used to continuously acquire multiple frames of preview images via a camera;

[0056] The second acquisition module is used to acquire gyroscope data corresponding to each frame of preview image through gyroscope sensor and second magnetic data corresponding to each frame of preview image through magnetic sensor during the acquisition of each frame of preview image.

[0057] The second update module is used to update the magnetic weight queue based on the gyroscope data corresponding to each frame of the preview image. The magnetic weight queue includes the weights corresponding to the second magnetic data for each frame of the preview image.

[0058] The fourth determining module is used to determine the average magnetic data and obtain the first magnetic data based on the second magnetic data corresponding to each frame of the preview image and the weight corresponding to each frame of the preview image in the magnetic weight queue.

[0059] As an example of this application, the second update module is used for:

[0060] If the acquired preview image is the first frame preview image, the weight corresponding to the first frame preview image is determined to be 1; if the acquired preview image is not the first frame preview image, the weight corresponding to the currently acquired preview image is determined based on the gyroscope data corresponding to the currently acquired preview image and the weight corresponding to the previously acquired preview image; the weight corresponding to the currently acquired preview image and the magnetic data are stored in the magnetic weight queue.

[0061] Thirdly, an electronic device is provided, comprising a processor and a memory. The memory stores a program that supports the electronic device in executing the image processing method provided in the first aspect, and stores data related to implementing the image processing method described in the first aspect. The processor is configured to execute the program stored in the memory. The electronic device may further include a communication bus for establishing a connection between the processor and the memory.

[0062] Fourthly, a computer-readable storage medium is provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform the image processing method described in the first aspect.

[0063] Fifthly, a computer program product containing instructions is provided, which, when run on a computer, causes the computer to perform the image processing method described in the first aspect.

[0064] The technical effects achieved by the second, third, fourth, and fifth aspects mentioned above are similar to those achieved by the corresponding technical means in the first aspect mentioned above, and will not be repeated here.

[0065] Figure 1 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0066] Figure 2 is a block diagram of a software system for an electronic device provided in an embodiment of this application;

[0067] Figure 3 is a schematic diagram of an application scenario provided by an embodiment of this application;

[0068] Figure 4 is a schematic diagram of another application scenario provided by an embodiment of this application;

[0069] Figure 5 is a schematic diagram of another application scenario provided by an embodiment of this application;

[0070] Figure 6 is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0071] Figure 7 is a schematic diagram of a coordinate system provided in an embodiment of this application;

[0072] Figure 8 is a schematic diagram of a celestial altitude angle provided in an embodiment of this application;

[0073] Figure 9 is a flowchart illustrating another image processing method provided in this application.

[0074] Figure 10 is a schematic flowchart of a frame stabilization operation method provided in an embodiment of this application;

[0075] Figure 11 is a schematic flowchart of a method for adjusting the size of a prediction box provided in an embodiment of this application.

[0076] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0077] It should be understood that "multiple" as mentioned in this application refers to two or more. In the description of this application, unless otherwise stated, " / " indicates "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist, for example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, to facilitate a clear description of the technical solutions of this application, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and that "first," "second," etc., do not necessarily imply differences.

[0078] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0079] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0080] With technological advancements, the use cases for camera functions on electronic devices such as smartphones are becoming increasingly diverse. For instance, users can use these devices to photograph celestial bodies like the moon and sun. Furthermore, to obtain richer celestial images, electronic devices offer various shooting modes. For example, to obtain clear images of the moon, electronic devices provide a moon-viewing mode (also known as moon mode, moon-gazing mode, moon-shooting mode, etc.). When photographing the sun, electronic devices offer a sunrise / sunset mode, which allows for atmospheric rendering and image quality enhancement.

[0081] However, when electronic devices detect the presence of celestial bodies in the captured image, they process the image for sharpness. However, electronic devices may misdetect celestial bodies, such as misidentifying streetlights or other light sources as the moon or the sun, which can lead to errors in image processing.

[0082] To improve the accuracy of image processing, this application provides an image processing method. In this method, an electronic device can perform celestial object detection on a first preview image captured by a camera during image acquisition. If celestial objects such as the moon or sun are present in the first preview image, the first coordinate information of the identified celestial object in the first preview image is determined. If the first coordinate information is the same as the second coordinate information of the actual celestial object in the first preview image, then the first preview image is updated to a second preview image, and the image quality of the second preview image is superior to that of the first preview image. Thus, by comparing the position of the celestial object identified in the preview image with the actual position of the celestial object in the preview image, the presence of a celestial object in the preview image can be further determined, improving the accuracy of image processing.

[0083] Before providing a detailed explanation of the image processing method provided in the embodiments of this application, the electronic equipment involved in the embodiments of this application will be described first.

[0084] As an example, this method can be applied to electronic devices with cameras. As an example and not a limitation, the electronic device can be, but is not limited to, tablet computers, desktop computers, laptop computers, handheld computers, laptops, in-vehicle devices, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), mobile phones, smartwatches, etc., and this application embodiment does not limit this.

[0085] Figure 1 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Referring to Figure 1, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0086] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0087] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0088] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0089] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0090] In some embodiments, the processor 110 may include one or more interfaces, such as an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0091] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0092] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0093] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0094] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is an integer greater than 1.

[0095] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0096] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's image sensor. The light signal is converted into an electrical signal, and the image sensor transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimizations on image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be integrated into the camera 193.

[0097] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is an integer greater than 1.

[0098] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP performs Fourier transforms on the frequency energy.

[0099] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0100] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0101] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions, such as saving music, video, and other files on the external memory card.

[0102] Internal memory 121 can be used to store computer-executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created by electronic device 100 during use (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0103] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D and application processor.

[0104] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the touch operation intensity based on pressure sensor 180A. Electronic device 100 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than the pressure threshold is applied to the SMS application icon, a command to view an SMS message is executed. When a touch operation with an intensity greater than or equal to the pressure threshold is applied to the SMS application icon, a command to create a new SMS message is executed.

[0105] The gyroscope sensor 180B can be used to determine the motion attitude of the electronic device 100. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the electronic device 100 about three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the shake of the electronic device 100, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the electronic device 100 by moving in the opposite direction, thus achieving image stabilization. The gyroscope sensor 180B can also be used in navigation and motion-sensing game scenarios.

[0106] The barometric pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates altitude using the air pressure value measured by the barometric pressure sensor 180C to assist in positioning and navigation.

[0107] The magnetic sensor 180D (or magnetic force sensor) includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip cover. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover based on the magnetic sensor 180D. Based on the detected opening and closing state of the cover or the flip cover, the electronic device 100 can set features such as automatic flip unlocking.

[0108] In some embodiments, the magnetic sensor can also detect magnetic data in the current environment.

[0109] The accelerometer 180E can detect the magnitude of acceleration of electronic device 100 in various directions (generally three axes). When electronic device 100 is stationary, it can detect the magnitude and direction of gravity. The accelerometer 180E can also be used to identify the attitude of electronic device 100, and can be applied to applications such as screen orientation switching and pedometers.

[0110] A distance sensor 180F is used to measure distance. Electronic device 100 can measure distance via infrared or laser. In some embodiments, during a shooting scenario, electronic device 100 can utilize the distance sensor 180F for distance measurement to achieve fast focusing.

[0111] The ambient light sensor 180L is used to sense the brightness of ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 194 based on the sensed ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also work with the proximity sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0112] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touch display." Touch sensor 180K detects touch operations applied to or near it. Touch sensor 180K can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0113] The software system of electronic device 100 will be described next.

[0114] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses a layered Android system as an example to illustrate the software system of electronic device 100.

[0115] Figure 2 is a block diagram of a software system of an electronic device 100 provided in an embodiment of this application. Referring to Figure 2, the layered architecture divides the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime, the system layer, and the kernel layer.

[0116] The application layer can include a series of application packages. As shown in Figure 2, application packages can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0117] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications within the application layer. The application framework layer includes predefined functions. As shown in Figure 2, the application framework layer may include a window manager, content providers, a view system, a phone manager, a resource manager, and a notification manager. The window manager manages window programs. It can obtain the screen size, determine the presence of a status bar, lock the screen, and capture the screen. The content provider stores and retrieves data, making this data accessible to applications. This data may include video, images, audio, incoming and outgoing phone calls, browsing history and bookmarks, and a phone book. The view system includes visual controls, such as controls for displaying text and controls for displaying images. The view system can be used to build the application's display interface, which can consist of one or more views, such as a view displaying SMS notification icons, a view displaying text, and a view displaying images. The phone manager provides communication functions for the electronic device 100, such as managing call status (including connection and disconnection). The resource manager provides various resources for the application, such as localized strings, icons, images, layout files, and video files. The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify of download completion or message alerts. The notification manager can also display notifications as icons or scrolling text in the system's top status bar, such as notifications from background applications. Furthermore, the notification manager can appear as dialog boxes on the screen, such as displaying text messages in the status bar, emitting sounds, vibrating electronic devices, or flashing indicator lights.

[0118] The Android Runtime consists of the core libraries and the virtual machine. The Android runtime is responsible for scheduling and managing the Android system. The core libraries consist of two parts: one part contains the functionalities that Java needs to call, and the other part is the core Android library itself. The application layer and application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0119] The system library can include multiple functional modules, such as a surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), and 2D graphics engines (e.g., SGL). The surface manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The media libraries support playback and recording of various common audio and video formats, as well as still image files. The media libraries support various audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The 3D graphics processing libraries are used for 3D graphics drawing, image rendering, compositing, and layer processing. The 2D graphics engine is the drawing engine for 2D graphics.

[0120] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0121] The following example, using a scene of capturing a photograph, illustrates the workflow of the software and hardware of the electronic device 100.

[0122] When touch sensor 180K receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, touch operation timestamp, etc.). The raw input event is stored in the kernel layer. The application framework layer retrieves the raw input event from the kernel layer and identifies the control corresponding to the raw input event. Taking a single-click operation as an example, where the corresponding control is the camera application icon, the camera application calls the interface of the application framework layer to launch the camera application, and then calls the kernel layer to launch the camera driver, capturing still images or videos through camera 193.

[0123] Next, taking a mobile phone as an example, we will explain the application scenarios involved in the embodiments of this application.

[0124] Please refer to Figure 3, which is a schematic diagram of an application scenario provided by an embodiment of this application. In one application scenario, a user may take pictures using the camera of an electronic device, such as when the electronic device is taking pictures of the moon. After activating the camera, the electronic device can display the shooting interface shown in Figure 3(a) on the screen, which displays a preview image A captured by the camera. During the shooting process, the user can adjust the zoom ratio of the current camera. For example, the user can click the zoom control to change the zoom ratio from 1× to 10×. In response to the zoom ratio adjustment operation, the electronic device captures images through the camera according to the adjusted zoom ratio, obtaining a preview image B as shown in Figure 3(b). In this case, the electronic device can also detect whether the preview image B contains celestial bodies such as the moon and the sun. If the preview image B contains the moon, the electronic device can determine the first coordinate information of the moon identified in the preview image B, and outline the moon's location in the preview image B using a predicted bounding box (which can be a square). It can also determine the second coordinate information of the moon in the preview image when the image is captured according to the current azimuth and elevation angles of the electronic device's camera. If the first and second coordinate information are the same, the electronic device can enter the moon-viewing shooting mode. For example, the electronic device can reduce the exposure of the preview image B to obtain the preview image C shown in Figure 3(c). If the user clicks the shooting control P1 in this case, the electronic device responds to the click operation on the shooting control P1 and adjusts the image exposure based on the preview image C. If the user needs to view the captured image, refer to Figure 3(d), where the user can click the image viewing control P2 in the shooting interface. The electronic device responds to the click operation on the image viewing control P2 and can display the first target image E shown in Figure 3(e).

[0125] In another possible scenario, in response to the zoom adjustment of camera S1, the electronic device captures an image using camera S1 at the adjusted zoom level, obtaining a preview image B as shown in Figure 4(a). With the first and second coordinate information remaining the same, the electronic device reduces the exposure of preview image B, obtaining a preview image C as shown in Figure 4(b). It then displays a preview image D captured by another camera S2 in preview image C in a picture-in-picture format. If the user clicks the shooting control P1 in this situation, the phone responds to the click operation by adjusting the image exposure based on preview images D and C. If the user needs to view the captured image, as shown in Figure 4(c), the user can click the image viewing control P2 in the shooting interface. The phone, in response to the click operation of the image viewing control P2, can display the first target image E as shown in Figure 4(d).

[0126] It should be noted that the camera S1 that captures the preview image C can be a telephoto camera of the electronic device, and the camera S2 that captures the preview image D can be the main camera of the electronic device.

[0127] In another possible scenario, if a user takes a picture of the sun, the electronic device, after activating the camera, can display the shooting interface shown in Figure 5(a) on the screen. This shooting interface displays a preview image F captured by the camera. During the user's shooting process, the electronic device can also detect whether the preview image F contains celestial bodies such as the sun and moon. If the preview image F contains the sun, the electronic device can determine the first coordinate information of the sun identified in the preview image F, and outline the sun's location in the preview image F with a prediction box (which can be a square). It can also determine the second coordinate information of the sun in the preview image when the image is captured according to the current azimuth and elevation angles of the electronic device's camera. If the first coordinate information and the second coordinate information are the same, the electronic device can enter the sunrise / sunset mode. For example, the electronic device can display the current mode "sunrise / sunset" indicator in the preview image F and optimize the preview image F, such as enhancing the image quality of the preview image F and adjusting the color temperature and hue of the preview image F, to obtain the preview image G shown in Figure 5(b). If the user clicks the shooting control P1 in this situation, the electronic device responds to the click operation on the shooting control P1 by performing image exposure based on the preview image G. If the user needs to view the captured image, as shown in Figure 5(c), the user can click the image viewing control P2 in the shooting interface. In response to the click operation on the image viewing control P2, the electronic device can display the second target image H as shown in Figure 5(d).

[0128] It should be noted that the embodiments of this application are only illustrated using the application scenarios shown in Figures 3 and 5 above as examples, and do not constitute a limitation on the embodiments of this application.

[0129] Based on the application scenarios provided in the above embodiments, the image processing method provided in this application will be described next. Please refer to FIG6, which is a schematic flowchart of an image processing method according to an example. As an example and not a limitation, the method is described with reference to an electronic device, and the method may include some or all of the following:

[0130] Step 601: Capture a first preview image using the camera of the electronic device.

[0131] It should be noted that the first preview image can be an image captured when the camera is first turned on, or an image captured by the camera before the zoom level exceeds a preset magnification threshold. For example, the first preview image can be preview image A shown in Figure 3(a) above. Alternatively, the first preview image can be preview image F shown in Figure 5(a).

[0132] Of course, during the image acquisition process of the electronic device's camera, the user may adjust the camera's zoom level to obtain the desired image. Therefore, the first preview image can also be a preview image acquired by the camera after the zoom level exceeds a preset threshold. For example, the first preview image can also be preview image B shown in Figure 3(b).

[0133] It should be noted that the preset multiplier threshold can be set in advance according to needs. For example, the preset multiplier threshold can be 10x (or written as 10X), 15x, 20x or 30x, etc.

[0134] Step 602: During the image acquisition process via the camera of the electronic device, perform celestial detection on the first preview image acquired by the camera.

[0135] In the process of image acquisition through the camera of an electronic device, in order to accurately process the acquired preview image, the electronic device can perform celestial detection on the first preview image, that is, detect whether the first preview image contains celestial bodies such as the sun or the moon.

[0136] In some embodiments, the electronic device may be equipped with a neural network model capable of object recognition. Thus, the electronic device can identify various objects in the first preview image using the neural network model, and detect the presence of celestial bodies such as the moon or sun by recognizing each object. Alternatively, the electronic device may have AI (artificial intelligence) recognition capabilities. In this case, the electronic device can use AI recognition to identify various objects in the first preview image to detect the presence of celestial bodies such as the moon or sun in the first preview image. This application does not impose specific limitations on this aspect.

[0137] Normally, electronic devices enter the moon-viewing shooting mode when the camera's zoom level is greater than or equal to a preset zoom level threshold and the moon is present in the preview image. Therefore, if the first preview image is a preview image captured when the camera's zoom level is greater than or equal to the preset zoom level threshold, it is highly likely that the user wants to enter the moon-viewing shooting mode. Thus, the electronic device can first detect whether the moon is present in the first preview image, and if the moon is not present, it can then detect whether the sun is present.

[0138] Similarly, if the zoom level of the camera is less than a preset zoom level threshold, the electronic device can first detect whether the sun exists in the first preview image, and if the sun does not exist, it can then detect whether the moon exists.

[0139] Step 603: If a celestial body is present in the first preview image, determine the first coordinate information of the celestial body identified in the first preview image.

[0140] In some embodiments, the electronic device can establish a Cartesian coordinate system in the first preview screen and determine the coordinates of the celestial body identified in the first preview image in the Cartesian coordinate system to obtain the first coordinate information.

[0141] Step 604: Acquire sensor data collected by each of the multiple sensors in the electronic device.

[0142] It should be noted that the electronic device can perform step 604 when it detects the presence of a celestial body in the first preview image, or it can perform step 604 during image acquisition after the camera is started. This application embodiment does not impose specific limitations on this. Of course, in some embodiments, to reduce the power consumption of the electronic device, it can acquire sensor data collected by each of the multiple sensors when it detects the presence of a celestial body in the first preview image. Furthermore, the accompanying drawings of this application embodiment illustrate step 604 as an example when a celestial body is present in the first preview image.

[0143] As an example, these multiple sensors include a magnetometer, a positioning device, a gravity sensor, and a gyroscope sensor. A magnetometer, also known as a magnetometer, can be used to detect magnetic data indicating the location of an electronic device. The positioning device is used to determine the location of the electronic device. The gravity sensor is used to detect the orientation of a camera within the electronic device. The gyroscope sensor is used to detect vibrations within the electronic device, and so on.

[0144] In some embodiments, after acquiring sensor data from each of the multiple sensors, the electronic device can determine the azimuth and elevation differences between the camera and the celestial body based on the sensor data acquired by each sensor. For example, the operation of determining the azimuth and elevation differences between the camera and the celestial body based on the sensor data acquired by each sensor can refer to steps 605-608 below.

[0145] It is worth noting that the second coordinate information is determined by collecting sensor data from multiple sensors, which makes the scheme more systematic and improves the accuracy of determining the coordinates of celestial bodies in the preview image.

[0146] Step 605: Based on the first magnetic data collected by the magnetic sensor and the gravity data collected by the gravity sensor, determine the azimuth and elevation angles of the camera, and obtain the first azimuth and the first elevation angle respectively.

[0147] In some embodiments, the operation of the electronic device determining the azimuth and elevation angles of the camera based on the first magnetic data collected by the magnetic sensor and the gravity data detected by the gravity sensor, and obtaining the first azimuth and elevation angles respectively, includes: determining a rotation matrix based on the first magnetic data and gravity data, the rotation matrix being used to switch coordinate systems; obtaining a first vector of the camera in the coordinate system of the electronic device; and determining the first elevation angle and the first azimuth angle based on the rotation matrix and the first vector.

[0148] It should be noted that the rotation matrix can transform the electronic device coordinate system to the world coordinate system (which can also be referred to as the Earth coordinate system in this embodiment). For example, the electronic device coordinate system is a coordinate system established with reference to the electronic device. The electronic device coordinate system can be shown in Figure 7(a), and the world coordinate system (Earth coordinate system) can be shown in Figure 7(b).

[0149] It is worth noting that determining the rotation matrix using the first magnetic force data and gravity data makes the determined rotation matrix more accurate.

[0150] As an example, the rotation matrix can be determined from gravity data collected by a gravity sensor and first magnetic data collected by a magnetometer. Exemplarily, an electronic device can determine the rotation matrix based on the gravity vector and the magnetic field vector. The gravity data can be the acceleration data of the electronic device in three-dimensional space, typically represented as a three-dimensional vector g = (gx, gy, gz), where g can also be called the gravity vector. The magnetic data can be the magnetic field data of the electronic device in three-dimensional space, typically represented as a three-dimensional vector m = (mx, my, mz), where m can also be called the magnetic field vector.

[0151] In some embodiments, the electronic device may determine the unit vector of the gravity vector by means of the following first formula (1):

[0152] It should be noted that in the first formula (1) above, g is the gravity vector, g norm It is the unit vector of the gravity vector.

[0153] In some embodiments, the electronic device may determine the magnetic field vector of the horizontal component by means of the following second formula (2):

[0154] It should be noted that in the second formula (2) above, h norm is the magnetic field vector of the horizontal component.

[0155] In some embodiments, the electronic device can multiply the horizontal component of the magnetic field vector by the unit vector of the gravity vector to obtain the eastward vector, that is, e = h. norm *g norm , e=(ex, ey, ez); The electronic device constructs a rotation matrix based on the eastward vector, the horizontal component of the magnetic field vector, and the unit vector of the gravity vector. This rotation matrix R can be expressed by the following third formula (3):

[0156] It should be noted that electronic devices can also determine the rotation matrix in other ways. For example, electronic devices can represent the rotation matrix using an identity matrix. This application does not impose specific limitations on this.

[0157] As an example, an electronic device can determine the first vector of the camera orientation in the electronic device's coordinate system, which is c = (cx, cy, cz). Typically, the first vector of the camera orientation can be represented as (0, 0, -1), which is the negative Z-axis direction of the electronic device.

[0158] In some embodiments, the operation of the electronic device to determine the first elevation angle and the first azimuth angle based on the rotation matrix and the first vector includes: multiplying the rotation matrix by the first vector to obtain a second vector of the camera in the Earth coordinate system; determining the first elevation angle based on the Z-axis component of the second vector in the Earth coordinate system; and determining the first azimuth angle based on the X-axis component and the Y-axis component of the second vector in the Earth coordinate system.

[0159] Since the rotation matrix is ​​used to transform the electronic device's coordinate system to the Earth's coordinate system, or vice versa, multiplying the first vector by the rotation matrix yields the camera's second vector in the Earth's coordinate system. That is, `cearth = R*c`, where `cearth` is the second vector.

[0160] It is worth noting that coordinate system transformation can be performed quickly and accurately using rotation matrices.

[0161] As an example, the operation of an electronic device to determine a first elevation angle based on the Z-axis component of a second vector in the Earth coordinate system includes: determining the arcsine function value of the Z-axis component of the second vector in the Earth coordinate system to obtain the first elevation angle, i.e., h1 = arcsin(cearthz). Alternatively, determining the arccosine function value of the Z-axis component of the second vector in the Earth coordinate system to obtain the first elevation angle, i.e., h1 = π / 2 - arccos(cearthz).

[0162] Of course, the electronic device may also determine the first elevation angle based on the Z-axis component of the second vector in the Earth coordinate system through other operations, which will not be described in detail in this embodiment.

[0163] As an example, the operation of an electronic device to determine a first azimuth angle based on the X-axis and Y-axis components of a second vector in the Earth coordinate system includes: determining the arctangent function values ​​of the X-axis and Y-axis components of the second vector in the Earth coordinate system to obtain the first azimuth angle, i.e., A = arctan2(cearthy, crearthz). Alternatively, determining the arctangent function values ​​of the X-axis and Y-axis components of the second vector in the Earth coordinate system to obtain the first azimuth angle.

[0164] It should be noted that the electronic device may also include other operations in determining the first azimuth angle based on the X-axis and Y-axis components of the second vector in the Earth coordinate system, which will not be described in detail in this embodiment.

[0165] In some embodiments, the electronic device can comprehensively determine the azimuth and elevation angles of the camera based on the first magnetic data collected by the magnetic sensor and the gravity data collected by the gravity sensor, thereby obtaining the first azimuth and the first elevation angle, respectively. Alternatively, the first azimuth and the first elevation angle can be determined in other ways. For example, the electronic device can determine the first azimuth angle based on the first magnetic data collected by the magnetic sensor and determine the first elevation angle based on the gravity data collected by the gravity sensor. That is, the electronic device can determine the azimuth angle of the camera based on the magnetic field direction of the first magnetic data collected by the magnetic sensor, and determine the elevation angle of the camera based on the gravity data.

[0166] Step 606: Determine the coordinates of the celestial body to obtain the third coordinate, and determine the coordinates of the electronic device to obtain the fourth coordinate.

[0167] In some embodiments, Kepler's first equation describes the shape of a planet's orbit, i.e., the planet's orbit around the Sun is an ellipse with the Sun at one focus of the ellipse. Kepler's second equation, also known as the Keplerian time equation, describes the second velocity of the planet's motion in its orbit, indicating that the area velocity swept by the planet as it moves along the elliptical path is constant. Therefore, an electronic device can determine the coordinates of a celestial body using Kepler's first and second equations based on its system time, obtaining a third coordinate (which can be called cosmic coordinates). Furthermore, based on the system time and the position information collected by the positioning device, the electronic device's coordinates can be determined, obtaining a fourth coordinate.

[0168] It should be noted that Kepler's first equation can be represented as the fourth equation (4) below, and Kepler's second equation can be represented as the fifth equation (5) below. The fourth equation (4) can be:

[0169] It should be noted that in the above fourth formula (4), r is the distance from the planet to the sun, a is the semi-major axis of the ellipse, e is the eccentricity of the ellipse, and θ is the position angle of the planet on its orbit.

[0170] As an example, the fifth formula (5) can be: M = E - esinE (5)

[0171] It should be noted that in the fifth formula (5) above, M is the mean anomaly and E is the eccentric anomaly.

[0172] Step 607: Based on the third and fourth coordinates, determine the altitude and azimuth angles of the celestial body relative to the electronic device, and obtain the second azimuth angle and the second altitude angle.

[0173] In some embodiments, the third coordinates determined by the electronic device can be the longitude φ and latitude λ of the electronic device's location. After step 606, the electronic device can determine the fourth coordinates of the celestial body (also called equatorial coordinates), namely the right ascension α and declination δ of the celestial body, and determine the current system time as t1 (usually expressed in Greenwich Mean Time). In this case, the electronic device can determine the sidereal time of its current location, i.e., LST = GST + λ. Wherein, LST is the sidereal time of the electronic device's current location, and GST is Greenwich Mean Time, which can be determined from the system time t1. Then, the electronic device can determine the hour angle based on the sidereal time of its current location and the right ascension α of the celestial body. The hour angle is the difference between the right ascension of the celestial body and the sidereal time of the electronic device's location, i.e., hour angle H = LST - α. The electronic device can determine the second azimuth angle and the second altitude angle based on the hour angle, the electronic device's location, and the celestial body's location.

[0174] To facilitate understanding of the process of determining the second elevation angle, please refer to Figure 8, which provides a schematic diagram of elevation angle calculation in this application embodiment.

[0175] In some embodiments, the electronic device determines the second altitude angle based on the hour angle, the location of the electronic device, and the celestial body using the sixth formula (6) below. The electronic device determines the second azimuth angle based on the hour angle, the location of the electronic device, and the celestial body using the seventh formula (7) below.

[0176] As an example, the sixth formula (6) can be: sin(h2)=sin(δ)sin(φ)+cos(δ)cos(φ)cos(H) (6)

[0177] It should be noted that in the sixth formula (6) above, h2 is the second altitude angle.

[0178] As an example, the seventh formula (7) can be:

[0179] It should be noted that in the seventh formula (7) above, A2 is the second altitude angle.

[0180] Step 608: Determine the azimuth difference between the first azimuth and the second azimuth to obtain the azimuth difference between the camera and the celestial body, and determine the altitude difference between the first altitude and the second altitude to obtain the altitude difference between the camera and the celestial body.

[0181] As an example, an electronic device can subtract the first azimuth angle from the second azimuth angle to obtain the azimuth difference, and subtract the first altitude angle from the second altitude angle to obtain the altitude difference.

[0182] Step 609: Determine the second coordinate information based on the azimuth difference, elevation difference, and the camera's field of view.

[0183] In some embodiments, the electronic device can convert the azimuth difference and elevation difference to the Cartesian coordinate system where the first preview image is located; and convert the angular coordinates in the Cartesian coordinate system into proportional coordinates according to the field of view angle to obtain the second coordinate information.

[0184] For example, referring to Figure 9, when the electronic device is in landscape mode, it can represent the azimuth difference (e.g., 45 degrees) and elevation difference (e.g., 30 degrees) in a Cartesian coordinate system centered on the screen of the electronic device. Then, the angular coordinates (also called angular units) can be converted to proportional coordinates (also called proportional units), that is, the elevation difference of 30 degrees is converted to 0.5, and the azimuth difference of 45 degrees is converted to 0.5.

[0185] In some embodiments, the field of view of the camera can be determined based on the zoom ratio of the camera. When the zoom ratio is different, the second coordinate information determined by the electronic device is also different.

[0186] Step 610: Determine whether the first coordinate information and the second coordinate information are the same. If they are the same, then perform the operation in step 611 below. If not, image acquisition can continue, and the image will not be processed.

[0187] Because false detections may occur during the detection of celestial bodies in the first preview image—for example, streetlights or other lighting objects in the first preview image might be detected as the moon or the sun—the electronic device can compare the first coordinate information and the second coordinate information to determine if they are the same. If the first coordinate information and the second coordinate information are different, it is likely that a false detection has occurred in the first preview image, meaning that there is no celestial body in the first preview image. Therefore, image acquisition can continue without processing the preview image. If the first coordinate information and the second coordinate information are the same, it indicates that there is indeed a celestial body in the first preview image. Therefore, the electronic device can perform the following step 611.

[0188] Step 611: If the first coordinate information and the second coordinate information are the same, the electronic device can update the first preview image according to the type of the identified celestial body to obtain the second preview image.

[0189] It should be noted that the image quality of the second preview image is better than that of the first preview image.

[0190] In some embodiments, if the celestial body in the first preview image is the moon, the system enters a moon-viewing mode. In moon-viewing mode, the electronic device can improve the clarity of the moon in the first preview image. If the celestial body in the first preview image is the sun, the system enters a sunrise / sunset mode. In sunrise / sunset mode, the electronic device performs operations such as image quality enhancement on the first preview image.

[0191] In some embodiments, if the celestial body in the first preview image is the moon, and the zoom ratio of the camera is greater than or equal to a preset zoom ratio threshold, the electronic device can enter the moon-viewing shooting mode; if the zoom ratio of the camera is less than the preset zoom ratio threshold, the electronic device continues to display the first preview image without processing it. Of course, the electronic device can also enter the moon-viewing shooting mode when the zoom ratio of the camera is not greater than the preset zoom ratio threshold, and this application embodiment does not impose specific limitations on this.

[0192] It should be noted that when an electronic device enters moon-viewing mode or sunrise / sunset mode, a prompt message can be displayed on the shooting interface to indicate the current shooting mode. This prompt message can be displayed in at least one of the following forms: text, image, or control.

[0193] As an example, if the celestial body in the first preview image is the moon, the electronic device can acquire the super-resolution model corresponding to the moon's current position. The first preview image is then processed using the super-resolution model to obtain the second preview image.

[0194] It should be noted that super-resolution models can improve image resolution and restore image details and textures; therefore, electronic devices can acquire super-resolution models. Of course, to improve image processing efficiency and the quality of the processed image, different super-resolution models can be used for different lunar phase images.

[0195] It should be noted that after processing the first preview image using a super-resolution model, the resolution of the second preview image is greater than that of the first preview image.

[0196] In some embodiments, the super-resolution model can process the entire first preview image to improve its overall resolution. Alternatively, the super-resolution model can process the region of the moon within the first preview image to improve the resolution of that region. It should be noted that the moon region refers to the area in the first preview image where the moon is displayed.

[0197] For example, when the first lunar phase is a new moon, the electronic device can acquire a super-resolution model corresponding to the new moon image. The super-resolution model of the new moon image can be used to improve the resolution of the region where the moon is located in the new moon image, that is, to improve the resolution of the region where the moon is located in the first preview image. When the first lunar phase is a crescent moon, the electronic device can acquire a super-resolution model corresponding to the crescent moon image.

[0198] It is worth noting that by processing the first preview image using a super-resolution model corresponding to the current lunar phase, the image resolution is improved in a more targeted manner, ensuring not only the clarity of the moon in the image but also the clarity of other objects in the image.

[0199] As an example, if the celestial body in the first preview image is the moon, the electronic device can also update the first preview image to the second preview image in other ways. For example, the electronic device can reduce the exposure and / or exposure time of the first preview image to obtain the second preview image.

[0200] Because the moon was too bright when the electronic device's camera captured the first preview image, the moon area appeared blurry in the first preview image. Normally, images with lower exposure and / or shorter exposure times are clearer when displaying areas in strong light. Therefore, to obtain a clear image of the moon, the electronic device can obtain a second preview image by reducing the exposure and / or exposure time of the first preview image.

[0201] Because the moon's light is strong in the first preview image while the light on other objects is weak, reducing the exposure and / or exposure time of the first preview image results in a very clear moon in the second preview image, but other objects may not be imaged or may be blurry. Therefore, the electronic device can further reduce the exposure and / or exposure time of the moon region in the first preview image to ensure the clarity of the second preview image.

[0202] In some embodiments, when displaying a second preview image, if the electronic device receives a shooting operation, the electronic device responds to the shooting operation by exposing the second preview image to obtain a target image, which is an image of the second preview image.

[0203] If the celestial body in the currently displayed preview image is the moon, and the user is satisfied with the captured image, the user can trigger a shooting operation. This shooting operation can be performed by the user clicking the shooting control in photo mode, or by the user clicking the recording control in video mode, etc. Upon receiving the shooting operation, the electronic device can expose the second preview image to obtain the target image.

[0204] In some embodiments, in response to a shooting operation, the electronic device can acquire a second preview image in raw format (raw format is a photo format or an image format), convert the second preview image in raw format to an image in RGB format (RGB format is a photo format or an image format), detect the position of the moon in the second preview image, and perform dynamic range correction (DRC) processing on the second preview image to obtain the target image.

[0205] In some embodiments, if the celestial body in the first preview image is the sun, the electronic device can enhance the first preview image and adjust the color temperature and hue of the preview image to obtain a second preview image. For example, the color temperature and hue can be adjusted towards a warmer tone.

[0206] In some embodiments, when the first coordinate information and the second coordinate information are the same, the electronic device can also display a shooting mode recommendation interface based on the type of celestial body in the first preview image. This shooting mode recommendation interface is used to prompt the user whether to enter the shooting mode recommended by the electronic device. For example, if the celestial body in the first preview image is the moon, the user is prompted whether to enter the moon-viewing shooting mode; or, if the celestial body in the first preview image is the sun, the user is prompted whether to enter the sunrise / sunset mode. Upon receiving a touch operation to enter the recommended shooting mode, the electronic device enters the recommended shooting mode.

[0207] It should be noted that, to facilitate understanding of the operations in steps 604-611 above, this application embodiment provides a schematic diagram of a celestial body detection algorithm, as shown in Figure 9. Simply put, a magnetic sensor can be used to determine the first azimuth angle of the camera, a gravity sensor can be used to determine the first elevation angle of the camera, system time is used to determine the third coordinate, and the positioning device and system time can be used to determine the fourth coordinate. The third and fourth coordinates can determine the elevation and azimuth angles of the celestial body. The azimuth angle difference is determined based on the first azimuth angle and the azimuth angle of the celestial body; the elevation angle difference is determined based on the first elevation angle and the elevation angle of the celestial body; the field of view angle of the camera is determined based on the zoom ratio of the electronic device's camera; the second coordinate information of the celestial body in the shooting interface can be determined based on the field of view angle, the elevation angle difference, and the azimuth angle difference; the AI ​​model can identify the first coordinate information of the celestial body identified in the first preview image; it is determined whether the first coordinate information and the second coordinate information are consistent; if they are inconsistent, the algorithm ends; if they are consistent, at least one operation is performed, such as mode recommendation, image quality enhancement, or super-resolution fusion.

[0208] In this embodiment of the application, by comparing the celestial positions identified in the preview image with the actual display positions of celestial bodies in the preview image, it is possible to further determine whether there are celestial bodies in the preview image, thereby improving the accuracy of image processing.

[0209] It should be noted that during the shooting process using an electronic device, the camera may shake. To achieve image stabilization, the electronic device can perform frame stabilization based on magnetic data. Referring to Figure 10, this process may include the following steps:

[0210] Step 1001: Continuously capture multiple frames of preview images using a camera.

[0211] It should be noted that the process of a camera capturing images is dynamic, meaning that the camera can continuously capture multiple frames of preview images.

[0212] Step 1002: During the acquisition of each frame of preview image, the gyroscope data corresponding to each frame of preview image is acquired through the gyroscope sensor, and the second magnetic data corresponding to each frame of preview image is acquired through the magnetic sensor.

[0213] In other words, for each frame of preview image captured, the electronic device will collect gyroscope data once through the gyroscope sensor and magnetic data once through the magnetometer to obtain the second magnetic data.

[0214] Step 1003: For each frame of preview image acquired by the electronic device, the magnetic weight queue is updated based on the gyroscope data corresponding to each frame of preview image.

[0215] It should be noted that the magnetic weight queue includes the weights corresponding to the magnetic data for each frame of the preview image.

[0216] In some embodiments, the operation of updating the magnetic weight queue based on the gyroscope data corresponding to each frame of preview image by the electronic device includes: if the acquired preview image is the first frame of preview image, determining that the weight corresponding to the first frame of preview image is 1; if the acquired preview image is not the first frame of preview image, determining the weight corresponding to the currently acquired preview image based on the gyroscope data corresponding to the currently acquired preview image and the weight corresponding to the previously acquired preview image; and storing the weight corresponding to the currently acquired preview image and the magnetic data corresponding to the magnetic weight queue.

[0217] In some embodiments, when the acquired preview image is not the first frame preview image, the electronic device can determine the weight corresponding to the currently acquired preview image based on the gyroscope data corresponding to the currently acquired preview image and the weight corresponding to the previously acquired preview image, using the following eighth formula (8). This eighth formula can be:

[0218] It should be noted that in the eighth formula (8) above, W i (t) represents the weight of the preview image acquired at the current time t, W i (t-1) represents the weight corresponding to the previous frame preview image, g(t) represents the gyroscope data size at the current time t, and τ is a hyperparameter used to adjust the influence of gyroscope data on weight updates. When g(t) is very small, such as close to 0, it indicates that the electronic device is not shaking much. A value close to 1 means that the weight W i (t) does not decay much.

[0219] In some embodiments, the electronic device may also determine the weight of the currently acquired preview image based on the gyroscope data corresponding to the currently acquired preview image and the weight corresponding to the previously acquired preview image in other ways. For example, if the gyroscope data corresponding to the currently acquired preview image is the same as the gyroscope data corresponding to the previously acquired preview image, the weight corresponding to the previously acquired preview image is determined as the weight corresponding to the currently acquired preview image. If the gyroscope data corresponding to the currently acquired preview image is different from the gyroscope data corresponding to the previously acquired preview image, the weight corresponding to the currently acquired preview image is determined by the eighth formula (8) described above. Of course, it can also be determined in other ways, and this application embodiment does not impose specific limitations on this.

[0220] In some embodiments, after updating the magnetic weight queue, the electronic device can determine the average magnetic data based on the second magnetic data corresponding to each frame of the preview image and the weight corresponding to each frame of the preview image in the magnetic weight queue, thereby obtaining the first magnetic data. This operation may include the operations described in steps 1004-100X below.

[0221] Step 1004: Determine whether the queue length in the magnetic weight queue is greater than or equal to the preset length. If yes, proceed to step 1005 below. If no, proceed to step 1006 below.

[0222] It should be noted that the preset length can be set in advance according to requirements. For example, the preset length can be 10, 20, etc. This application embodiment does not limit this.

[0223] Step 1005: Delete the second magnetic data and its corresponding weight that were first added to the magnetic weight queue, so that the length of the magnetic weight queue is less than the preset length, and then perform the operation in step 1006 below.

[0224] Step 1006: Determine the average magnetic data in the magnetic data in the magnetic weight queue to obtain the first magnetic data.

[0225] In some embodiments, the electronic device can determine the average magnetic force data in the magnetic force weight queue using the following ninth formula (9) to obtain the first magnetic force data. The ninth formula (9) can be:

[0226] It should be noted that in the above ninth formula (9), N is the number of magnetic data in the magnetic weight queue, and M... j W represents the magnetic data corresponding to the i-th frame preview image. i (t) represents the weight corresponding to the i-th frame preview image.

[0227] In some embodiments, the operation of determining the average magnetic data and obtaining the first magnetic data based on the second magnetic data corresponding to each frame of preview image and the weights corresponding to each frame of preview image in the magnetic weight queue can include not only the operations described in steps 1004-1006, but also other operations. For example, the electronic device can randomly select M sets of magnetic data and corresponding weights from the magnetic weight queue, or select M sets of magnetic data and corresponding weights added later to the magnetic weight queue, and then determine the average magnetic data of the M sets of magnetic data according to the ninth formula described above to obtain the first magnetic data. Here, M is an integer greater than 1 and less than or equal to a preset length.

[0228] In this embodiment, the magnetic data collected by the magnetic sensor may fluctuate significantly. To obtain accurate magnetic data, the electronic device performs frame stabilization operation on the gyroscope data collected by the gyroscope sensor, thereby reducing the impact of electronic device jitter on the collected magnetic data.

[0229] In some embodiments, the electronic device can not only perform frame stabilization based on magnetic force data, but also determine whether the magnetic force data sensed by the magnetic sensor is accurate. Referring to Figure 11, this process may include the following operations:

[0230] Step 1101: Acquire the second magnetic data collected by the magnetic sensor.

[0231] Step 1102: Obtain the current location of the electronic device through the positioning device.

[0232] Step 1103: Based on the current location of the electronic device, determine the geomagnetic magnitude of the current geographical location of the electronic device using the world magnetic model to obtain the third magnetic force data.

[0233] It should be noted that the world magnetic model is used to determine the magnitude of the geomagnetic field in different geographical locations, and the world magnetic model can refer to relevant technologies. The embodiments of this application do not impose specific limitations.

[0234] Step 1104: Determine the deviation between the second and third magnetic force data.

[0235] Step 1105: Determine the display size of the prediction box based on the deviation value.

[0236] It should be noted that the display position of the prediction box is the position of the celestial body in the preview image.

[0237] In some embodiments, if the deviation value is less than or equal to the deviation threshold, the size of the prediction box is determined to be the default size, i.e., the size of the prediction box is not adjusted. If the deviation value is greater than the deviation threshold, the size of the prediction box is adjusted, and the larger the deviation value, the larger the prediction box. For example, the prediction box is adjusted to 1.1 times, 1.3 times, etc., the default size.

[0238] In this embodiment, since the magnetic sensor is easily affected by interference when collecting magnetic data, the confidence level of the magnetic data collected by the magnetic sensor can be determined by comparing the collected magnetic data with theoretical magnetic data. Simultaneously, the size of the prediction box is adjusted based on the deviation value of the magnetic data, thereby improving the richness and reliability of the display.

[0239] This application provides an image processing apparatus, which can be implemented as part or all of an electronic device by software, hardware, or a combination of both. The electronic device can be the one shown in FIG1. ​​The apparatus includes: a detection module, a determination module, and an update module.

[0240] The detection module performs celestial object detection on the first preview image captured by the camera during the image acquisition process.

[0241] The first determining module is used to determine the first coordinate information of the celestial body identified in the first preview image when there is a celestial body in the first preview image;

[0242] The first update module is used to update the first preview image to a second preview image when the first coordinate information and the second coordinate information are the same. The image quality of the second preview image is better than that of the first preview image. The second coordinate information is the coordinate information of the celestial body in the first preview image determined by multiple sensors in the electronic device.

[0243] As an example of this application, the device also includes:

[0244] The acquisition module is used to acquire sensor data collected by each of the multiple sensors in the electronic device during the process of acquiring images through the camera;

[0245] The second determining module is used to determine the azimuth and elevation differences between the camera and the celestial body based on the sensor data collected by each sensor.

[0246] The third determining module is used to determine the second coordinate information based on the azimuth difference, elevation difference, and the field of view angle of the camera.

[0247] As an example of this application, the multiple sensors include a magnetometer, a positioning device, and a gravity sensor.

[0248] The second determining module is used for:

[0249] Based on the first magnetic data collected by the magnetic sensor and the gravity data collected by the gravity sensor, the azimuth and elevation angles of the camera are determined, resulting in the first azimuth and the first elevation angle. The coordinates of the celestial body are determined, resulting in the third coordinate, and the coordinates of the electronic device are determined, resulting in the fourth coordinate. Based on the third and fourth coordinates, the elevation and azimuth angles of the celestial body relative to the electronic device are determined, resulting in the second azimuth and the second elevation angle. The azimuth angle difference between the first and second azimuth angles is determined, resulting in the azimuth angle difference between the camera and the celestial body. The elevation angle difference between the first and second elevation angles is also determined, resulting in the elevation angle difference between the camera and the celestial body.

[0250] As an example of this application, the second determining module is used for:

[0251] Based on the first magnetic force data and gravity data, a rotation matrix is ​​determined, which is used to switch the coordinate system; the first vector of the camera in the electronic device coordinate system is obtained; based on the rotation matrix and the first vector, the first elevation angle and the first azimuth angle are determined.

[0252] As an example of this application, the second determining module is used for:

[0253] Multiply the rotation matrix by the first vector to obtain the second vector of the camera in the Earth coordinate system; determine the first elevation angle based on the Z-axis component of the second vector in the Earth coordinate system; determine the first azimuth angle based on the X-axis and Y-axis components of the second vector in the Earth coordinate system.

[0254] As an example of this application, the third determining module is used for:

[0255] The azimuth and elevation differences are converted to the Cartesian coordinate system where the first preview image is located; based on the field of view angle, the angular coordinates in the Cartesian coordinate system are converted to proportional coordinates to obtain the second coordinate information.

[0256] As an example of this application, the electronic device includes a magnetometer and a gyroscope sensor;

[0257] The device also includes:

[0258] The first acquisition module is used to continuously acquire multiple frames of preview images via a camera;

[0259] The second acquisition module is used to acquire gyroscope data corresponding to each frame of preview image through gyroscope sensor and second magnetic data corresponding to each frame of preview image through magnetic sensor during the acquisition of each frame of preview image.

[0260] The second update module is used to update the magnetic weight queue based on the gyroscope data corresponding to each frame of the preview image. The magnetic weight queue includes the weights corresponding to the second magnetic data for each frame of the preview image.

[0261] The fourth determining module is used to determine the average magnetic data and obtain the first magnetic data based on the second magnetic data corresponding to each frame of the preview image and the weight corresponding to each frame of the preview image in the magnetic weight queue.

[0262] As an example of this application, the second update module is used for:

[0263] If the acquired preview image is the first frame preview image, the weight corresponding to the first frame preview image is determined to be 1; if the acquired preview image is not the first frame preview image, the weight corresponding to the currently acquired preview image is determined based on the gyroscope data corresponding to the currently acquired preview image and the weight corresponding to the previously acquired preview image; the weight corresponding to the currently acquired preview image and the magnetic data are stored in the magnetic weight queue.

[0264] In this embodiment of the application, by comparing the celestial positions identified in the preview image with the actual display positions of celestial bodies in the preview image, it is possible to further determine whether there are celestial bodies in the preview image, thereby improving the accuracy of image processing.

[0265] It should be noted that the image processing device provided in the above embodiments is only illustrated by the division of the above functional modules when performing image processing. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0266] The functional units and modules in the above embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of the embodiments of this application.

[0267] The image processing apparatus and image processing method embodiments provided in the above embodiments belong to the same concept. The specific working process and technical effects of the units and modules in the above embodiments can be found in the method embodiment section, and will not be repeated here.

[0268] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line, DSL) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer, or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0269] The above-described embodiments are optional embodiments provided by this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the technical scope disclosed in this application should be included within the protection scope of this application.

Claims

An image processing method, characterized in that, The method, applied in electronic devices, includes: performing celestial object detection on a first preview image captured by a camera during image acquisition; determining first coordinate information of the celestial object identified in the first preview image if a celestial object is present in the first preview image; updating the first preview image to a second preview image if the first coordinate information and second coordinate information are the same, wherein the image quality of the second preview image is better than that of the first preview image, and the second coordinate information is the coordinate information of the celestial object in the first preview image determined by multiple sensors in the electronic device. The method as described in claim 1, characterized in that, Before updating the first preview image to the second preview image when the first coordinate information and the second coordinate information are the same, the method further includes: acquiring sensor data collected by each of the multiple sensors in the electronic device during the process of acquiring images through the camera; determining the azimuth difference and elevation difference between the camera and the celestial body based on the sensor data collected by each sensor; and determining the second coordinate information based on the azimuth difference, the elevation difference, and the field of view angle of the camera. The method as described in claim 2, characterized in that, The plurality of sensors include a magnetometer, a positioning device, and a gravity sensor. The step of determining the azimuth and elevation differences between the camera and the celestial body based on sensor data collected by each sensor includes: determining the azimuth and elevation angles of the camera based on first magnetic data collected by the magnetometer and gravity data collected by the gravity sensor, obtaining a first azimuth angle and a first elevation angle respectively; determining the coordinates of the celestial body to obtain a third coordinate, and determining the coordinates of the electronic device to obtain a fourth coordinate; determining the elevation and azimuth angles of the celestial body relative to the electronic device based on the third and fourth coordinates, obtaining a second azimuth angle and a second elevation angle; determining the azimuth difference between the first and second azimuth angles to obtain the azimuth difference between the camera and the celestial body, and determining the elevation difference between the first and second elevation angles to obtain the elevation difference between the camera and the celestial body. The method as described in claim 3, characterized in that, The step of determining the azimuth and elevation angles of the camera based on the first magnetic data collected by the magnetic sensor and the gravity data collected by the gravity sensor, and obtaining the first azimuth and elevation angles respectively, includes: determining a rotation matrix based on the first magnetic data and the gravity data, the rotation matrix being used to switch coordinate systems; obtaining a first vector of the camera in the electronic device coordinate system; and determining the first elevation angle and the first azimuth angle based on the rotation matrix and the first vector. The method as described in claim 4, characterized in that, The step of determining the first elevation angle and the first azimuth angle based on the rotation matrix and the first vector includes: multiplying the rotation matrix by the first vector to obtain a second vector of the camera in the Earth coordinate system; determining the first elevation angle based on the Z-axis component of the second vector in the Earth coordinate system; and determining the first azimuth angle based on the X-axis and Y-axis components of the second vector in the Earth coordinate system. The method as described in claim 2, characterized in that, Determining the second coordinate information based on the azimuth difference, the elevation difference, and the field of view angle of the camera includes: converting the azimuth difference and the elevation difference to the Cartesian coordinate system where the first preview image is located; and converting the angular coordinates in the Cartesian coordinate system to proportional coordinates based on the field of view angle to obtain the second coordinate information. The method as described in claim 3, characterized in that, The electronic device includes a magnetometer and a gyroscope sensor. Before determining the azimuth and elevation angles of the camera based on the first magnetic data collected by the magnetometer and the gravity data collected by the gravity sensor, and obtaining the first azimuth and elevation angles respectively, the method further includes: continuously acquiring multiple frames of preview images through the camera; during the acquisition of each frame of preview images, acquiring gyroscope data corresponding to each frame of preview images through the gyroscope sensor, and acquiring second magnetic data corresponding to each frame of preview images through the magnetometer; updating the magnetic weight queue based on the gyroscope data corresponding to each frame of preview images, the magnetic weight queue including the weights corresponding to the second magnetic data corresponding to each frame of preview images; determining the average magnetic data based on the second magnetic data corresponding to each frame of preview images and the weights corresponding to each frame of preview images in the magnetic weight queue, and obtaining the first magnetic data. The method as described in claim 7, characterized in that, The step of updating the magnetic weight queue based on the gyroscope data corresponding to each preview image includes: when the acquired preview image is the first preview image, determining the weight corresponding to the first preview image to be 1; when the acquired preview image is not the first preview image, determining the weight corresponding to the currently acquired preview image based on the gyroscope data corresponding to the currently acquired preview image and the weight corresponding to the previously acquired preview image; and storing the weight corresponding to the currently acquired preview image and the magnetic data corresponding to the magnetic weight queue. An electronic device, characterized in that, The electronic device includes a processor and a memory; the memory is used to store programs that support the electronic device in performing the method as described in any one of claims 1-8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-8.

Images