Image processing method, electronic device, and readable storage medium

Abstract

The present application belongs to the technical field of terminals. Disclosed are an image processing method, an electronic device, and a readable storage medium. The method is applied to an electronic device, and comprises: in the process of a camera collecting an image, performing celestial-body detection on a first preview image collected by the camera; when there is a celestial body in the first preview image, determining first coordinate information of the celestial body identified in the first preview image; and when the first coordinate information is identical to second coordinate information, updating the first preview image to a second preview image, wherein the second coordinate information is coordinate information of the celestial body in the first preview image, which coordinate information is determined on the basis of a plurality of sensors in the electronic device. In the present application, by means of comparing the position of a celestial body identified in a preview image with the position at which the celestial body may theoretically be displayed in the preview image in real life, it is possible to further determine whether there is a celestial body in the preview image, thereby improving the accuracy of image processing.

Description

Image processing methods, electronic devices, and readable storage media

[0001] This application claims priority to Chinese patent application filed on August 9, 2024, with application number 202411101245.3 and entitled "Image Processing Method, Electronic Device and Readable Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of terminal technology, and in particular to an image processing method, an electronic device, and a readable storage medium. Background Technology

[0003] With the development of terminal technology, the camera functions of electronic devices are becoming increasingly sophisticated. For example, electronic devices can include multiple camera modes, such as moon-viewing mode, portrait mode, and sunrise / sunset mode. In moon-viewing mode or sunrise / sunset mode, electronic devices can capture clear images of celestial bodies through their cameras.

[0004] However, when electronic devices detect the presence of celestial bodies in a captured image, they may perform image sharpness processing. However, electronic devices may also misdetect celestial bodies, leading to errors in image processing. Summary of the Invention

[0005] This application provides an image processing method, an electronic device, and a readable storage medium, which can reduce the possibility of image processing errors caused by celestial body identification errors in related technologies. The technical solution is as follows:

[0006] Firstly, an image processing method is provided for use in electronic devices, the method comprising:

[0007] During the process of acquiring images through a camera, celestial body detection is performed on the first preview image acquired by the camera; if a celestial body is present in the first preview image, the first coordinate information of the celestial body identified in the first preview image is determined; if the first coordinate information and the second coordinate information are the same, the first preview image is updated to a second preview image, the image quality of the second preview image is better than that of the first preview image, and the second coordinate information is the coordinate information of the celestial body in the first preview image determined by multiple sensors in the electronic device.

[0008] Thus, by comparing the celestial positions identified in the preview image with the theoretical positions in the preview image in real life, it is possible to further determine whether there are celestial bodies in the preview image, thereby improving the accuracy of image processing.

[0009] As an example of the present application, in the case that the first coordinate information and the second coordinate information are the same, the electronic device can further determine the second coordinate information before updating the first preview image to the second preview image.

[0010] That is, in the process of collecting the image by the camera, the sensor data collected by each of the plurality of sensors in the electronic device is obtained; the azimuth angle difference and the elevation angle difference between the camera and the celestial body are determined according to the sensor data collected by each of the sensors; and the second coordinate information is determined according to the azimuth angle difference, the elevation angle difference, and the field of view angle of the camera.

[0011] In this way, the second coordinate information is determined by the sensor data collected by the plurality of sensors, so that the scheme is more systematic, and the accuracy of determining the coordinate of the celestial body in the preview image is improved.

[0012] In some embodiments, the electronic device can determine the second coordinate information in the case that the celestial body is identified in the first preview image, or can determine the second coordinate information in the process of collecting the image after the camera is started. Of course, in some embodiments, in order to reduce the power consumption of the electronic device, the electronic device can determine the second coordinate information in the case that the celestial body is identified in the first preview image.

[0013] As an example of the present application, the plurality of sensors include a magnetic sensor, a positioning device, and a gravity sensor.

[0014] The operation of determining, by the electronic device, the azimuth angle difference and the elevation angle difference between the camera and the celestial body according to the sensor data collected by each of the sensors includes:

[0015] According to the first magnetic force data collected by the magnetic sensor and the gravity data collected by the gravity sensor, the azimuth angle and the elevation angle of the camera are determined, and a first azimuth angle and a first elevation angle are obtained; the coordinate of the celestial body is determined, and a third coordinate is obtained; the coordinate of the electronic device is determined, and a fourth coordinate is obtained; the elevation angle and the azimuth angle of the celestial body relative to the electronic device are determined according to the third coordinate and the fourth coordinate, and a second azimuth angle and a second elevation angle are obtained; the azimuth angle difference between the first azimuth angle and the second azimuth angle is determined, and the azimuth angle difference between the camera and the celestial body is obtained; and the elevation angle difference between the first elevation angle and the second elevation angle is determined, and the elevation angle difference between the camera and the celestial body is obtained.

[0016] In this way, by respectively determining the elevation angle difference and the azimuth angle difference between the camera and the celestial body, the position of the celestial body can be accurately determined.

[0017] As an example of the present application, the operation of determining, by the electronic device, the azimuth angle and the elevation angle of the camera according to the first magnetic force data collected by the magnetic sensor and the gravity data detected by the gravity sensor, and obtaining a first azimuth angle and a first elevation angle, includes:

[0018] According to the first magnetic force data and the gravity data, a rotation matrix is determined, the rotation matrix being used for switching of coordinate systems; a first vector of the camera in an electronic device coordinate system is obtained; and according to the rotation matrix and the first vector, a first elevation angle and a first azimuth angle are determined.

[0019] In this way, the rotation matrix is determined by the first magnetic force data and the gravity data, so that the rotation matrix is more accurate.

[0020] As an example of the present application, the operation of determining, by the electronic device according to the rotation matrix and the first vector, the first elevation angle and the first azimuth angle includes:

[0021] The rotation matrix is multiplied by the first vector to obtain a second vector of the camera in a terrestrial coordinate system; the first elevation angle is determined according to a Z-axis component of the second vector in the terrestrial coordinate system; and the first azimuth angle is determined according to an X-axis component and a Y-axis component of the second vector in the terrestrial coordinate system.

[0022] In this way, the coordinate system conversion can be quickly and accurately performed by the rotation matrix, and the first azimuth angle and the second azimuth angle can be quickly determined.

[0023] As an example of the present application, the operation of determining, by the electronic device according to the azimuth angle difference, the elevation angle difference, and the field of view angle of the camera, the second coordinate information includes:

[0024] The azimuth angle difference and the elevation angle difference are converted into a plane rectangular coordinate system in which the first preview image is located; and according to the field of view angle, an angle coordinate in the plane rectangular coordinate system is converted into a proportional coordinate to obtain the second coordinate information.

[0025] In this way, by the field of view angle of the camera, the azimuth angle difference and the elevation angle difference between the camera and the celestial body can be converted into the first preview image, so that the position of the celestial body in the preview image can be accurately determined when the camera captures the celestial body.

[0026] As an example of the present application, the electronic device includes a magnetic force sensor and a gyroscope sensor;

[0027] Before the electronic device determines the azimuth angle and the elevation angle of the camera according to the first magnetic force data collected by the magnetic force sensor and the gravity data collected by the gravity sensor to obtain the first azimuth angle and the first elevation angle, respectively, a frame stabilization operation can also be performed.

[0028] That is, the electronic device continuously collects multiple frames of preview images through the camera; in the process of collecting each frame of preview image, the gyroscope sensor collects the gyroscope data corresponding to each frame of preview image, and the magnetic induction sensor collects the second magnetic force data corresponding to each frame of preview image; according to the gyroscope data corresponding to each frame of preview image, the magnetic force weight queue is updated, and the magnetic force weight queue includes the weight corresponding to the second magnetic force data corresponding to each frame of preview image; according to the second magnetic force data corresponding to each frame of preview image and the weight corresponding to each frame of preview image in the magnetic force weight queue, the average magnetic force data is determined, and the first magnetic force data is obtained.

[0029] Therefore, since the fluctuation of the magnetic force data collected by the magnetic induction sensor can be relatively large, in order to obtain accurate magnetic force data, the electronic device performs frame stabilization operation through the gyroscope data collected by the gyroscope sensor, thereby reducing the influence of the shaking of the electronic device on the collected magnetic force data.

[0030] As an example of the present application, the operation of updating the magnetic force weight queue according to the gyroscope data corresponding to each frame of preview image includes:

[0031] In the case that the collected preview image is the first frame of preview image, the weight corresponding to the first frame of preview image is determined as 1; in the case that the collected preview image is not the first frame of preview image, the weight corresponding to the currently collected preview image is determined according to the gyroscope data corresponding to the currently collected preview image and the weight corresponding to the last collected frame of preview image; the weight corresponding to the currently collected preview image and the magnetic force data are stored in the magnetic force weight queue.

[0032] As an example of the present application, the electronic device can also adjust the size of the prediction box.

[0033] That is, the electronic device obtains the second magnetic force data collected by the magnetic induction sensor; obtains the current location of the electronic device through the positioning device; according to the current location of the electronic device, determines the geomagnetic size of the current geographic location of the electronic device through the world magnetic model, to obtain the third magnetic force data; determines the deviation value between the second magnetic force data and the third magnetic force data; determines the display size of the prediction box according to the deviation value.

[0034] It should be noted that the display position of the prediction box is the position of the celestial body in the preview image.

[0035] Therefore, since the magnetic induction sensor is too susceptible to interference when collecting magnetic force data, by comparing the collected magnetic force data with the theoretical magnetic force data, the confidence of the magnetic force data collected by the magnetic induction sensor can be determined. At the same time, the size of the prediction box is adjusted through the deviation value of the magnetic force data, thereby improving the display richness and reliability.

[0036] In a second aspect, an image processing apparatus is provided, which has functions to implement the behaviors of the image processing method in the first aspect. The image processing apparatus includes at least one module for implementing the image processing method provided in the first aspect. The apparatus is applied in an electronic device, and the apparatus includes:

[0037] a detection module configured to perform celestial body detection on a first preview image captured by the camera during image capturing by the camera;

[0038] a first determination module configured to determine first coordinate information of the identified celestial body in the first preview image when the celestial body exists in the first preview image;

[0039] a first update module configured to update the first preview image to a second preview image when the first coordinate information is the same as second coordinate information, the second preview image having a better image quality than the first preview image, and the second coordinate information being coordinate information of the celestial body in the first preview image determined based on a plurality of sensors in the electronic device.

[0040] As an example of the present application, the apparatus further includes:

[0041] an acquisition module configured to acquire sensor data captured by each sensor of the plurality of sensors in the electronic device during image capturing by the camera;

[0042] a second determination module configured to determine an azimuth angle difference and an altitude angle difference between the camera and the celestial body based on the sensor data captured by each sensor;

[0043] a third determination module configured to determine the second coordinate information based on the azimuth angle difference, the altitude angle difference, and a field of view angle of the camera.

[0044] As an example of the present application, the plurality of sensors includes a magnetic sensor, a positioning device, and a gravity sensor.

[0045] The second determination module is configured to:

[0046] determine an azimuth angle and an altitude angle of the camera based on first magnetic data captured by the magnetic sensor and gravity data captured by the gravity sensor, to obtain a first azimuth angle and a first altitude angle respectively, determine a coordinate of the celestial body to obtain a third coordinate, and determine a coordinate of the electronic device to obtain a fourth coordinate, determine an altitude angle and an azimuth angle of the celestial body relative to the electronic device based on the third coordinate and the fourth coordinate, to obtain a second azimuth angle and a second altitude angle, determine an azimuth angle difference between the first azimuth angle and the second azimuth angle to obtain the azimuth angle difference between the camera and the celestial body, and determine an altitude angle difference between the first altitude angle and the second altitude angle to obtain the altitude angle difference between the camera and the celestial body.

[0047] As an example of the present application, the second determining module is configured to:

[0048] According to the first magnetic force data and the gravity data, a rotation matrix is determined, the rotation matrix being used for switching coordinate systems; a first vector of the camera in the electronic device coordinate system is obtained; according to the rotation matrix and the first vector, a first elevation angle and a first azimuth angle are determined.

[0049] As an example of the present application, the second determining module is configured to:

[0050] The rotation matrix is multiplied by the first vector to obtain a second vector of the camera in the earth coordinate system; according to a Z-axis component of the second vector in the earth coordinate system, the first elevation angle is determined; according to an X-axis component and a Y-axis component of the second vector in the earth coordinate system, the first azimuth angle is determined.

[0051] As an example of the present application, the third determining module is configured to:

[0052] The azimuth angle difference and the elevation angle difference are converted into a plane rectangular coordinate system in which the first preview image is located; according to a field of view angle, an angle coordinate in the plane rectangular coordinate system is converted into a proportional coordinate to obtain second coordinate information.

[0053] As an example of the present application, the electronic device comprises a magnetic force sensor and a gyroscope sensor;

[0054] The device further comprises:

[0055] The first acquisition module is configured to continuously acquire a plurality of preview images through the camera;

[0056] The second acquisition module is configured to, in the process of acquiring each preview image, acquire, through the gyroscope sensor, gyroscope data corresponding to each preview image, and acquire, through the magnetic force sensor, second magnetic force data corresponding to each preview image;

[0057] The second updating module is configured to update a magnetic force weight queue according to the gyroscope data corresponding to each preview image, the magnetic force weight queue comprising a weight corresponding to the second magnetic force data corresponding to each preview image;

[0058] The fourth determining module is configured to determine average magnetic force data according to the second magnetic force data corresponding to each preview image and the weight corresponding to each preview image in the magnetic force weight queue, to obtain the first magnetic force data.

[0059] As an example of the present application, the second updating module is configured to:

[0060] In a case where the collected preview image is a first frame preview image, the weight corresponding to the first frame preview image is determined as 1; in a case where the collected preview image is not the first frame preview image, the weight corresponding to the currently collected preview image is determined according to the gyroscope data corresponding to the currently collected preview image and the weight corresponding to the last collected preview image; and the weight corresponding to the currently collected preview image and the magnetic force data are stored in the magnetic force weight queue.

[0061] In a third aspect, an electronic device is provided, which includes a processor and a memory in its structure, the memory is configured to store a program supporting the electronic device to execute the image processing method provided in the first aspect, and store data involved in the image processing method provided in the first aspect. The processor is configured to execute the program stored in the memory. The electronic device can further include a communication bus configured to establish a connection between the processor and the memory.

[0062] In a fourth aspect, a computer readable storage medium is provided, which stores instructions, when the instructions are executed on a computer, the computer executes the image processing method provided in the first aspect.

[0063] In a fifth aspect, a computer program product is provided, which includes instructions, when the instructions are executed on a computer, the computer executes the image processing method provided in the first aspect.

[0064] The technical effects obtained by the second aspect, the third aspect, the fourth aspect and the fifth aspect are similar to the technical effects obtained by the corresponding technical means in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0065] FIG. 1 is a structural schematic diagram of an electronic device provided by an embodiment of the present application;

[0066] FIG. 2 is a block diagram of a software system of an electronic device provided by an embodiment of the present application;

[0067] FIG. 3 is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0068] FIG. 4 is a schematic diagram of another application scenario provided by an embodiment of the present application;

[0069] FIG. 5 is a schematic diagram of another application scenario provided by an embodiment of the present application;

[0070] FIG. 6 is a flow schematic diagram of an image processing method provided by an embodiment of the present application;

[0071] FIG. 7 is a schematic diagram of a coordinate system provided by an embodiment of the present application;

[0072] FIG. 8 is a schematic diagram of an altitude angle of a celestial body according to an embodiment of the present application;

[0073] FIG. 9 is a flowchart of another image processing method according to an embodiment of the present application;

[0074] FIG. 10 is a flowchart of a method of stable frame operation according to an embodiment of the present application;

[0075] FIG. 11 is a flowchart of a method of adjusting a size of a prediction box according to an embodiment of the present application. DETAILED DESCRIPTION

[0076] In order to make the objects, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0077] It should be understood that the “plurality” mentioned in the present application refers to two or more. In the description of the present application, unless otherwise specified, “ / ” represents the meaning of or, for example, A / B can represent A or B; “and / or” in the present application is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. In addition, in order to clearly describe the technical solutions of the present application, the same items or similar items with basically the same functions and roles are distinguished by using “first”, “second”, etc. The person skilled in the art can understand that “first”, “second”, etc. do not limit the quantity and execution order, and “first”, “second”, etc. also do not necessarily mean different.

[0078] In the present application, the reference to “one embodiment” or “some embodiments” means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Therefore, the statements “in one embodiment”, “in some embodiments”, “in other some embodiments”, “in further some embodiments” and the like appearing in different places in the specification are not necessarily all referring to the same embodiment, but mean “one or more but not all embodiments”, unless otherwise specifically emphasized. The terms “include”, “contain”, “have” and their variants mean “include but not limited to”, unless otherwise specifically emphasized.

[0079] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0080] With the development of technology, the use scenarios of the camera function of electronic devices such as mobile phones are becoming more and more rich, for example, a user can use an electronic device to shoot celestial bodies such as the moon and the sun. And in order to get rich celestial images, the electronic device has launched multiple celestial shooting modes. For example, in order to get a clear moon image, the electronic device provides a moon shooting mode (also can be called a moon mode, a moon viewing mode, a moon shooting mode, a moon watching mode, etc.). In the case of shooting the sun, the electronic device provides a sunrise / sunset mode, in which the electronic device can perform atmosphere rendering, image quality enhancement, and other operations on the shooting picture.

[0081] However, the electronic device will perform sharpness processing on the collected image in the case that the electronic device identifies that the collected image contains a celestial body, and the electronic device may misdetect the celestial body, for example, a light source such as a street lamp may be misidentified as the moon or the sun, which will cause image processing errors.

[0082] In order to improve the accuracy of image processing, the embodiments of the present application provide an image processing method, in which the electronic device can perform celestial body detection on a first preview image collected by a camera during the process of collecting the image by the camera; in the case that the first preview image contains a celestial body such as the moon or the sun, determine first coordinate information of the identified celestial body in the first preview image; if the first coordinate information is the same as second coordinate information of the actual celestial body in the first preview image, then update the first preview image to a second preview image, and the image quality of the second preview image is better than that of the first preview image. In this way, by comparing the position of the identified celestial body in the preview image with the display position of the celestial body in the preview image in actual life, it can be further determined whether the celestial body exists in the preview image, and the accuracy of image processing is improved.

[0083] Before explaining the image processing method provided by the embodiments of the present application in detail, the electronic device involved in the embodiments of the present application will be described.

[0084] As an example, the method can be applied in an electronic device with a camera. As an example but not limitation, the electronic device can be, but not limited to, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, a vehicle-mounted device, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a mobile phone, a smart watch, etc., and the embodiments of the present application do not limit this.

[0085] FIG. 1 is a structural schematic diagram of an electronic device according to an embodiment of the present application. Referring to FIG. 1, the electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0086] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than those illustrated, or combine certain components, or split certain components, or different arrangement of components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0087] The processor 110 can include one or more processing units such as: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0088] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0089] The memory in the processor 110 can also be configured to store instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can save instructions or data that the processor 110 has just used or repeatedly uses. If the processor 110 needs to use the instructions or data again, it can directly call from the memory. Avoiding repeated access, reducing the waiting time of the processor 110, thus improving the efficiency of the system.

[0090] In some embodiments, the processor 110 can include one or more interfaces, such as: can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0091] It can be understood that the interface connection relationship between the modules shown in the embodiments of the present application is only illustrative and does not constitute a structural limitation of the electronic device 100. In some other embodiments of the present application, the electronic device 100 can also use different interface connection modes or a combination of multiple interface connection modes in the above embodiments.

[0092] The wireless communication function of the electronic device 100 can be realized by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, the baseband processor, and the like.

[0093] The electronic device 100 realizes the display function through the GPU, the display screen 194, and the application processor, and the like. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs that execute program instructions to generate or change display information.

[0094] The display screen 194 is used to display images, videos, and the like. The display screen 194 includes a display panel. The display panel can use a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diode (QLED), and the like. In some embodiments, the electronic device 100 can include 1 or N display screens 194, and N is an integer greater than 1.

[0095] The electronic device 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor, and the like.

[0096] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing to convert it into an image visible to the naked eye. The ISP can also optimize the algorithm of the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature, and other parameters of the shooting scene. In some embodiments, the ISP can be arranged in the camera 193.

[0097] Camera 193 is used to capture still images or videos. Objects project optical images through the lens to the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into a standard RGB, YUV, etc. format image signal. In some embodiments, electronic device 100 can include one or N cameras 193, where N is an integer greater than 1.

[0098] The digital signal processor is used to process digital signals, in addition to being able to process digital image signals, it can also process other digital signals. For example, when electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0099] The video codec is used to compress or decompress digital video. Electronic device 100 can support one or more video codecs. In this way, electronic device 100 can play or record videos in multiple encoding formats, such as: moving picture experts group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0100] NPU is a neural-network (NN) computing processor, which learns from the structure of biological neural networks, such as the transmission mode between human brain neurons, and can quickly process input information and continuously self-learn. Through NPU, electronic device 100 can realize intelligent cognition and other applications, such as: image recognition, face recognition, speech recognition, text understanding, etc.

[0101] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to realize data storage functions. For example, music, video, etc. files are saved in the external memory card.

[0102] The internal memory 121 can be used to store computer executable program codes including instructions. The processor 110 performs various functional applications of the electronic device 100 and data processing by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program (such as a sound play function, an image play function, etc.) required by a function, etc. The data storage area can store data (such as audio data, a phone book, etc.) created by the electronic device 100 during use, etc. In addition, the internal memory 121 can include a high-speed random access memory, and can further include a non-volatile memory such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0103] The electronic device 100 can implement an audio function such as music play, recording, etc. through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor, etc.

[0104] The pressure sensor 180A is used to sense a pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194. There are many types of pressure sensors 180A, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, etc. The capacitive pressure sensor can include at least two parallel plates with conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation is applied to the display screen 194, the electronic device 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 can also calculate the position of the touch according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation instructions. For example, when a touch operation with a touch operation intensity less than a pressure threshold is applied to a short message application icon, an instruction to view a short message is executed. When a touch operation with a touch operation intensity greater than or equal to the pressure threshold is applied to the short message application icon, an instruction to create a new short message is executed.

[0105] The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake photography. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of shaking of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and lets the lens offset the shaking of the electronic device 100 by reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and motion sensing game scenarios.

[0106] The barometric pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude, assists in positioning and navigation by measuring the air pressure value by the barometric pressure sensor 180C.

[0107] The magnetic sensor 180D (or magnetic induction sensor) includes a Hall sensor. The electronic device 100 can detect the opening and closing of a flip cover by using the magnetic sensor 180D. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. The electronic device 100 sets features such as automatic unlocking of the flip cover according to the detected opening and closing state of the cover or the flip cover.

[0108] In some embodiments, the magnetic induction sensor can also detect magnetic force data in the current environment.

[0109] The acceleration sensor 180E can detect the acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the acceleration sensor 180E can detect the magnitude and direction of gravity. The acceleration sensor 180E can also be used to identify the posture of the electronic device 100 and applied to landscape / portrait switching, pedometer applications, etc.

[0110] The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance by infrared or laser. In some embodiments, in a shooting scenario, the electronic device 100 can use the distance sensor 180F to measure distance to achieve fast focusing.

[0111] The ambient light sensor 180L is used to sense the brightness of the ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the sensed brightness of the ambient light. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touch.

[0112] Touch sensor 180K, also referred to as "touch panel". Touch sensor 180K can be disposed on display screen 194, and touch screen, also referred to as "touch panel", is composed of touch sensor 180K and display screen 194. Touch sensor 180K is configured to detect a touch operation applied thereon or in the vicinity. Touch sensor 180K can transmit the detected touch operation to the application processor to determine the touch event type. Visual output related to the touch operation can be provided through display screen 194. In some other embodiments, touch sensor 180K can also be disposed on the surface of electronic device 100, which is different from the position where display screen 194 is located.

[0113] Next, the software system of electronic device 100 is described.

[0114] The software system of electronic device 100 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. Embodiments of the present application take the Android system with a layered architecture as an example to exemplarily describe the software system of electronic device 100.

[0115] FIG. 2 is a block diagram of a software system of electronic device 100 according to an embodiment of the present application. Referring to FIG. 2, the layered architecture divides software into several layers, each of which has a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, an application layer, an application framework layer, an Android runtime and a system layer, and a kernel layer.

[0116] The application layer can include a series of application packages. As shown in FIG. 2, the application packages can include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, and other applications.

[0117] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes some pre-defined functions. As shown in FIG. 2, the application framework layer can include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc. The window manager is used to manage window programs. The window manager can acquire a display screen size, determine whether there is a status bar, lock a screen, and capture a screen, etc. The content provider is used to store and acquire data, and make the data accessible to applications. The data can include videos, images, audios, dialed and received calls, browsing history and bookmarks, a phone book, etc. The view system includes visual controls, such as a control for displaying text, a control for displaying pictures, etc. The view system can be used to build a display interface of an application. The display interface can be composed of one or more views, such as a view for displaying a short message notification icon, a view for displaying text, and a view for displaying pictures. The phone manager is used to provide communication functions of the electronic device 100, such as management of a call state (including call connection, call hang-up, etc.). The resource manager provides various resources for applications, such as localized strings, icons, pictures, layout files, video files, etc. The notification manager enables an application to display notification information in a status bar. The notification information can be used to convey a message of a notification type, which can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify a download completion, a message reminder, etc. The notification manager can also be a notification in a form of a chart or a scroll bar text in a top status bar of a system, such as a notification of an application running in the background. The notification manager can also be a notification in a form of a dialog window on a screen, such as a text information prompt in a status bar, a prompt sound, a vibration of the electronic device, a blinking of an indicator light, etc.

[0118] The Android runtime includes a core library and a virtual machine. The Android runtime is responsible for scheduling and management of the Android system. The core library includes two parts: one part is a function function required to be called by the java language, and the other part is the core library of the Android. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java files of the application layer and the application framework layer into binary files. The virtual machine is used to perform functions of management of an object life cycle, stack management, thread management, security and exception management, and garbage collection, etc.

[0119] The system library can include a plurality of functional modules, such as a surface manager, media libraries, a three-dimensional graphics processing library (such as OpenGL ES), a 2D graphics engine (such as SGL), and the like. The surface manager is used to manage the display subsystem and provides a plurality of applications with fusion of 2D and 3D layers. The media libraries support playback and recording of a plurality of commonly used audio, video formats, and still image files, and the like. The media libraries can support a plurality of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, and the like. The three-dimensional graphics processing library is used to implement three-dimensional graphics drawing, image rendering, composition, and layer processing, and the like. The 2D graphics engine is a drawing engine for 2D drawing.

[0120] The kernel layer is a layer between hardware and software. The kernel layer at least includes a display driver, a camera driver, an audio driver, and a sensor driver.

[0121] The following describes, by way of example, the working flow of the software and hardware of the electronic device 100 in the context of capturing a photograph.

[0122] When the touch sensor 180K receives a touch operation, a corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, a timestamp of the touch operation, and the like). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer and identifies the control corresponding to the raw input event. Taking the touch operation as a single-click operation and the control corresponding to the single-click operation as the control of the camera application icon as an example, the camera application calls an interface of the application framework layer, starts the camera application, and then calls the kernel layer to start the camera driver to capture a still image or a video through the camera 193.

[0123] Next, the application scenarios involved in the embodiments of the present application are explained and described by taking a mobile phone as an example of the electronic device.

[0124] Please refer to FIG. 3, which is a schematic diagram of an application scenario provided by the embodiment of the present application. In an application scenario, a user may take a picture through the camera of an electronic device in the process of using the electronic device. For example, when the electronic device takes a picture of the moon, the electronic device may display a shooting interface as shown in (a) of FIG. 3 on the display screen after starting the camera. The shooting interface displays a preview image A collected by the camera. During the process of taking a picture through the electronic device, the user may adjust the zoom ratio of the current camera. For example, the user may click a zoom control to change the zoom ratio from 1x to 10x. The electronic device may collect an image through the camera according to the adjusted zoom ratio in response to the adjustment operation of the zoom ratio of the camera, and obtain a preview image B as shown in (b) of FIG. 3. In this case, the electronic device may also detect whether the preview image B contains celestial bodies such as the moon and the sun. In the case where the preview image B contains the moon, the electronic device may determine the first coordinate information of the moon identified in the preview image B, and determine the second coordinate information of the moon in the preview image when the image is collected according to the current azimuth and elevation angle of the camera of the electronic device. If the first coordinate information and the second coordinate information are the same, the electronic device may enter a moon-viewing shooting mode. For example, the electronic device may reduce the exposure of the preview image B to obtain a preview image C as shown in (c) of FIG. 3. If the user clicks a shooting control P1 in this case, the electronic device may perform image exposure based on the preview image C in response to the clicking operation of the shooting control P1. If the user needs to view the taken image, the user may click an image viewing control P2 in the shooting interface, as shown in (d) of FIG. 3. The electronic device may display a first target image E as shown in (e) of FIG. 3 in response to the clicking operation of the image viewing control P2.

[0125] In another possible scenario, after the electronic device collects a preview image B through the camera S1 according to the adjusted zoom ratio in response to the adjustment operation of the zoom ratio of the camera S1, and the first coordinate information and the second coordinate information are the same, the electronic device reduces the exposure of the preview image B to obtain a preview image C as shown in (b) of FIG. 4, and displays a preview image D collected by another camera S2 of the electronic device in the preview image C in a picture-in-picture manner. If the user clicks the shooting control P1 in this case, the mobile phone may perform image exposure based on the preview image D and the preview image C in response to the clicking operation of the shooting control P1. If the user needs to view the taken image, the user may click the image viewing control P2 in the shooting interface, as shown in (c) of FIG. 4. The mobile phone may display a first target image E as shown in (d) of FIG. 4 in response to the clicking operation of the image viewing control P2.

[0126] It should be noted that the camera S1 collecting the preview image C can be a telephoto camera of the electronic device, and the camera S2 collecting the preview image D can be a main camera of the electronic device.

[0127] In another possible scenario, if the user takes a photo of the sun, the electronic device can display a shooting interface as shown in (a) of FIG. 5 on the display after starting the camera, and the shooting interface displays a preview image F collected by the camera. During the shooting process of the user through the electronic device, the electronic device can also detect whether the preview image F contains celestial bodies such as the sun and the moon. In the case where the preview image F contains the sun, the electronic device can determine first coordinate information of the sun identified in the preview image F in the preview image F, and the sun is framed in the preview image F by a prediction box (which can be a square box), and determine second coordinate information of the sun in the preview image when the image is collected according to the current azimuth and elevation angle of the camera of the electronic device; if the first coordinate information and the second coordinate information are the same, the electronic device can enter the sunrise / sunset mode. For example, the electronic device can display the current mode "sunrise / sunset" identifier in the preview image F, and optimize the preview image F, such as performing quality enhancement on the preview image F, adjusting the color temperature and tone of the preview image F, to obtain a preview image G as shown in (b) of FIG. 5. If the user clicks the shooting control P1 in this case, the electronic device responds to the click operation on the shooting control P1 and performs image exposure based on the preview image G. If the user needs to view the captured image, referring to (c) of FIG. 5, the user can click the image viewing control P2 in the shooting interface. The electronic device responds to the click operation on the image viewing control P2 and can display a second target image H as shown in (d) of FIG. 5.

[0128] It should be noted that the above-mentioned application scenarios shown in FIGS. 3- and 5 are only used as examples for description of the embodiments of the present application, and do not limit the embodiments of the present application.

[0129] Based on the execution of the application scenarios provided in the above embodiments, the image processing method provided by the embodiments of the present application will be introduced next. Please refer to FIG. 6, which is a flowchart of an image processing method according to an example, which is used as an example and not limited, and the method is used as an example for description of the electronic device, and the method can include the following parts or all contents:

[0130] Step 601: collecting a first preview image by a camera of the electronic device.

[0131] It should be noted that the first preview image can be an image collected when the camera is just started, or an image collected by the camera before the zoom ratio is greater than the preset ratio threshold. For example, the first preview image can be the preview image A shown in (a) of FIG. 3. Alternatively, the first preview image can be the preview image F shown in (a) of FIG. 5.

[0132] Of course, during the image collection process of the camera of the electronic device, the user can also adjust the zoom ratio of the camera in order to obtain the desired image. Therefore, the first preview image can also be a preview image collected by the camera after the zoom ratio of the camera is greater than the preset ratio threshold. For example, the first preview image can also be the preview image B shown in (b) of FIG. 3.

[0133] It should be noted that the preset ratio threshold can be set in advance according to requirements, for example, the preset ratio threshold can be 10 times (or written as 10X), 15 times, 20 times, or 30 times, etc.

[0134] Step 602: During the image collection process by the camera of the electronic device, performing celestial body detection on a first preview image collected by the camera.

[0135] During the image collection process by the camera of the electronic device, in order to accurately process the collected preview image, the electronic device can perform celestial body detection on the first preview image, that is, detect whether the first preview image contains celestial bodies such as the sun or the moon.

[0136] In some embodiments, the electronic device can be provided with a neural network model, which can perform object recognition. In this way, the electronic device can identify each object in the first preview image through the set neural network model, and detect whether there is a moon or a sun by identifying each object. Alternatively, the electronic device can have an AI (artificial intelligence) recognition function. In this way, the electronic device can identify each object in the first preview image through the AI recognition function to detect whether there is a moon or a sun in the first preview image. The embodiments of the present application do not make specific limitations in this regard.

[0137] Since the electronic device usually enters the moon-viewing shooting mode when the zoom ratio of the camera is greater than or equal to the preset ratio threshold and the moon exists in the preview image, if the first preview image is a preview image collected when the zoom ratio of the camera is greater than or equal to the preset ratio threshold, it is very likely that the user wants to enter the moon-viewing shooting mode. Therefore, the electronic device can first detect whether the moon exists in the first preview image, and continue to detect whether the sun exists in the case where the moon does not exist.

[0138] Similarly, in a case where the zoom ratio of the camera is less than the preset zoom ratio threshold, the electronic device can first detect whether the sun exists in the first preview image, and in a case where the sun does not exist, continue to detect whether the moon exists.

[0139] Step 603: In a case where the celestial body exists in the first preview image, determining first coordinate information of the identified celestial body in the first preview image.

[0140] In some embodiments, the electronic device can establish a plane rectangular coordinate system in the first preview image, and determine the coordinates of the identified celestial body in the plane rectangular coordinate system in the first preview image to obtain the first coordinate information.

[0141] Step 604: Obtaining sensor data collected by each sensor of the plurality of sensors in the electronic device.

[0142] It should be noted that the electronic device can perform the operation of step 604 in a case where the celestial body is identified to exist in the first preview image, or can perform the operation of step 604 during image collection after the camera is started, and the embodiments of the present application do not make specific limitations thereon. Of course, in some embodiments, in order to reduce the power consumption of the electronic device, the electronic device can obtain the sensor data collected by each sensor of the plurality of sensors in a case where the celestial body is identified to exist in the first preview image. In addition, the embodiments of the present application are described by taking the operation of step 604 in a case where the celestial body exists in the first preview image as an example.

[0143] As an example, the plurality of sensors include a magnetic inductor, a positioning device, a gravity sensor, a gyroscope sensor, etc. The magnetic inductor can also be referred to as a magnetometer, which can be used to detect the magnetic force data of the position where the electronic device is located, the positioning device is used to determine the position where the electronic device is located, the gravity sensor is used to detect the orientation of the camera in the electronic device, and the gyroscope sensor is used to detect the shaking condition of the electronic device, etc.

[0144] In some embodiments, after obtaining the sensor data collected by each sensor of the plurality of sensors, the electronic device can determine the azimuth angle difference and the elevation angle difference between the camera and the celestial body according to the sensor data collected by each sensor. For example, the electronic device can determine the azimuth angle difference and the elevation angle difference between the camera and the celestial body according to the sensor data collected by each sensor, which can refer to the operations of steps 605-608 described below.

[0145] It is worth noting that the second coordinate information is determined by the sensor data collected by the plurality of sensors, so that the scheme is more systematic, and the accuracy of determining the coordinates of the celestial body in the preview image is improved.

[0146] Step 605: determining the azimuth angle and the elevation angle of the camera according to the first magnetic force data collected by the magnetic force sensor and the gravity data collected by the gravity sensor, to obtain the first azimuth angle and the first elevation angle respectively.

[0147] In some embodiments, the operation of determining the azimuth angle and the elevation angle of the camera according to the first magnetic force data collected by the magnetic force sensor and the gravity data detected by the gravity sensor, to obtain the first azimuth angle and the first elevation angle respectively, includes: determining a rotation matrix according to the first magnetic force data and the gravity data, the rotation matrix being used for switching the coordinate system; obtaining a first vector of the camera in the electronic device coordinate system; and determining the first elevation angle and the first azimuth angle according to the rotation matrix and the first vector.

[0148] It should be noted that the rotation matrix can convert the electronic device coordinate system to the world coordinate system (which can also be referred to as the earth coordinate system in the embodiments of the present application). For example, the electronic device coordinate system can be established with reference to the electronic device, and can be as shown in (a) of FIG. 7. The world coordinate system (earth coordinate system) can be as shown in (b) of FIG. 7.

[0149] It should be noted that the rotation matrix is determined by the first magnetic force data and the gravity data, so that the determined rotation matrix is more accurate.

[0150] As an example, the rotation matrix can be determined by the gravity data collected by the gravity sensor and the first magnetic force data collected by the magnetic force sensor. For example, the electronic device can determine the rotation matrix according to the gravity vector and the magnetic field vector. The gravity data can be acceleration data of the electronic device in a three-dimensional space, which is usually represented as a three-dimensional vector g=(gx, gy, gz), and g can also be referred to as a gravity vector. The magnetic force data can be magnetic field data of the electronic device in a three-dimensional space, which is usually represented as a three-dimensional vector m=(mx, my, mz), and m can also be referred to as a magnetic field vector.

[0151] In some embodiments, the electronic device can determine the unit vector of the gravity vector by the following first formula (1):

[0152] It should be noted that in the above first formula (1), g is the gravity vector, and g norm is the unit vector of the gravity vector.

[0153] In some embodiments, the electronic device can determine the horizontal component of the magnetic field vector by the following second formula (2):

[0154] It should be noted that in the above second formula (2), hnorm a magnetic field vector in a horizontal component.

[0155] In some embodiments, the electronic device can multiply the magnetic field vector in the horizontal component and the unit vector of the gravity vector to obtain an east vector, i.e., e = h norm *g norm , e = (ex, ey, ez); the electronic device constructs a rotation matrix according to the east vector, the magnetic field vector in the horizontal component and the unit vector of the gravity vector. The rotation matrix R can be represented by the following third formula (3):

[0156] It should be noted that the electronic device can also determine the rotation matrix in other ways, for example, the electronic device can represent the rotation matrix by a unit matrix, and the embodiments of the present application do not make specific limitations.

[0157] As an example, the electronic device can determine a first vector of the camera orientation in the electronic device coordinate system, which is c = (cx, cy, cz) in the electronic device coordinate system. Generally, the first vector of the camera orientation can be represented as (0, 0, -1), i.e., the negative Z axis direction of the electronic device.

[0158] In some embodiments, the operation of determining the first elevation angle and the first azimuth angle according to the rotation matrix and the first vector includes: multiplying the rotation matrix and the first vector to obtain a second vector of the camera in the earth coordinate system; determining the first elevation angle according to the Z axis component of the second vector in the earth coordinate system; and determining the first azimuth angle according to the X axis component and the Y axis component of the second vector in the earth coordinate system.

[0159] Since the rotation matrix is used to convert the electronic device coordinate system into the earth coordinate system, or convert the earth coordinate system into the electronic device coordinate system. Therefore, the first vector is multiplied by the rotation matrix to obtain the second vector of the camera in the earth coordinate system. That is, cearth = R * c, and cearth is the second vector.

[0160] It should be noted that the coordinate system conversion can be quickly and accurately performed through the rotation matrix.

[0161] As an example, the operation of determining the first elevation angle according to the Z axis component of the second vector in the earth coordinate system includes: determining the inverse sine function value of the Z axis component of the second vector in the earth coordinate system to obtain the first elevation angle, i.e., h1 = arcsin(cearthz). Or, determining the inverse cosine function value of the Z axis component of the second vector in the earth coordinate system to obtain the first elevation angle, i.e., h1 = π / 2 - arccos(cearthz).

[0162] Of course, the manner in which the electronic device determines the first elevation angle according to the Z-axis component of the second vector in the earth coordinate system can further include other operations, which will not be repeated one by one in the embodiments of the present application.

[0163] As an example, the operation of determining the first azimuth angle according to the X-axis component and the Y-axis component of the second vector in the earth coordinate system includes: determining the arctangent function value of the X-axis component and the Y-axis component of the second vector in the earth coordinate system to obtain the first azimuth angle, that is, A = arctan2(cearthy, cearthz). Alternatively, the arctangent function value of the X-axis component and the Y-axis component of the second vector in the earth coordinate system is determined to obtain the first azimuth angle.

[0164] It should be noted that the manner in which the electronic device determines the first azimuth angle according to the X-axis component and the Y-axis component of the second vector in the earth coordinate system can further include other operations, which will not be repeated one by one in the embodiments of the present application.

[0165] In some embodiments, the electronic device can determine the azimuth angle and the elevation angle of the camera according to the first magnetic force data collected by the magnetic force sensor and the gravity data collected by the gravity sensor, to obtain the first azimuth angle and the first elevation angle respectively. The first azimuth angle and the first elevation angle can also be determined in other manners. For example, the electronic device can determine the first azimuth angle according to the first magnetic force data collected by the magnetic force sensor, and determine the first elevation angle according to the gravity data collected by the gravity sensor. That is, the electronic device can determine the azimuth angle of the camera according to the magnetic field direction of the first magnetic force data collected by the magnetic force sensor, and determine the elevation angle of the camera according to the gravity data.

[0166] Step 606: determining the coordinates of the celestial body to obtain a third coordinate, and determining the coordinates of the electronic device to obtain a fourth coordinate.

[0167] In some embodiments, since the first Kepler equation describes the shape of the planetary orbit, that is, the orbit of the planet around the sun is an ellipse, and the sun is located at one focus of the ellipse. The second Kepler equation, also known as the Kepler time equation, describes the second speed of the planet moving on the orbit, which indicates that the area speed swept by the planet moving on the elliptical path is constant. Therefore, the electronic device can determine the coordinates of the celestial body according to the system time of the electronic device, to obtain a third coordinate (which can be referred to as a cosmic coordinate) by the first Kepler equation and the second Kepler equation, and determine the coordinates of the electronic device according to the system time and the position information collected by the positioning device, to obtain a fourth coordinate.

[0168] It should be noted that the first Kepler equation can be as shown in the fourth formula (4) below, and the second Kepler equation can be as shown in the fifth formula (5) below. The fourth formula (4) can be:

[0169] It should be noted that in the above fourth formula (4), r is the distance from the planet to the sun, a is the semi-major axis of the ellipse, e is the eccentricity of the ellipse, and θ is the position angle of the planet on the orbit.

[0170] As an example, the fifth formula (5) can be: M = E - esinE (5)

[0171] It should be noted that in the above fifth formula (5), M is the mean anomaly, and E is the eccentric anomaly.

[0172] Step 607: determining the altitude angle and the azimuth angle of the celestial body relative to the electronic device according to the third coordinate and the fourth coordinate, to obtain the second azimuth angle and the second altitude angle.

[0173] In some embodiments, the third coordinate determined by the electronic device can be the longitude φ and the latitude λ of the position where the electronic device is located. Through the above step 606, the electronic device can determine the fourth coordinate (which can also be referred to as equatorial coordinate) of the celestial body, i.e., the right ascension α and the declination δ of the celestial body, and determine the current system time as t1 (usually represented in Greenwich Standard Time). In this case, the electronic device can determine the sidereal time of the current position where the electronic device is located, i.e., LST = GST + λ. Wherein, LST is the sidereal time of the current position where the electronic device is located, and GST is the Greenwich Sidereal Time, which can be determined by the system time t1. Then, the electronic device can determine the hour angle according to the sidereal time of the current position where the electronic device is located and the right ascension α of the celestial body, and the hour angle is the difference between the sidereal time of the right ascension and the position where the electronic device is located, i.e., the hour angle H = LST - α. The electronic device can determine the second azimuth angle and the second altitude angle according to the hour angle, the position where the electronic device is located, and the celestial body.

[0174] In order to facilitate understanding of the determination process of the second altitude angle, referring to FIG. 8, an altitude angle calculation diagram is provided in the embodiments of the present application.

[0175] In some embodiments, the electronic device determines the second altitude angle according to the hour angle, the position where the electronic device is located, and the celestial body by the following sixth formula (6). The electronic device determines the second azimuth angle according to the hour angle, the position where the electronic device is located, and the celestial body by the following seventh formula (7).

[0176] As an example, the sixth formula (6) can be: sin (h2) = sin (δ) sin (φ) + cos (δ) cos (φ) cos (H) (6)

[0177] It should be noted that in the above sixth formula (6), h2 is the second altitude angle.

[0178] As an example, the seventh formula (7) can be:

[0179] It should be noted that in the above seventh formula (7), A2 is the second altitude angle.

[0180] Step 608: determining an azimuth angle difference between the first azimuth angle and the second azimuth angle to obtain an azimuth angle difference between the camera and the celestial body, and determining an altitude angle difference between the first altitude angle and the second altitude angle to obtain an altitude angle difference between the camera and the celestial body.

[0181] As an example, the electronic device can subtract the first azimuth angle from the second azimuth angle to obtain the azimuth angle difference, and subtract the first altitude angle from the second altitude angle to obtain the altitude angle difference.

[0182] Step 609: determining the second coordinate information according to the azimuth angle difference, the altitude angle difference, and a field of view angle of the camera.

[0183] In some embodiments, the electronic device can convert the azimuth angle difference and the altitude angle difference into a planar rectangular coordinate system in which the first preview image is located; and convert an angle coordinate in the planar rectangular coordinate system into a proportional coordinate according to the field of view angle to obtain the second coordinate information.

[0184] For example, referring to FIG. 9, when the electronic device is in a horizontal screen state, the electronic device can represent the azimuth angle difference (e.g., 45 degrees) and the altitude angle difference (e.g., 30 degrees) in a planar rectangular coordinate system, which is established with the screen of the electronic device as the center. Then, the angle coordinate (which can also be referred to as an angle unit) can be converted into a proportional coordinate (which can also be referred to as a proportional unit), i.e., the altitude angle difference of 30 degrees is converted into 0.5, and the azimuth angle difference of 45 degrees is converted into 0.5.

[0185] In some embodiments, the field of view angle of the camera can be determined according to a zoom ratio of the camera, and the second coordinate information determined by the electronic device is different under different zoom ratios.

[0186] Step 610: determining whether the first coordinate information and the second coordinate information are the same, if yes, performing the operation of step 611 below, if not, the image acquisition can continue, and the image will not be processed.

[0187] Since false detection may occur in the detection of the celestial body in the first preview image, such as the street lamp or other lighting in the first preview image may be detected as the moon or the sun, in order to avoid false detection of the celestial body, the electronic device can compare the first coordinate information and the second coordinate information to determine whether the first coordinate information and the second coordinate information are the same. If the first coordinate information and the second coordinate information are not the same, it is likely that false detection occurs in the first preview image, that is, there is no celestial body in the first preview image, so image acquisition can continue and the preview image will not be processed. In the case where the first coordinate information and the second coordinate information are the same, it is indicated that the celestial body indeed exists in the first preview image. Therefore, the electronic device can perform the operation of step 611.

[0188] Step 611: In the case where the first coordinate information and the second coordinate information are the same, the electronic device can update the first preview image according to the type of the identified celestial body to obtain a second preview image.

[0189] It should be noted that the image quality of the second preview image is better than that of the first preview image.

[0190] In some embodiments, in the case where the celestial body in the first preview image is the moon, the moon shooting mode is entered. In the moon shooting mode, the electronic device can improve the clarity of the moon in the first preview image. In the case where the celestial body in the first preview image is the sun, the sunrise / sunset mode is entered. In the sunrise / sunset mode, the electronic device performs image quality enhancement or other operations on the first preview image.

[0191] In some embodiments, in the case where the celestial body in the first preview image is the moon, if the zoom ratio of the camera is greater than or equal to a preset ratio threshold, the electronic device can enter the moon shooting mode; if the zoom ratio of the camera is less than the preset ratio threshold, the electronic device continues to display the first preview image without processing the first preview image. Of course, the electronic device can also enter the moon shooting mode when the zoom ratio of the camera is not greater than the preset ratio threshold, and the embodiments of the present application do not make specific limitations thereto.

[0192] It should be noted that in the case where the electronic device enters the moon shooting mode or the sunrise / sunset mode, the current shooting mode can be prompted to the user through a prompt information in the shooting interface. The prompt information can be represented in at least one of the forms of text, pattern, control, etc.

[0193] As an example, in the case where the celestial body in the first preview image is the moon, the electronic device can obtain a super-resolution model corresponding to the current phase of the moon. The first preview image is processed through the super-resolution model to obtain a second preview image.

[0194] It should be noted that the super-resolution model can improve the resolution of the image and restore the details and textures of the image, and therefore the electronic device can obtain the super-resolution model. Of course, in order to improve the efficiency of image processing and improve the quality of the processed image, different super-resolution models can be used for different lunar phase images.

[0195] It should be noted that the resolution of the second preview image is greater than that of the first preview image after the first preview image is processed by the super-resolution model.

[0196] In some embodiments, the super-resolution model can process the entire first preview image to improve the overall resolution of the first preview image. Of course, the super-resolution model can also process the region of the first preview image where the moon is located to improve the resolution of the moon region in the first preview image. It should be noted that the moon region refers to the region of the first preview image where the moon is displayed.

[0197] For example, when the first lunar phase state is a new moon, the electronic device can obtain a super-resolution model corresponding to the new moon image. The super-resolution model of the new moon image can be used to improve the resolution of the region where the moon is located in the new moon image, that is, to improve the resolution of the region where the moon is located in the first preview image. When the first lunar phase state is a crescent moon, the electronic device can obtain a super-resolution model corresponding to the crescent moon image.

[0198] It should be noted that the first preview image is processed by the super-resolution model corresponding to the current lunar phase state, thereby more specifically improving the resolution of the image, not only ensuring the clarity of the moon in the image, but also ensuring the clarity of other objects in the image.

[0199] As an example, when the celestial body in the first preview image is the moon, the electronic device can also update the first preview image to the second preview image by other means, for example, the electronic device can reduce the exposure and / or exposure time of the first preview image to obtain the second preview image.

[0200] Since the brightness of the moon is large when the camera of the electronic device captures the first preview image, the moon region in the first preview image is displayed fuzzy. Generally, an image with low exposure and / or short exposure time is clearer when displaying an image region with strong light. Therefore, in order to obtain a clear moon image, the electronic device can obtain the second preview image by reducing the exposure and / or exposure time of the first preview image.

[0201] Because the moon's light is strong in the first preview image while the light on other objects is weak, reducing the exposure and / or exposure time of the first preview image results in a very clear moon in the second preview image, but other objects may not be imaged or may be blurry. Therefore, the electronic device can further reduce the exposure and / or exposure time of the moon region in the first preview image to ensure the clarity of the second preview image.

[0202] In some embodiments, when displaying a second preview image, if the electronic device receives a shooting operation, the electronic device responds to the shooting operation by exposing the second preview image to obtain a target image, which is an image of the second preview image.

[0203] If the celestial body in the currently displayed preview image is the moon, and the user is satisfied with the captured image, the user can trigger a shooting operation. This shooting operation can be performed by the user clicking the shooting control in photo mode, or by the user clicking the recording control in video mode, etc. Upon receiving the shooting operation, the electronic device can expose the second preview image to obtain the target image.

[0204] In some embodiments, in response to a shooting operation, the electronic device can acquire a second preview image in raw format (raw format is a photo format or an image format), convert the second preview image in raw format to an image in RGB format (RGB format is a photo format or an image format), detect the position of the moon in the second preview image, and perform dynamic range correction (DRC) processing on the second preview image to obtain the target image.

[0205] In some embodiments, if the celestial body in the first preview image is the sun, the electronic device can enhance the first preview image and adjust the color temperature and hue of the preview image to obtain a second preview image. For example, the color temperature and hue can be adjusted towards a warmer tone.

[0206] In some embodiments, when the first coordinate information and the second coordinate information are the same, the electronic device can also display a shooting mode recommendation interface based on the type of celestial body in the first preview image. This shooting mode recommendation interface is used to prompt the user whether to enter the shooting mode recommended by the electronic device. For example, if the celestial body in the first preview image is the moon, the user is prompted whether to enter the moon-viewing shooting mode; or, if the celestial body in the first preview image is the sun, the user is prompted whether to enter the sunrise / sunset mode. Upon receiving a touch operation to enter the recommended shooting mode, the electronic device enters the recommended shooting mode.

[0207] It should be noted that, to facilitate understanding of the operations in steps 604-611 above, this application embodiment provides a schematic diagram of a celestial body detection algorithm, as shown in Figure 9. Simply put, a magnetic sensor can be used to determine the first azimuth angle of the camera, a gravity sensor can be used to determine the first elevation angle of the camera, system time is used to determine the third coordinate, and the positioning device and system time can be used to determine the fourth coordinate. The third and fourth coordinates can determine the elevation and azimuth angles of the celestial body. The azimuth angle difference is determined based on the first azimuth angle and the azimuth angle of the celestial body; the elevation angle difference is determined based on the first elevation angle and the elevation angle of the celestial body; the field of view angle of the camera is determined based on the zoom ratio of the electronic device's camera; the second coordinate information of the celestial body in the shooting interface can be determined based on the field of view angle, the elevation angle difference, and the azimuth angle difference; the AI ​​model can identify the first coordinate information of the celestial body identified in the first preview image; it is determined whether the first coordinate information and the second coordinate information are consistent; if they are inconsistent, the algorithm ends; if they are consistent, at least one operation is performed, such as mode recommendation, image quality enhancement, or super-resolution fusion.

[0208] In this embodiment of the application, by comparing the celestial positions identified in the preview image with the actual display positions of celestial bodies in the preview image, it is possible to further determine whether there are celestial bodies in the preview image, thereby improving the accuracy of image processing.

[0209] It should be noted that during the shooting process using an electronic device, the camera may shake. To achieve image stabilization, the electronic device can perform frame stabilization based on magnetic data. Referring to Figure 10, this process may include the following steps:

[0210] Step 1001: Continuously capture multiple frames of preview images using a camera.

[0211] It should be noted that the process of a camera capturing images is dynamic, meaning that the camera can continuously capture multiple frames of preview images.

[0212] Step 1002: During the acquisition of each frame of preview image, the gyroscope data corresponding to each frame of preview image is acquired through the gyroscope sensor, and the second magnetic data corresponding to each frame of preview image is acquired through the magnetic sensor.

[0213] In other words, for each frame of preview image captured, the electronic device will collect gyroscope data once through the gyroscope sensor and magnetic data once through the magnetometer to obtain the second magnetic data.

[0214] Step 1003: For each frame of preview image acquired by the electronic device, the magnetic weight queue is updated based on the gyroscope data corresponding to each frame of preview image.

[0215] It should be noted that the magnetic force weight queue includes the weight corresponding to the magnetic force data corresponding to each frame of the preview image.

[0216] In some embodiments, the electronic device updates the magnetic force weight queue according to the gyroscope data corresponding to each frame of the preview image, including: in a case where the collected preview image is the first frame of the preview image, determining that the weight corresponding to the first frame of the preview image is 1; in a case where the collected preview image is not the first frame of the preview image, determining the weight corresponding to the currently collected preview image according to the gyroscope data corresponding to the currently collected preview image and the weight corresponding to the collected last frame of the preview image; and storing the weight corresponding to the currently collected preview image and the magnetic force data correspondingly to the magnetic force weight queue.

[0217] In some embodiments, in a case where the collected preview image is not the first frame of the preview image, the electronic device can determine the weight corresponding to the currently collected preview image according to the gyroscope data corresponding to the currently collected preview image and the weight corresponding to the collected last frame of the preview image through the following eighth formula (8). The eighth formula can be:

[0218] It should be noted that in the above eighth formula (8), W i (t) is the weight corresponding to the preview image collected at the current time t, W i (t-1) is the weight corresponding to the last collected frame of the preview image, g(t) is the gyroscope data size corresponding to the current time t, and τ is a hyperparameter for adjusting the size of the influence of the gyroscope data on the weight update. When g(t) is very small, such as close to 0, it indicates that the electronic device is not shaken much, and at this time close to 1, which means that the weight W i (t) is not attenuated much.

[0219] In some embodiments, the electronic device can also determine the weight corresponding to the currently collected preview image according to the gyroscope data corresponding to the currently collected preview image and the weight corresponding to the collected last frame of the preview image through other manners, such as: in a case where the gyroscope data corresponding to the currently collected preview image is the same as the gyroscope data corresponding to the last frame of the preview image, determining that the weight corresponding to the last frame of the preview image is the weight corresponding to the current preview image. In a case where the gyroscope data corresponding to the currently collected preview image is not the same as the gyroscope data corresponding to the last frame of the preview image, the weight corresponding to the currently collected preview image is determined through the above eighth formula (8). Of course, it can also be determined through other manners, and the embodiments of the present application do not make specific limitations in this regard.

[0220] In some embodiments, after updating the magnetic force weight queue, the electronic device can determine the average magnetic force data according to the second magnetic force data corresponding to each frame of preview image and the weight corresponding to each frame of preview image in the magnetic force weight queue, to obtain the first magnetic force data. The operation can include the operations of steps 1004-100X.

[0221] Step 1004: Determine whether the queue length in the magnetic force weight queue is greater than or equal to the preset length, if yes, perform the operation of step 1005 below, if not, perform the operation of step 1006 below.

[0222] It should be noted that the preset length can be set in advance according to the demand, for example, the preset length can be 10, 20, etc., and the embodiments of the present application do not limit this.

[0223] Step 1005: Delete the second magnetic force data and the corresponding weight that are first added to the magnetic force weight queue, so that the length of the magnetic force weight queue is less than the preset length, and perform the operation of step 1006 below.

[0224] Step 1006: Determine the average magnetic force data of the magnetic force data in the magnetic force weight queue to obtain the first magnetic force data.

[0225] In some embodiments, the electronic device can determine the average magnetic force data of the magnetic force data in the magnetic force weight queue to obtain the first magnetic force data through the following ninth formula (9). The ninth formula (9) can be:

[0226] It should be noted that in the above ninth formula (9), N is the number of magnetic force data in the magnetic force weight queue, M j is the magnetic force data corresponding to the i-th frame of preview image, W i (t) is the weight corresponding to the i-th frame of preview image.

[0227] In some embodiments, the operation of determining the average magnetic force data according to the second magnetic force data corresponding to each frame of preview image and the weight corresponding to each frame of preview image in the magnetic force weight queue to obtain the first magnetic force data can not only include the operations of steps 1004-1006 above, but also include other operations. For example, the electronic device can randomly select M groups of magnetic force data and corresponding weights from the magnetic force weight queue, or select the M groups of magnetic force data and corresponding weights that are added later in the magnetic force weight queue, and then determine the average magnetic force data of the M groups of magnetic force data according to the above ninth formula to obtain the first magnetic force data. Wherein, M is an integer greater than 1 and less than or equal to the preset length.

[0228] In the embodiments of the present application, the fluctuation of the magnetic force data collected by the magnetic force sensor can be relatively large. In order to obtain accurate magnetic force data, the electronic device performs frame stabilization operation by using the gyroscope data collected by the gyroscope sensor, thereby reducing the influence of the shaking of the electronic device on the collected magnetic force data.

[0229] In some embodiments, the electronic device can not only perform frame stabilization operation according to the magnetic force data, but also determine whether the magnetic force data sensed by the magnetic force sensor is accurate. Referring to FIG. 11, the process can include the following operations:

[0230] In step 1101, the second magnetic force data collected by the magnetic force sensor is obtained.

[0231] In step 1102, the current position of the electronic device is obtained by using the positioning apparatus.

[0232] In step 1103, the geomagnetic size of the current geographic position of the electronic device is determined by using the world magnetic model according to the current position of the electronic device, and the third magnetic force data is obtained.

[0233] It should be noted that the world magnetic model is used to determine the geomagnetic size of different geographic positions, and the world magnetic model can be referred to related technologies, and the embodiments of the present application are not limited specifically.

[0234] In step 1104, the deviation value between the second magnetic force data and the third magnetic force data is determined.

[0235] In step 1105, the display size of the prediction box is determined according to the deviation value.

[0236] It should be noted that the display position of the prediction box is the position of the celestial body in the preview image.

[0237] In some embodiments, in the case that the deviation value is less than or equal to the deviation threshold value, the size of the prediction box is determined as the default size, that is, the size of the prediction box is not adjusted. In the case that the deviation value is greater than the deviation threshold value, the size of the prediction box is adjusted, and the greater the deviation value, the larger the prediction box. For example, the prediction box is adjusted to 1.1 times, 1.3 times, etc. of the default size.

[0238] In the embodiments of the present application, the magnetic force sensor is too susceptible to interference when collecting magnetic force data. Therefore, by comparing the collected magnetic force data with the theoretical magnetic force data, the confidence of the magnetic force data collected by the magnetic force sensor can be determined. At the same time, the size of the prediction box is adjusted by using the deviation value of the magnetic force data, thereby improving the display richness and reliability.

[0239] An image processing apparatus is provided in the embodiments of the present application. The apparatus can be implemented by software, hardware or combination of both as part or whole of an electronic device. The electronic device can be the electronic device shown in FIG. 1. The apparatus comprises a detection module, a determination module and an updating module.

[0240] The detection module performs celestial body detection on a first preview image collected by the camera during image collection by the camera.

[0241] The first determination module determines first coordinate information of the identified celestial body in the first preview image when the celestial body exists in the first preview image.

[0242] The first updating module updates the first preview image to a second preview image when the first coordinate information is the same as second coordinate information, the image quality of the second preview image is better than that of the first preview image, and the second coordinate information is coordinate information of the celestial body in the first preview image determined based on a plurality of sensors in the electronic device.

[0243] As an example of the present application, the apparatus further comprises:

[0244] The acquisition module acquires sensor data collected by each sensor of the plurality of sensors in the electronic device during image collection by the camera.

[0245] The second determination module determines an azimuth angle difference and an altitude angle difference between the camera and the celestial body based on the sensor data collected by each sensor.

[0246] The third determination module determines the second coordinate information based on the azimuth angle difference, the altitude angle difference and a field of view angle of the camera.

[0247] As an example of the present application, the plurality of sensors comprises a magnetic sensor, a positioning device and a gravity sensor.

[0248] The second determination module is configured to:

[0249] determine an azimuth angle and an altitude angle of the camera based on first magnetic data collected by the magnetic sensor and gravity data collected by the gravity sensor, to obtain a first azimuth angle and a first altitude angle respectively; determine a coordinate of the celestial body to obtain a third coordinate, and determine a coordinate of the electronic device to obtain a fourth coordinate; determine an altitude angle and an azimuth angle of the celestial body relative to the electronic device based on the third coordinate and the fourth coordinate, to obtain a second azimuth angle and a second altitude angle; determine an azimuth angle difference between the first azimuth angle and the second azimuth angle to obtain the azimuth angle difference between the camera and the celestial body, and determine an altitude angle difference between the first altitude angle and the second altitude angle to obtain the altitude angle difference between the camera and the celestial body.

[0250] As an example of the present application, the second determining module is configured to:

[0251] According to the first magnetic force data and the gravity data, a rotation matrix is determined, the rotation matrix being used for switching of coordinate systems; a first vector of the camera in the electronic device coordinate system is obtained; according to the rotation matrix and the first vector, a first elevation angle and a first azimuth angle are determined.

[0252] As an example of the present application, the second determining module is configured to:

[0253] The rotation matrix is multiplied by the first vector to obtain a second vector of the camera in the earth coordinate system; according to a Z-axis component of the second vector in the earth coordinate system, the first elevation angle is determined; according to an X-axis component and a Y-axis component of the second vector in the earth coordinate system, the first azimuth angle is determined.

[0254] As an example of the present application, the third determining module is configured to:

[0255] The azimuth angle difference and the elevation angle difference are converted into a plane rectangular coordinate system in which the first preview image is located; according to a field of view angle, an angle coordinate in the plane rectangular coordinate system is converted into a proportional coordinate to obtain second coordinate information.

[0256] As an example of the present application, the electronic device comprises a magnetic force sensor and a gyroscope sensor;

[0257] The device further comprises:

[0258] The first collecting module is configured to continuously collect a plurality of preview images through the camera;

[0259] The second collecting module is configured to, in the process of collecting each preview image, collect, through the gyroscope sensor, gyroscope data corresponding to each preview image, and collect, through the magnetic force sensor, second magnetic force data corresponding to each preview image;

[0260] The second updating module is configured to update, according to the gyroscope data corresponding to each preview image, a magnetic force weight queue, the magnetic force weight queue comprising a weight corresponding to the second magnetic force data corresponding to each preview image;

[0261] The fourth determining module is configured to determine, according to the second magnetic force data corresponding to each preview image and the weight corresponding to each preview image in the magnetic force weight queue, average magnetic force data to obtain the first magnetic force data.

[0262] As an example of the present application, the second updating module is configured to:

[0263] In a case that the collected preview image is the first frame preview image, the weight corresponding to the first frame preview image is determined as 1; in a case that the collected preview image is not the first frame preview image, the weight corresponding to the collected preview image is determined according to the gyroscope data corresponding to the collected preview image and the weight corresponding to the last frame preview image collected; and the weight corresponding to the collected preview image and the magnetic force data are stored in the magnetic force weight queue.

[0264] In the embodiments of the present application, by comparing the position of the identified celestial body in the preview image with the display position of the celestial body in the preview image in actual life, whether the celestial body exists in the preview image can be further determined, and the accuracy of image processing is improved.

[0265] It should be noted that the image processing apparatus provided in the above embodiments is only used for example to divide the above functions into different functional modules when performing image processing, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the above described functions.

[0266] Each functional unit and module in the above embodiments can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit, and the integrated unit can be realized in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only used for mutual distinction, and are not used to limit the protection scope of the embodiments of the present application.

[0267] The image processing apparatus and the image processing method provided in the above embodiments belong to the same concept, and the specific working process of the unit and module in the above embodiments and the resulting technical effects can be referred to the method embodiment part, which will not be repeated here.

[0268] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general purpose computer, a special purpose computer, a computer network or other programmable apparatus. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, such as the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium accessible by a computer, or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (such as floppy disk, hard disk, magnetic tape), optical media (such as digital versatile disc (DVD)) or semiconductor media (such as solid state disk (SSD)) and the like.

[0269] The above is an optional embodiment provided by the present application, and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the technical scope disclosed by the present application shall be included in the protection scope of the present application.

Claims

1. A method of processing an image, characterized by, Applied to an electronic device, the method comprises: In the process of collecting images through a camera, celestial body detection is performed on a first preview image collected by the camera; In the case where there is a celestial body in the first preview image, first coordinate information of the identified celestial body in the first preview image is determined; In the case where the first coordinate information and second coordinate information are the same, the first preview image is updated to a second preview image, the image quality of the second preview image is better than that of the first preview image, and the second coordinate information is coordinate information of the celestial body in the first preview image determined based on a plurality of sensors in the electronic device.

2. The method of claim 1, wherein, Before the case where the first coordinate information and the second coordinate information are the same, the first preview image is updated to the second preview image, it further comprises: In the process of collecting images through the camera, sensor data collected by each sensor of the plurality of sensors in the electronic device is obtained; According to the sensor data collected by each sensor, an azimuth angle difference and an elevation angle difference between the camera and the celestial body are determined; According to the azimuth angle difference, the elevation angle difference, and a field of view angle of the camera, the second coordinate information is determined.

3. The method of claim 2, wherein, The plurality of sensors comprises a magnetic sensor, a positioning device, and a gravity sensor; According to the sensor data collected by each sensor, an azimuth angle difference and an elevation angle difference between the camera and the celestial body are determined, comprising: According to first magnetic force data collected by the magnetic sensor and gravity data collected by the gravity sensor, an azimuth angle and an elevation angle of the camera are determined, respectively obtaining a first azimuth angle and a first elevation angle; The coordinates of the celestial body are determined, obtaining a third coordinate, and the coordinates of the electronic device are determined, obtaining a fourth coordinate; According to the third coordinate and the fourth coordinate, the elevation angle and the azimuth angle of the celestial body relative to the electronic device are determined, obtaining a second azimuth angle and a second elevation angle; The azimuth angle difference between the first azimuth angle and the second azimuth angle is determined, obtaining the azimuth angle difference between the camera and the celestial body, and the elevation angle difference between the first elevation angle and the second elevation angle is determined, obtaining the elevation angle difference between the camera and the celestial body.

4. The method of claim 3, wherein, According to the first magnetic force data collected by the magnetic sensor and the gravity data collected by the gravity sensor, an azimuth angle and an elevation angle of the camera are determined, respectively obtaining a first azimuth angle and a first elevation angle, comprising: According to the first magnetic force data and the gravity data, a rotation matrix is determined, the rotation matrix is used for switching coordinate systems; A first vector of the camera in the electronic device coordinate system is obtained; According to the rotation matrix and the first vector, the first elevation angle and the first azimuth angle are determined.

5. The method of claim 4, wherein, According to the rotation matrix and the first vector, the first elevation angle and the first azimuth angle are determined, comprising: The rotation matrix is multiplied by the first vector to obtain a second vector of the camera in the earth coordinate system; According to the Z-axis component of the second vector in the earth coordinate system, the first elevation angle is determined; Determine the first azimuth angle according to the X-axis component and the Y-axis component of the second vector in the earth coordinate system.

6. The method of claim 2, wherein, The determining the second coordinate information according to the azimuth angle difference, the elevation angle difference and the field of view angle of the camera comprises: Converting the azimuth angle difference and the elevation angle difference into a plane rectangular coordinate system in which the first preview image is located; Converting an angle coordinate in the plane rectangular coordinate system into a proportional coordinate according to the field of view angle to obtain the second coordinate information.

7. The method of claim 3, wherein, The electronic device comprises a magnetic force sensor and a gyroscope sensor; Before determining the azimuth angle and the elevation angle of the camera according to the first magnetic force data collected by the magnetic force sensor and the gravity data collected by the gravity sensor to obtain the first azimuth angle and the first elevation angle respectively, the method further comprises: Continuously collecting multiple preview images through the camera; In the process of collecting each preview image, collecting gyroscope data corresponding to each preview image through the gyroscope sensor and collecting second magnetic force data corresponding to each preview image through the magnetic force sensor; Updating a magnetic force weight queue according to the gyroscope data corresponding to each preview image, the magnetic force weight queue comprising a weight corresponding to the second magnetic force data corresponding to each preview image; Determining average magnetic force data according to the second magnetic force data corresponding to each preview image and the weight corresponding to each preview image in the magnetic force weight queue to obtain the first magnetic force data.

8. The method of claim 7, wherein, The updating the magnetic force weight queue according to the gyroscope data corresponding to each preview image comprises: In the case that the collected preview image is a first preview image, determining that the weight corresponding to the first preview image is 1; In the case that the collected preview image is not the first preview image, determining the weight corresponding to the currently collected preview image according to the gyroscope data corresponding to the currently collected preview image and the weight corresponding to the last collected preview image; Storing the weight corresponding to the currently collected preview image and the magnetic force data corresponding to the currently collected preview image into the magnetic force weight queue.

9. An electronic device, comprising: The structure of the electronic device comprises a processor and a memory; The memory is used to store a program supporting the electronic device to execute the method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores instructions, when the instructions are run on the computer, the computer executes the method according to any one of claims 1-8.

Images