A method, apparatus and computer-readable storage medium for rendering star trails over time.

Abstract

This invention discloses a method, device, and computer-readable storage medium for time-lapse star trail rendering. The method includes: acquiring a set of celestial bodies based on a first image frame, and determining a first position of each celestial body within the first image frame; predicting a second position of each celestial body in a second image frame following the first image frame based on device data, celestial body data, and shooting parameters; performing feature point detection on the first and second image frames to determine the first and second positions belonging to the same celestial body; and during the time-lapse star trail rendering process, selecting a preset image frame as the background image for rendering, and rendering a connecting line between the first and second positions of each celestial body on the background image. This invention avoids the impact of clouds on time-lapse star trail rendering, improves the final effect of the rendering, and enhances the user experience.

Description

Technical Field

[0001] This invention relates to the field of mobile communications, and more particularly to a method, apparatus, and computer-readable storage medium for rendering star trails over time. Background Technology

[0002] In existing technologies, with the continuous development of smart terminal devices, star trail photography has become a popular advanced shooting experience for end users. In the ordinary star trail creation process, the principle is pixel-by-pixel comparison. That is, the first image is taken as the reference image, and each pixel of the second image is compared with the corresponding pixel of the reference image. If a pixel value in the second image is brighter than the reference image, then that pixel value is used to replace the pixel value in the reference image, generating a new image. Using this new image as the reference image, each pixel of the third image is compared with the corresponding pixel of the reference image. If a pixel value in the third image is brighter than the reference image, then that pixel value is used to replace the pixel value in the reference image, generating a new image, and so on.

[0003] The above star trail drawing method can produce beautiful star trails under ideal conditions with no clouds. However, if there are clouds in the sky, this method will cause problems. Specifically, the clouds will continuously overlap onto the final generated image, eventually resulting in a white blur.

[0004] Therefore, existing star trail rendering schemes cannot effectively eliminate the impact of interference factors such as clouds on the final rendering effect, and the user experience needs to be improved. Summary of the Invention

[0005] To address the aforementioned technical deficiencies in the existing technology, this invention proposes a method for rendering star trails with time delay, the method comprising:

[0006] When entering the viewfinder for time-lapse star trail rendering, a set of celestial bodies is obtained based on the first image frame, and the first position of each celestial body in the first image frame is determined within the set of celestial bodies.

[0007] Based on device data, celestial data, and shooting parameters, predict the second position of each celestial body in the second image frame following the first image frame.

[0008] Feature point detection is performed on the first image frame and the second image frame to determine the first position and the second position belonging to the same celestial body.

[0009] During the time-lapse rendering of star trails, a preset image frame is selected as the background image for rendering, and the connecting line between the first position and the second position of each celestial body is rendered on the background image.

[0010] Optionally, when entering the viewfinder for time-lapse star trail rendering, obtaining a set of celestial bodies based on a first image frame and determining the first position of each celestial body in the first image frame within the set of celestial bodies includes:

[0011] When entering the viewfinder for star trail time-lapse rendering, acquire the initial image frame.

[0012] The initial image frame is analyzed to identify star trail drawing interference objects in the initial image frame.

[0013] Optionally, the step of obtaining a set of celestial bodies based on a first image frame and determining the first position of each celestial body in the first image frame within the set of celestial bodies when entering the viewfinder interface for time-lapse star trail rendering further includes:

[0014] If the object interfering with the star trail rendering is a cloud layer, then multiple initial image frames are continuously acquired within a preset time period.

[0015] The image frame with the fewest clouds among the initial image frames is selected as the preset image frame.

[0016] Optionally, the step of obtaining a set of celestial bodies based on a first image frame and determining the first position of each celestial body in the first image frame within the set of celestial bodies when entering the viewfinder interface for time-lapse star trail rendering further includes:

[0017] If the object interfering with the star trail rendering is the moon, then multiple initial image frames will be continuously acquired within a preset time period.

[0018] By fusing at least two image frames from multiple initial image frames, the image frame after removing the moon is used as the preset image frame.

[0019] Optionally, the step of obtaining a set of celestial bodies based on a first image frame and determining the first position of each celestial body in the first image frame within the set of celestial bodies when entering the viewfinder interface for time-lapse star trail rendering further includes:

[0020] Calculate the number of star sets in the multiple initial image frames, and select the initial image frame with the largest number of stars as the first image frame.

[0021] In the first image frame, identify celestial bodies whose brightness exceeds a threshold and determine the first position of the celestial body to be drawn in the first image frame.

[0022] Optionally, predicting the second position of each celestial body in the second image frame following the first image frame based on device data, celestial body data, and shooting parameters includes:

[0023] Obtain the exposure duration from the shooting parameters.

[0024] Based on device data, celestial data, celestial drawing parameters, and the exposure duration, predict the second position of the celestial body to be drawn in the second image frame following the first image frame.

[0025] Optionally, the step of performing feature point detection on the first image frame and the second image frame to determine the first location and the second location belonging to the same celestial body includes:

[0026] The third position of the celestial body to be drawn is determined in the second image frame.

[0027] Feature point detection is performed on the first image frame and the second image frame. In the second image frame, the celestial body corresponding to the second position that is closest to the third position is identified as the same celestial body to be drawn.

[0028] Optionally, in the process of rendering star trails over time, selecting a preset image frame as the background image for rendering, and rendering the connecting line between the first position and the second position of each celestial body on the background image, includes:

[0029] The feature point detection is performed on every two consecutive image frames.

[0030] Keeping the background image unchanged, render the connecting lines between the same celestial body to be drawn in every two consecutive image frames until the star trail delay rendering is completed.

[0031] The present invention also proposes a star trail time-lapse rendering device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the star trail time-lapse rendering method as described in any of the preceding claims.

[0032] The present invention also proposes a computer-readable storage medium storing a star trail time-lapse rendering program, which, when executed by a processor, implements the steps of the star trail time-lapse rendering method as described in any of the preceding claims.

[0033] The star trail time-lapse rendering method, device, and computer-readable storage medium of the present invention, upon entering the viewfinder interface for star trail time-lapse rendering, acquires a set of celestial bodies based on a first image frame and determines the first position of each celestial body in the first image frame within the set of celestial bodies; predicts the second position of each celestial body in a second image frame following the first image frame based on device data, celestial body data, and shooting parameters; performs feature point detection on the first and second image frames to determine the first and second positions belonging to the same celestial body; during the star trail time-lapse rendering process, selects a preset image frame as the background image for rendering, and renders the connecting line between the first and second positions of each celestial body on the background image. This achieves a user-friendly star trail time-lapse rendering scheme, avoids the impact of clouds on star trail time-lapse rendering, improves the final effect of star trail time-lapse rendering, and enhances the user experience. Attached Figure Description

[0034] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0035] Figure 1 This is a schematic diagram of the hardware structure of a mobile terminal according to the present invention;

[0036] Figure 2 This is a communication network system architecture diagram provided in an embodiment of the present invention;

[0037] Figure 3 This is a flowchart of the first embodiment of the star trail delay rendering method of the present invention;

[0038] Figure 4 This is a flowchart of the second embodiment of the star trail delay rendering method of the present invention;

[0039] Figure 5 This is a flowchart of the third embodiment of the star trail delay rendering method of the present invention;

[0040] Figure 6 This is a flowchart of the fourth embodiment of the star trail delay rendering method of the present invention;

[0041] Figure 7 This is a flowchart of the fifth embodiment of the star trail delay rendering method of the present invention;

[0042] Figure 8 This is a flowchart of the sixth embodiment of the star trail delay rendering method of the present invention;

[0043] Figure 9 This is a flowchart of the seventh embodiment of the star trail delay rendering method of the present invention;

[0044] Figure 10 This is a flowchart of the eighth embodiment of the star trail delay rendering method of the present invention;

[0045] Figure 11 This is a schematic diagram of the correspondence between the positions of stars in the first embodiment of the star trail delay rendering method of the present invention;

[0046] Figure 12 This is a comparison image of cloud processing effects in the first embodiment of the star trail time-lapse rendering method of the present invention. Detailed Implementation

[0047] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0048] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0049] Terminals can be implemented in various forms. For example, the terminals described in this invention may include mobile terminals such as mobile phones, tablets, laptops, handheld computers, personal digital assistants (PDAs), portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, etc., as well as fixed terminals such as digital TVs and desktop computers.

[0050] The following description will use a mobile terminal as an example. Those skilled in the art will understand that, apart from elements specifically designed for mobile purposes, the construction according to embodiments of the present invention can also be applied to fixed-type terminals.

[0051] Please see Figure 1 This is a schematic diagram of the hardware structure of a mobile terminal implementing various embodiments of the present invention. The mobile terminal 100 may include: an RF (Radio Frequency) unit 101, a WiFi module 102, an audio output unit 103, an A / V (Audio / Video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111, etc. Those skilled in the art will understand that... Figure 1 The mobile terminal structure shown does not constitute a limitation on the mobile terminal. The mobile terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0052] The following is combined Figure 1 A detailed introduction to each component of the mobile terminal:

[0053] The radio frequency unit 101 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 110; additionally, it transmits uplink data to the base station. Typically, the radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, and a duplexer. Furthermore, the radio frequency unit 101 can also communicate wirelessly with networks and other devices. The aforementioned wireless communications may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), and TDD-LTE (Time Division Duplexing-Long Term Evolution).

[0054] WiFi is a short-range wireless transmission technology. Mobile terminals using the WiFi module 102 can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 1 WiFi module 102 is shown, but it is understood that it is not a necessary component of a mobile terminal and can be omitted as needed without changing the nature of the invention.

[0055] The audio output unit 103 can convert audio data received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109 into audio signals and output them as sound when the mobile terminal 100 is in call signal receiving mode, call mode, recording mode, voice recognition mode, broadcast receiving mode, etc. Furthermore, the audio output unit 103 can also provide audio output related to specific functions performed by the mobile terminal 100 (e.g., call signal receiving sound, message receiving sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0056] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos acquired by an image capture device (such as a camera) in video capture mode or image capture mode. The processed image frames can be displayed on the display unit 106. The image frames processed by the GPU 1041 can be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) in operating modes such as telephone call mode, recording mode, and voice recognition mode, and can process such sound into audio data. The processed audio (voice) data can be converted into a format that can be transmitted to a mobile communication base station via the radio frequency unit 101 in telephone call mode. The microphone 1042 can implement various types of noise cancellation (or suppression) algorithms to eliminate (or suppress) noise or interference generated during the reception and transmission of audio signals.

[0057] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 1061 according to the ambient light level, and the proximity sensor can turn off the display panel 1061 and / or backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, tapping), etc. Other sensors that may be configured in the phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0058] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0059] User input unit 107 can be used to receive input numerical or character information, and generate key signal inputs related to user settings and function control of the mobile terminal. Specifically, user input unit 107 may include touch panel 1071 and other input devices 1072. Touch panel 1071, also known as touch screen, can collect touch operations on or near the user (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near touch panel 1071), and drive corresponding connection devices according to a pre-set program. Touch panel 1071 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to processor 110, and can receive and execute commands from processor 110. In addition, touch panel 1071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may also include other input devices 1072. Specifically, other input devices 1072 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc., without being limited here.

[0060] Furthermore, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. Subsequently, the processor 110 provides corresponding visual output on the display panel 1061 based on the type of touch event. Although in Figure 1 In this embodiment, the touch panel 1071 and the display panel 1061 are two independent components to realize the input and output functions of the mobile terminal. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to realize the input and output functions of the mobile terminal. The specific implementation is not limited here.

[0061] Interface unit 108 serves as an interface through which at least one external device can connect to mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, and so on. Interface unit 108 may be used to receive input (e.g., data, power, etc.) from the external device and transmit the received input to one or more elements within mobile terminal 100, or it may be used to transmit data between mobile terminal 100 and the external device.

[0062] The memory 109 can be used to store software programs and various data. The memory 109 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback, image playback, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory 109 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0063] The processor 110 is the control center of the mobile terminal. It connects various parts of the mobile terminal via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 109, and by calling data stored in the memory 109, it performs various functions and processes data of the mobile terminal, thereby providing overall monitoring of the mobile terminal. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 110.

[0064] The mobile terminal 100 may also include a power supply 111 (such as a battery) that supplies power to various components. Preferably, the power supply 111 can be logically connected to the processor 110 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0065] although Figure 1 As not shown, the mobile terminal 100 may also include a Bluetooth module, etc., which will not be described in detail here.

[0066] To facilitate understanding of the embodiments of the present invention, the communication network system on which the mobile terminal of the present invention is based is described below.

[0067] Please see Figure 2 , Figure 2 This invention provides a communication network system architecture diagram. The communication network system is an LTE system based on the universal mobile communication technology. The LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203, and the operator's IP services 204, which are connected in sequence.

[0068] Specifically, UE201 can be the aforementioned terminal 100, which will not be elaborated here.

[0069] E-UTRAN202 includes eNodeB2021 and other eNodeB2022s. Among them, eNodeB2021 can connect to other eNodeB2022s via backhaul (e.g., X2 interface), and eNodeB2021 connects to EPC203. eNodeB2021 can provide UE201 with access to EPC203.

[0070] EPC203 may include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving Gateway) 2034, PGW (Packet Data Network Gateway) 2035, and PCRF (Policy and Charging Rules Function) 2036, etc. Among them, MME2031 is the control node that handles signaling between UE201 and EPC203, providing bearer and connection management. HSS2032 provides registers to manage functions such as the Home Location Register (not shown in the diagram) and stores user-specific information such as service characteristics and data rates. All user data can be sent through SGW2034. PGW2035 can provide UE 201 IP address allocation and other functions. PCRF2036 is the policy and charging control decision point for service data flow and IP bearer resources. It selects and provides available policy and charging control decisions for the policy and charging enforcement function unit (not shown in the figure).

[0071] IP services 204 may include the Internet, intranet, IMS (IP Multimedia Subsystem), or other IP services.

[0072] Although the above description uses the LTE system as an example, those skilled in the art should understand that the present invention is not only applicable to the LTE system, but also to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, and future new network systems, etc., which are not limited here.

[0073] Based on the aforementioned mobile terminal hardware structure and communication network system, various embodiments of the method of the present invention are proposed.

[0074] Example 1

[0075] Figure 3 This is a flowchart of the first embodiment of the star trail time-lapse rendering method of the present invention. A star trail time-lapse rendering method, the method comprising:

[0076] S1. When entering the viewfinder for time-lapse star trail rendering, obtain the set of stars based on the first image frame, and determine the first position of each celestial body in the first image frame within the set of stars.

[0077] S2. Based on the device data, celestial data, and shooting parameters, predict the second position of each celestial body in the second image frame following the first image frame.

[0078] S3. Perform feature point detection on the first image frame and the second image frame to determine the first position and the second position belonging to the same celestial body.

[0079] S4. During the star trail time-lapse rendering process, a preset image frame is selected as the background image for rendering, and the connecting line between the first position and the second position of each celestial body is rendered on the background image.

[0080] Optionally, in this embodiment, when entering the viewfinder for time-lapse star trail rendering, a set of celestial bodies is obtained based on a first image frame, and the first position of each celestial body in the first image frame is determined within the set of celestial bodies. This first position is the actual position of each celestial body in the first image frame. Specifically, a list of celestial bodies within the viewfinder is obtained based on astronomical data and device data, and the list is sorted according to the star density parameter in the celestial body data to obtain the set of celestial bodies. The star rendering parameters are set, consisting of star rendering brightness parameters, star rendering color parameters, and star rendering size parameters.

[0081] Optionally, in this embodiment, the second position of each celestial body in a second image frame following the first image frame is predicted based on device data, celestial body data, and imaging parameters. Specifically, the predicted second position in the second image frame is drawn based on the aforementioned celestial body rendering parameters.

[0082] Optionally, in this embodiment, feature point detection is performed on the first image frame and the second image frame to determine the first location and the second location belonging to the same celestial body. Please refer to... Figure 11The prediction diagram shown demonstrates feature point detection for each image. The point in the right image closest to the predicted position is identified as the corresponding star, which is the same star as star 1 in the left image. Specifically, Fast feature point detection is used to detect stars in the image. Fast feature point detection is a commonly used feature point method in images. The principle is as follows: A circle with a radius of 3 centered at pixel p contains 16 pixels (p1, p2, ..., p16). A threshold is defined. The pixel differences between p1, p9, and the center p are calculated. If their absolute values ​​are all less than the threshold, p cannot be a feature point and is directly discarded; otherwise, it is considered a candidate point for further investigation. If p is a candidate point, the pixel differences between p1, p9, p5, p13, and the center p are calculated. If at least three of their absolute values ​​exceed the threshold, they are considered candidate points for further investigation; otherwise, they are directly discarded. If p is a candidate point, calculate the pixel differences between the 16 points p1 to p16 and the center p. If at least 9 of them exceed the threshold, they are considered feature points; otherwise, they are directly discarded. Non-maximum suppression is applied to the image: Calculate the FAST score (score) of the feature point. Within a neighborhood (e.g., 3x3 or 5x5) centered on feature point p, calculate the s value (the sum of the absolute values ​​of the differences between the 16 points and the center) for each feature point if there are multiple feature points. If p has the largest response value among all feature points in the neighborhood, it is retained; otherwise, it is suppressed. If there is only one feature point (corner point) in the neighborhood, it is retained. The score calculation formula is as follows (V represents the score, and t represents the threshold):

[0083]

[0084] Optionally, in this embodiment, during the star trail time-lapse rendering process, a preset image frame is selected as the background image for rendering, and the connecting line between the first position and the second position of each celestial body is rendered on the background image. Please refer to... Figure 12 The comparison images shown show two traditional pixel matching schemes. In this scheme, if a pixel in a later frame is brighter than one in a previous frame, that pixel is replaced with the brighter one. The resulting image is a global overlay, where stars are connected, but clouds are also superimposed. The two images below, however, are based on star detection using feature points and star trail prediction. This allows for a clear understanding of star positions and the fact that a particular star in a later frame is the same as one in a previous frame. Therefore, in this embodiment, OpenGL can be used to directly render a line between these two coordinates, leaving the rest of the image unchanged. Consequently, clouds are not superimposed because only the star connection operation is performed. It can be seen that in this embodiment, the final generated image contains only the content of the current frame, except for the star connection operation, and does not superimpose other bright spots.

[0085] The beneficial effects of this embodiment are as follows: upon entering the viewfinder interface for star trail time-lapse rendering, a set of celestial bodies is acquired based on a first image frame, and the first position of each celestial body in the first image frame is determined within the set of celestial bodies; the second position of each celestial body in a second image frame following the first image frame is predicted based on device data, celestial body data, and shooting parameters; feature point detection is performed on the first and second image frames to determine the first and second positions belonging to the same celestial body; during the star trail time-lapse rendering process, a preset image frame is selected as the background image for rendering, and a connecting line between the first and second positions of each celestial body is rendered on the background image. This achieves a user-friendly star trail time-lapse rendering scheme, avoids the impact of clouds on star trail time-lapse rendering, improves the final effect of star trail time-lapse rendering, and enhances the user experience.

[0086] Example 2

[0087] Figure 4 This is a flowchart of a second embodiment of the star trail time-lapse rendering method of the present invention. Based on the above embodiment, the step of obtaining a set of celestial bodies according to a first image frame when entering the viewfinder interface for star trail time-lapse rendering, and determining the first position of each celestial body in the first image frame within the set of celestial bodies, includes:

[0088] S11. When entering the viewfinder for star trail time-lapse rendering, acquire the initial image frame.

[0089] S12. Analyze the initial image frame and identify star trail drawing interference objects in the initial image frame.

[0090] Optionally, in this embodiment, after entering the viewfinder interface for star trail time-lapse rendering, the initial image frame is acquired when the device stabilizes within a preset time.

[0091] Optionally, in this embodiment, the initial image frame is parsed to identify star trail rendering interference objects in the initial image frame, wherein the star trail rendering interference objects include streetlights, auroras, clouds, the moon, and other relatively bright objects within the field of view.

[0092] The beneficial effect of this embodiment is that it acquires an initial image frame when entering the viewfinder interface for star trail time-lapse rendering; it then analyzes the initial image frame to identify interference objects in the star trail rendering. This provides a method for detecting interference objects to achieve a user-friendly star trail time-lapse rendering scheme, avoiding the impact of clouds on star trail time-lapse rendering, improving the final effect of star trail time-lapse rendering, and enhancing the user experience.

[0093] Example 3

[0094] Figure 5This is a flowchart of the third embodiment of the star trail time-lapse rendering method of the present invention. Based on the above embodiment, the step of obtaining a set of celestial bodies according to a first image frame when entering the viewfinder interface for star trail time-lapse rendering, and determining the first position of each celestial body in the first image frame within the set of celestial bodies, further includes:

[0095] S13. If the object interfering with the star trail drawing is a cloud layer, then multiple initial image frames are continuously acquired within a preset time period.

[0096] S14. Select the image frame with the fewest clouds from the multiple initial image frames as the preset image frame.

[0097] Optionally, in this embodiment, if the object interfering with the star trail mapping is a cloud layer, multiple initial image frames are acquired continuously within a preset time period. For example, during a stable period, multiple initial image frames are acquired at intervals and continuously.

[0098] Optionally, in this embodiment, considering that the cloud layer is dynamically changing, the image frame with the fewest clouds is selected from the multiple initial image frames as the preset image frame.

[0099] Optionally, in this embodiment, when selecting the image frame with the fewest clouds from among the multiple initial image frames as the preset image frame, if the proportion of clouds exceeds a preset value, the coverage area of ​​the cloud layer is reduced.

[0100] The beneficial effect of this embodiment is that, by identifying clouds as the interfering object in star trail rendering, multiple initial image frames are continuously acquired within a preset time period; the image frame with the fewest clouds is selected from these initial image frames as the preset image frame. This provides a cloud reduction method for achieving a user-friendly star trail time-lapse rendering scheme, avoiding the impact of clouds on star trail time-lapse rendering, improving the final effect of star trail time-lapse rendering, and enhancing the user experience.

[0101] Example 4

[0102] Figure 6 This is a flowchart of the fourth embodiment of the star trail time-lapse rendering method of the present invention. Based on the above embodiment, the step of obtaining a set of celestial bodies according to a first image frame when entering the viewfinder interface for star trail time-lapse rendering, and determining the first position of each celestial body in the first image frame within the set of celestial bodies, further includes:

[0103] S15. If the object interfering with the star trail drawing is the moon, then multiple initial image frames are continuously acquired within a preset time period.

[0104] S16. By fusing at least two image frames from multiple initial image frames, the image frame after removing the moon is used as the preset image frame.

[0105] Optionally, in this embodiment, if the object interfering with the star trail rendering is the moon or other dynamic highlighted objects, multiple initial image frames are continuously acquired within a preset time period.

[0106] Optionally, in this embodiment, by fusing at least two image frames from multiple initial image frames, the image frame after removing the moon or other highlighted dynamic objects is used as the preset image frame.

[0107] The beneficial effect of this embodiment is that, by identifying the moon as the interfering object in the star trail rendering, multiple initial image frames are continuously acquired within a preset time period; by fusing at least two image frames from the multiple initial image frames, the image frame after removing the moon is used as the preset image frame. This provides a dynamic highlighting object elimination method for achieving a user-friendly star trail time-lapse rendering scheme, avoiding the impact of clouds on star trail time-lapse rendering, improving the final effect of star trail time-lapse rendering, and enhancing the user experience.

[0108] Example 5

[0109] Figure 7 This is a flowchart of the fifth embodiment of the star trail time-lapse rendering method of the present invention. Based on the above embodiment, the step of obtaining a set of celestial bodies according to a first image frame when entering the viewfinder interface for star trail time-lapse rendering, and determining the first position of each celestial body in the first image frame within the set of celestial bodies, further includes:

[0110] S17. Calculate the number of star sets in the multiple initial image frames, and select the initial image frame with the largest number as the first image frame.

[0111] S18. Among the stars in the first image frame, determine the celestial body to be drawn whose brightness exceeds the threshold, and determine the first position of the star to be drawn in the first image frame.

[0112] Optionally, in this embodiment, in order to draw more star trails, the number of star sets in multiple initial image frames is calculated, and the initial image frame with the most stars is selected as the first image frame.

[0113] Optionally, in this embodiment, in order to make the drawn star trails clear and not complicated, the celestial bodies to be drawn whose brightness exceeds the threshold are identified among the stars in the first image frame, and the first position of the star to be drawn in the first image frame is determined.

[0114] The beneficial effect of this embodiment is that, by calculating the number of star sets in multiple initial image frames, the initial image frame with the largest number is selected as the first image frame; among the stars in the first image frame, celestial bodies whose brightness exceeds a threshold are identified as to be drawn, and the first position of the star to be drawn in the first image frame is determined. This provides star trail drawing filtering conditions for a user-friendly star trail time-lapse drawing scheme, avoids the impact of clouds on star trail time-lapse drawing, improves the final effect of star trail time-lapse drawing, and enhances the user experience.

[0115] Example 6

[0116] Figure 8 This is a flowchart of the sixth embodiment of the star trail time-lapse rendering method of the present invention. Based on the above embodiment, the step of predicting the second position of each celestial body in the second image frame after the first image frame according to device data, celestial body data, and shooting parameters includes:

[0117] S21. Obtain the exposure duration from the shooting parameters.

[0118] S22. Based on the device data, celestial data, celestial drawing parameters, and the exposure duration, predict the second position of the celestial body to be drawn in the second image frame after the first image frame.

[0119] Optionally, in this embodiment, the device data is acquired, wherein the device data includes time data, GPS data, WiFi data, gravity sensor data, and magnetic sensor data.

[0120] Optionally, in this embodiment, the current location of the device is obtained based on the GPS data or the WiFi data, and the shooting angle of the device is obtained based on the gravity sensor data and the magnetic sensor data.

[0121] Optionally, in this embodiment, the longitude and latitude of the current location of the mobile terminal device are obtained based on GPS data or WIFI data, and are denoted as Position(latitude, longitude).

[0122] Optionally, in this embodiment, the current shooting angle α of the mobile terminal device is obtained based on data from sensors such as gravity sensor data and magnetic sensor data.

[0123] Optionally, in this embodiment, astronomical data contained in the celestial data are obtained, including Earth rotation angle data and star catalog data.

[0124] Optionally, in this embodiment, the rotation angle of the Earth at the current time is obtained based on the time data and the Earth rotation angle data, and the total number of celestial bodies, their encoding information, name information, position information, brightness information, color information, apparent size information, right ascension information, and declination information are obtained based on the star catalog data.

[0125] Optionally, in this embodiment, the current rotation angle θ of the Earth is obtained based on the current time, etc.

[0126] Optionally, in this embodiment, star catalog data (such as the Hubble Telescope guide catalog) is acquired, and the star catalog data should include, but is not limited to, the following information:

[0127] StarList = {S1, S2, S3, ..., S} n};

[0128] S i ={index,name,location,Vmag,Bmag,VisualAngle,……};

[0129] location i ={RA,Dec};

[0130] Where n is the total number of celestial bodies recorded in the star catalog, and index, name, location, Vmag, Bmag, and VisualAngle are the celestial bodies S, respectively. i The encoding information, name information, location information, brightness information, color information, apparent size information, etc.; RA and Dec are for celestial body S. i Location information i The information on right ascension and declination.

[0131] Optionally, in this embodiment, a list of celestial bodies within the viewfinder is obtained based on the celestial body's position information, the device's current position, the device's shooting angle, the device's field of view, and the device's zoom level.

[0132] Optionally, in this embodiment, the celestial bodies in the celestial body list are sorted by brightness according to the brightness information, and the set of stars composed of displayable celestial bodies is obtained according to the star density parameter.

[0133] Optionally, in this embodiment, for each celestial body in the star catalog data: based on its location data, the current position and angle α of the mobile terminal device, the field of view (FOV) of the lens, the digital zoom factor (f), etc., it is calculated whether the celestial body is within the current viewfinder.

[0134] Optionally, in this embodiment, the list of celestial bodies in the current viewfinder is returned:

[0135] S inside ={S1,S2,S3,……,S m}

[0136] Where m represents the number of celestial bodies in the current viewfinder.

[0137] Optionally, in this embodiment, the star density parameter Tsparse is obtained; this parameter can be a fixed preset parameter, or a dynamically changing parameter based on the user's shooting time, region, weather conditions, etc., or a parameter set by the user; wherein: Tsparse∈[0,1].

[0138] Optionally, in this embodiment, for S inside The celestial bodies within the range are sorted according to their brightness parameter Vmag.

[0139] Optionally, in this embodiment, the number of stars that can be displayed is calculated as: m′=m*Tsparse.

[0140] Optionally, in this embodiment, after determining the number of displayable stars, the set S′ of stars within the sorted m′ is obtained. inside ={S1,S2,S3,……,S m，}

[0141] Optionally, in this embodiment, the star drawing brightness parameter, the star drawing color parameter, and the star drawing size parameter are set according to one or more of the following: fixed brightness, random brightness, fixed color, random color, fixed size, random size, the position information, the brightness information, the color information, and the apparent size information.

[0142] Optionally, in this embodiment, the brightness parameters of the celestial body are calculated according to predetermined rules; the calculation rules include, but are not limited to: fixed brightness, random brightness, brightness calculated from location and Vmag, etc.

[0143] Optionally, in this embodiment, the color parameters of the celestial body are calculated according to predetermined rules; the calculation rules include, but are not limited to: fixed color, random color, color calculated by location and Bmag, etc.

[0144] Optionally, in this embodiment, the size parameters of the celestial body are calculated according to predetermined rules; the calculation rules include, but are not limited to: fixed size, random size, size calculated by location, VisualAngle, etc.

[0145] The beneficial effect of this embodiment is that by obtaining the exposure time from the shooting parameters, and based on device data, celestial data, celestial drawing parameters, and the exposure time, the second position of the celestial body to be drawn in the second image frame after the first image frame is predicted. This provides a predictive method for determining the second position to achieve a user-friendly star trail time-lapse drawing scheme, avoiding the impact of clouds on star trail time-lapse drawing, improving the final image quality of star trail time-lapse drawing, and enhancing the user experience.

[0146] Example 7

[0147] Figure 9 This is a flowchart of the seventh embodiment of the star trail time-lapse rendering method of the present invention. Based on the above embodiment, the step of performing feature point detection on the first image frame and the second image frame to determine the first position and the second position belonging to the same celestial body includes:

[0148] S31. Determine the third position of the celestial body to be drawn in the second image frame.

[0149] S32. Perform feature point detection on the first image frame and the second image frame, and determine the celestial body corresponding to the second position that is closest to the third position in the second image frame as the same celestial body to be drawn.

[0150] Optionally, in this embodiment, a third position of the celestial body to be drawn is determined in the second image frame. This third position is the actual image position of the celestial body to be drawn in the second image frame.

[0151] Optionally, in this embodiment, feature point detection is performed on the first image frame and the second image frame. In the second image frame, the celestial body corresponding to the second position that is closest to the third position is determined as the same celestial body to be drawn. It can be understood that in this embodiment, the corresponding position of each celestial body to be drawn in the two consecutive image frames is obtained according to the above scheme.

[0152] The beneficial effect of this embodiment is that by determining the third position of the celestial body to be drawn in the second image frame, and performing feature point detection on the first and second image frames, the celestial body corresponding to the second position closest to the third position in the second image frame is identified as the same celestial body to be drawn. This provides a solution for a user-friendly star trail time-lapse rendering scheme, identifying the corresponding positions of two consecutive frames belonging to the same celestial body, avoiding the impact of clouds on star trail time-lapse rendering, improving the final effect of star trail time-lapse rendering, and enhancing the user experience.

[0153] Example 8

[0154] Figure 10This is a flowchart of the eighth embodiment of the star trail time-lapse rendering method of the present invention. Based on the above embodiment, the step of selecting a preset image frame as the background image for rendering during the star trail time-lapse rendering process, and rendering the connecting line between the first position and the second position of each celestial body on the background image, includes:

[0155] S41. Perform the feature point detection for every two consecutive image frames.

[0156] S42. Keeping the background image unchanged, render the connecting lines between the same celestial body to be drawn in every two consecutive image frames until the star trail delay rendering is completed.

[0157] Optionally, in this embodiment, since the corresponding positions of two consecutive frames belonging to the same star have been determined in the above embodiments, the star orbit segment of the same star can be formed by connecting the two corresponding positions with a connecting line of a certain preset curvature.

[0158] Optionally, in this embodiment, the background image is kept unchanged, and the connecting lines between the same celestial body to be drawn in every two consecutive image frames are rendered until the star trail time-lapse rendering ends. When the star trail time-lapse rendering ends, it is determined whether the clouds in the original background image exist in the last frame image. If not, the clouds in the original background image are eliminated through image fusion, thereby obtaining a cleaner star trail time-lapse rendering image.

[0159] The beneficial effect of this embodiment is that, by performing feature point detection on every two consecutive image frames, while keeping the background image unchanged, the connecting lines between the same celestial body to be drawn in every two consecutive image frames are rendered until the star trail time-lapse rendering is completed. This provides a cloud-elimination star trail time-lapse rendering method to achieve a user-friendly solution, avoiding the impact of clouds on star trail time-lapse rendering, improving the final effect of the star trail time-lapse rendering, and enhancing the user experience.

[0160] Example 9

[0161] Based on the above embodiments, the present invention also proposes a star trail time-lapse rendering device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the star trail time-lapse rendering method as described in any of the above embodiments.

[0162] It should be noted that the above-described device embodiments and method embodiments belong to the same concept. The specific implementation process can be found in the method embodiments, and the technical features in the method embodiments are also applicable to the device embodiments, which will not be repeated here.

[0163] Example 10

[0164] Based on the above embodiments, the present invention also proposes a computer-readable storage medium storing a star trail time-lapse drawing program, which, when executed by a processor, implements the steps of the star trail time-lapse drawing method as described in any of the above embodiments.

[0165] It should be noted that the above-described medium embodiments and method embodiments belong to the same concept. The specific implementation process can be found in the method embodiments, and the technical features in the method embodiments are also applicable to the medium embodiments, which will not be repeated here.

[0166] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0167] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0168] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0169] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method for rendering star trails with time delay, characterized in that, The method includes: When entering the viewfinder for star trail time-lapse rendering, a set of stars is obtained based on the first image frame, and the first position of each celestial body in the first image frame is determined in the set of stars. Based on device data, celestial data, and shooting parameters, predict the second position of each celestial body in the second image frame following the first image frame; Feature point detection is performed on the first image frame and the second image frame to determine the first position and the second position belonging to the same celestial body; During the time-lapse rendering of star trails, a preset image frame is selected as the background image for rendering, and the connecting line between the first position and the second position of each celestial body is rendered on the background image. The feature point detection is performed on every two consecutive image frames, and the background image is kept unchanged. The connecting line between the same celestial body to be rendered in every two consecutive image frames is rendered until the time-lapse rendering of star trails ends. When entering the viewfinder for time-lapse star trail rendering, the process of acquiring a set of celestial bodies based on a first image frame and determining the first position of each celestial body within the first image frame in the set of celestial bodies includes: When entering the viewfinder for star trail time-lapse rendering, acquire the initial image frame; Analyze the initial image frame and identify star trail drawing interference objects in the initial image frame; If the object interfering with the star trail rendering is a cloud layer, then multiple initial image frames are continuously acquired within a preset time period. Select the image frame with the fewest clouds from the multiple initial image frames as the preset image frame; If the object interfering with the star trail drawing is the moon, then multiple initial image frames will be continuously acquired within a preset time. By fusing at least two image frames from multiple initial image frames, the image frame after removing the moon is used as the preset image frame; Calculate the number of star sets in multiple initial image frames, and select the initial image frame with the largest number of stars as the first image frame; In the first image frame, identify celestial bodies whose brightness exceeds a threshold and determine the first position of the celestial body to be drawn in the first image frame.

2. The star trail time-lapse rendering method according to claim 1, characterized in that, The step of predicting the second position of each celestial body in the second image frame following the first image frame based on device data, celestial body data, and shooting parameters includes: Obtain the exposure duration from the shooting parameters; Based on device data, celestial data, celestial drawing parameters, and the exposure duration, predict the second position of the celestial body to be drawn in the second image frame following the first image frame.

3. The star trail time-lapse rendering method according to claim 2, characterized in that, The step of performing feature point detection on the first image frame and the second image frame to determine the first location and the second location belonging to the same celestial body includes: Determine the third position of the celestial body to be drawn in the second image frame; Feature point detection is performed on the first image frame and the second image frame. In the second image frame, the celestial body corresponding to the second position that is closest to the third position is identified as the same celestial body to be drawn.

4. A star trail time-lapse rendering device, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the star trail time-lapse rendering method as described in any one of claims 1 to 3.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a star trail time-lapse drawing program, which, when executed by a processor, implements the steps of the star trail time-lapse drawing method as described in any one of claims 1 to 3.

Images