An adaptive constellation mapping method, device and computer readable storage medium

Abstract

The application discloses a kind of self-adapting constellation map method, equipment and computer readable storage medium, wherein the method comprises: according to the sensor information of current device, time information, position information, and the position information corresponding star table information, and the field angle information of camera of current device, it is calculated to obtain the first constellation range of current device under ideal state of shooting;Obtain the shooting preview image of the camera, and obtain the sky area in the shooting preview image by preset sky segmentation algorithm;In the range of the sky area, determine the star body in each constellation in the first constellation range falling into the sky area, and determine the number of the star body and in the position of the sky area;According to the number and the position determine target constellation, and the image of the target constellation is fused to the shooting preview image.The application greatly enhances the adaptability and stability of star sky shooting.

Description

Technical Field

[0001] This invention relates to the field of mobile communications, and more particularly to an adaptive constellation mapping method, device, and computer-readable storage medium. Background Technology

[0002] In current technology, with the continuous development of smart terminal devices, users' demands for device photography are also increasing. In particular, astrophotography has become a featured shooting function of mobile devices such as smartphones. Currently, most astrophotography methods enhance the images of existing celestial bodies, such as the moon, to make them more visible. However, in bright night skies such as those in cities, stars may be completely invisible, making image enhancement impossible and rendering the astrophotography function unusable. This results in significant deficiencies in the completeness, stability, and adaptability of the user's astrophotography experience. Summary of the Invention

[0003] To address the aforementioned technical deficiencies in the prior art, this invention proposes an adaptive constellation mapping method, which includes:

[0004] Based on the sensor information, time information, location information of the current device, the star catalog information corresponding to the location information, and the field of view information of the current device's camera, the first constellation range of the current device under ideal shooting conditions is calculated.

[0005] The camera captures a preview image, and a sky region in the preview image is obtained using a preset sky segmentation algorithm.

[0006] Within the range of the sky region, identify the celestial bodies that fall into the sky region from each constellation within the range of the first constellation, and determine the number of the celestial bodies and their positions within the sky region.

[0007] The target constellation is determined based on the quantity and the location, and the image of the target constellation is merged into the shooting preview image.

[0008] Optionally, the step of calculating the first constellation range of the current device under ideal shooting conditions based on the sensor information, time information, location information of the current device, the star catalog information corresponding to the location information, and the field of view information of the current device's camera includes:

[0009] The latitude and longitude information of the stars in each constellation on the preset celestial sphere is determined based on the star catalog information.

[0010] A celestial sphere model is created based on a preset graphics processing interface and the latitude and longitude information, and the celestial sphere model is rotated according to the time difference between the time information and the star catalog information.

[0011] Optionally, the step of calculating the first constellation range of the current device under ideal shooting conditions based on the sensor information, time information, location information of the current device, the star catalog information corresponding to the location information, and the field of view information of the current device's camera further includes:

[0012] Using the center of the celestial sphere model as the observation point, the shooting position and shooting orientation of the camera are determined based on the sensor information, the position information, and the field of view information.

[0013] The model view projection matrix of the celestial sphere model in a rotating state is obtained by calling the preset function of the graphics processing interface.

[0014] Optionally, the step of calculating the first constellation range of the current device under ideal shooting conditions based on the sensor information, time information, location information of the current device, the star catalog information corresponding to the location information, and the field of view information of the current device's camera further includes:

[0015] The coordinates of all celestial bodies on the celestial sphere model are transformed by matrix transformation based on the model view projection matrix to obtain the projection positions of all celestial bodies onto the preset view plane.

[0016] On the view plane, celestial bodies within a preset range and the constellations to which they belong are retained, and the constellations are designated as the first constellation range.

[0017] Optionally, acquiring the camera's preview image and obtaining the sky region in the preview image using a preset sky segmentation algorithm includes:

[0018] The sky segmentation model is obtained by training the sky mask map using a preset depth model and sky image data.

[0019] The captured preview image is input into the sky segmentation model, and the sky segmentation algorithm of the sky segmentation model is used to segment the sky in the captured preview image to obtain the sky region.

[0020] Optionally, within the range of the sky region, determining the number of celestial bodies falling into the sky region from each constellation within the first constellation range, and determining the position of the celestial bodies in the sky region, includes:

[0021] Within the first constellation, obtain the number of stars in each constellation.

[0022] If the constellation with the most stars is unique, then the constellation with the most stars will be the target constellation.

[0023] Optionally, determining the number of celestial bodies falling into the sky region from each constellation within the first constellation range, and determining the position of the celestial bodies in the sky region, further includes:

[0024] If there is more than one constellation with the most stars, then the constellations with the most stars will be included in the second constellation range.

[0025] Within the second constellation range, the distance between the geometric center of each constellation and the geometric center of the sky region is obtained, and the constellation with the smallest distance value is taken as the target constellation.

[0026] Optionally, determining the target constellation based on the quantity and the location, and fusing the image of the target constellation into the captured preview image, includes:

[0027] Set the star image parameters of the target constellation based on the image parameters of the captured preview image.

[0028] The image of the target constellation is fused into the captured preview image according to the celestial image parameters.

[0029] The present invention also proposes an adaptive constellation mapping device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the adaptive constellation mapping method as described in any of the preceding claims.

[0030] The present invention also proposes a computer-readable storage medium storing an adaptive constellation mapping program, which, when executed by a processor, implements the steps of the adaptive constellation mapping method as described in any of the preceding claims.

[0031] The adaptive constellation mapping method, device, and computer-readable storage medium of this invention calculate the first constellation range of the current device under ideal shooting conditions using sensor information, time information, location information, star catalog information corresponding to the location information, and the field of view information of the current device's camera. It then acquires a shooting preview image from the camera and obtains the sky region in the shooting preview image using a preset sky segmentation algorithm. Within the sky region, it identifies the stars falling into the sky region from each constellation within the first constellation range, and determines the number and position of each star within the sky region. Based on the number and position, it determines the target constellation and merges the image of the target constellation into the shooting preview image. This achieves a user-friendly constellation mapping processing scheme, freeing astrophotography from the limitations of lighting conditions and greatly enhancing the adaptability and stability of astrophotography. Attached Figure Description

[0032] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0033] Figure 1 This is a schematic diagram of the hardware structure of a mobile terminal according to the present invention;

[0034] Figure 2 This is a communication network system architecture diagram provided in an embodiment of the present invention;

[0035] Figure 3 This is a flowchart of the first embodiment of the adaptive constellation mapping method of the present invention;

[0036] Figure 4 This is a flowchart of the second embodiment of the adaptive constellation mapping method of the present invention;

[0037] Figure 5 This is a flowchart of the third embodiment of the adaptive constellation mapping method of the present invention;

[0038] Figure 6 This is a flowchart of the fourth embodiment of the adaptive constellation mapping method of the present invention;

[0039] Figure 7 This is a flowchart of the fifth embodiment of the adaptive constellation mapping method of the present invention;

[0040] Figure 8 This is a flowchart of the sixth embodiment of the adaptive constellation mapping method of the present invention;

[0041] Figure 9 This is a flowchart of the seventh embodiment of the adaptive constellation mapping method of the present invention;

[0042] Figure 10 This is a flowchart of the eighth embodiment of the adaptive constellation mapping method of the present invention;

[0043] Figure 11 This is a schematic diagram of a constellation under the theoretical shooting state of the sixth embodiment of the adaptive constellation mapping method of the present invention;

[0044] Figure 12 This is a preview diagram of the sixth embodiment of the adaptive constellation mapping method of the present invention.

[0045] Figure 13 This is the sky mask image of the sixth embodiment of the adaptive constellation mapping method of the present invention;

[0046] Figure 14 This is a schematic diagram of constellation selection in the sixth embodiment of the adaptive constellation mapping method of the present invention;

[0047] Figure 15 This is a schematic diagram of image fusion in the eighth embodiment of the adaptive constellation mapping method of the present invention. Detailed Implementation

[0048] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0049] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0050] Terminals can be implemented in various forms. For example, the terminals described in this invention may include mobile terminals such as mobile phones, tablets, laptops, handheld computers, personal digital assistants (PDAs), portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, etc., as well as fixed terminals such as digital TVs and desktop computers.

[0051] The following description will use a mobile terminal as an example. Those skilled in the art will understand that, apart from elements specifically designed for mobile purposes, the construction according to embodiments of the present invention can also be applied to fixed-type terminals.

[0052] Please see Figure 1This is a schematic diagram of the hardware structure of a mobile terminal implementing various embodiments of the present invention. The mobile terminal 100 may include: an RF (Radio Frequency) unit 101, a WiFi module 102, an audio output unit 103, an A / V (Audio / Video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111, etc. Those skilled in the art will understand that... Figure 1 The mobile terminal structure shown does not constitute a limitation on the mobile terminal. The mobile terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0053] The following is combined with Figure 1 A detailed introduction to each component of the mobile terminal:

[0054] The radio frequency unit 101 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 110; additionally, it transmits uplink data to the base station. Typically, the radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, and a duplexer. Furthermore, the radio frequency unit 101 can also communicate wirelessly with networks and other devices. The aforementioned wireless communications may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), and TDD-LTE (Time Division Duplexing-Long Term Evolution).

[0055] WiFi is a short-range wireless transmission technology. Mobile terminals, through the WiFi module 102, can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 1 WiFi module 102 is shown, but it is understood that it is not a necessary component of a mobile terminal and can be omitted as needed without changing the nature of the invention.

[0056] The audio output unit 103 can convert audio data received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109 into audio signals and output them as sound when the mobile terminal 100 is in call signal receiving mode, call mode, recording mode, voice recognition mode, broadcast receiving mode, etc. Furthermore, the audio output unit 103 can also provide audio output related to specific functions performed by the mobile terminal 100 (e.g., call signal receiving sound, message receiving sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0057] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos acquired by an image capture device (such as a camera) in video capture mode or image capture mode. The processed image frames can be displayed on the display unit 106. The image frames processed by the GPU 1041 can be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) in operating modes such as telephone call mode, recording mode, and voice recognition mode, and can process such sound into audio data. The processed audio (voice) data can be converted into a format that can be transmitted to a mobile communication base station via the radio frequency unit 101 in telephone call mode. The microphone 1042 can implement various types of noise cancellation (or suppression) algorithms to eliminate (or suppress) noise or interference generated during the reception and transmission of audio signals.

[0058] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 1061 according to the ambient light level, and the proximity sensor can turn off the display panel 1061 and / or backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, tapping), etc. Other sensors that may be configured in the phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0059] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0060] User input unit 107 can be used to receive input numerical or character information, and generate key signal inputs related to user settings and function control of the mobile terminal. Specifically, user input unit 107 may include touch panel 1071 and other input devices 1072. Touch panel 1071, also known as touch screen, can collect touch operations on or near the user (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near touch panel 1071), and drive corresponding connection devices according to a pre-set program. Touch panel 1071 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to processor 110, and can receive and execute commands from processor 110. In addition, touch panel 1071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may also include other input devices 1072. Specifically, other input devices 1072 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc., without being limited here.

[0061] Furthermore, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. Subsequently, the processor 110 provides corresponding visual output on the display panel 1061 based on the type of touch event. Although in Figure 1 In this embodiment, the touch panel 1071 and the display panel 1061 are two independent components to realize the input and output functions of the mobile terminal. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to realize the input and output functions of the mobile terminal. The specific implementation is not limited here.

[0062] Interface unit 108 serves as an interface through which at least one external device can connect to mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, and so on. Interface unit 108 may be used to receive input (e.g., data, power, etc.) from the external device and transmit the received input to one or more elements within mobile terminal 100, or it may be used to transmit data between mobile terminal 100 and the external device.

[0063] The memory 109 can be used to store software programs and various data. The memory 109 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback, image playback, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory 109 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0064] The processor 110 is the control center of the mobile terminal. It connects various parts of the mobile terminal via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 109, and by calling data stored in the memory 109, it performs various functions and processes data of the mobile terminal, thereby providing overall monitoring of the mobile terminal. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 110.

[0065] The mobile terminal 100 may also include a power supply 111 (such as a battery) that supplies power to various components. Preferably, the power supply 111 can be logically connected to the processor 110 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0066] although Figure 1 As not shown, the mobile terminal 100 may also include a Bluetooth module, etc., which will not be described in detail here.

[0067] To facilitate understanding of the embodiments of the present invention, the communication network system on which the mobile terminal of the present invention is based is described below.

[0068] Please see Figure 2 , Figure 2 This invention provides a communication network system architecture diagram. The communication network system is an LTE system based on the universal mobile communication technology. The LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203, and the operator's IP services 204, which are connected in sequence.

[0069] Specifically, UE201 can be the aforementioned terminal 100, which will not be elaborated here.

[0070] E-UTRAN202 includes eNodeB2021 and other eNodeB2022s. Among them, eNodeB2021 can connect to other eNodeB2022s via backhaul (e.g., X2 interface), and eNodeB2021 connects to EPC203. eNodeB2021 can provide UE201 with access to EPC203.

[0071] EPC203 may include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving Gateway) 2034, PGW (Packet Data Network Gateway) 2035, and PCRF (Policy and Charging Rules Function) 2036, etc. Among them, MME2031 is the control node that handles signaling between UE201 and EPC203, providing bearer and connection management. HSS2032 provides registers to manage functions such as the Home Location Register (not shown in the diagram) and stores user-specific information such as service characteristics and data rates. All user data can be sent through SGW2034. PGW2035 can provide UE 201 IP address allocation and other functions. PCRF2036 is the policy and charging control decision point for service data flow and IP bearer resources. It selects and provides available policy and charging control decisions for the policy and charging enforcement function unit (not shown in the figure).

[0072] IP services 204 may include the Internet, intranet, IMS (IP Multimedia Subsystem), or other IP services.

[0073] Although the above description uses the LTE system as an example, those skilled in the art should understand that the present invention is not only applicable to the LTE system, but also to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, and future new network systems, etc., which are not limited here.

[0074] Based on the aforementioned mobile terminal hardware structure and communication network system, various embodiments of the method of the present invention are proposed.

[0075] Example 1

[0076] Figure 3 This is a flowchart of the first embodiment of the adaptive constellation mapping method of the present invention. An adaptive constellation mapping method, the method comprising:

[0077] S1. Based on the sensor information, time information, and location information of the current device, as well as the star catalog information corresponding to the location information and the field of view information of the current device's camera, calculate the first constellation range of the current device under ideal shooting conditions.

[0078] S2. Obtain the shooting preview image of the camera, and obtain the sky area in the shooting preview image through a preset sky segmentation algorithm.

[0079] S3. Within the range of the sky region, determine the stars that fall into the sky region from each constellation within the range of the first constellation, and determine the number of the stars and their positions in the sky region.

[0080] S4. Determine the target constellation based on the quantity and the location, and fuse the image of the target constellation into the shooting preview image.

[0081] In this embodiment, a mobile phone is used as an example. First, the accelerometer, magnetometer, GPS information, current time, and FOV (Field of View) information of the mobile phone camera are acquired. Using this information, the constellations that the mobile phone camera can theoretically capture are calculated. Then, the image from the camera is acquired, and a sky segmentation algorithm is used to obtain the sky region in the image. Next, the number of stars in the sky region for each constellation from the first step is counted, and the focal constellation is selected. If there is only one constellation with the most stars, this constellation is selected as the target constellation. Otherwise, the geometric center of the constellation with the most stars and the geometric center of the sky region are counted, and the constellation whose geometric center is closest to the geometric center of the sky region is selected as the target constellation. Finally, the target constellation is mapped onto the sky region of the image.

[0082] The beneficial effect of this embodiment is that, by using the sensor information, time information, and location information of the current device, as well as the star catalog information corresponding to the location information and the field of view information of the current device's camera, the first constellation range of the current device under ideal shooting conditions is calculated; the shooting preview image of the camera is acquired, and the sky region in the shooting preview image is obtained through a preset sky segmentation algorithm; within the sky region, the stars falling into the sky region in each constellation within the first constellation range are determined, and the number of the stars and their positions in the sky region are determined; the target constellation is determined based on the number and the position, and the image of the target constellation is fused into the shooting preview image. This achieves a user-friendly constellation mapping processing scheme, making astrophotography no longer limited by the lighting environment, greatly enhancing the adaptability and stability of astrophotography.

[0083] Example 2

[0084] Figure 4This is a flowchart of the second embodiment of the adaptive constellation mapping method of the present invention. Based on the above embodiment, the step of calculating the first constellation range of the current device under ideal shooting conditions according to the sensor information, time information, position information of the current device, the star catalog information corresponding to the position information, and the field of view information of the camera of the current device includes:

[0085] S11. Determine the latitude and longitude information of the stars in each constellation on the preset celestial sphere based on the star catalog information.

[0086] S12. Create a celestial sphere model of the celestial sphere according to the preset graphics processing interface and the latitude and longitude information, and control the rotation of the celestial sphere model according to the time difference between the time information and the star catalog information.

[0087] Optionally, in this embodiment, the Earth is considered as a point, and outside the Earth is a large concentric sphere called the celestial sphere, on which stars (except the Sun) reside. The relative positions of the stars on the celestial sphere remain constant; their apparent movement from Earth is due to the rotation of the celestial sphere. Based on this, the calculation method in this embodiment includes the following two aspects: First, obtaining a star catalog J2000.0 from the National Astronomical Observatories, which contains the latitude and longitude of 693 stars in 88 constellations on the celestial sphere on January 1, 2000; Second, using the OpenGL image processing interface, a celestial sphere model is built based on the star catalog, and then the celestial sphere is rotated according to the time difference between the current time and J2000.0.

[0088] The beneficial effect of this embodiment is that it determines the latitude and longitude information of the stars in each constellation on a preset celestial sphere through the star catalog information; it creates a celestial sphere model based on a preset graphics processing interface and the latitude and longitude information; and it controls the rotation of the celestial sphere model according to the time difference between the time information and the star catalog information. This achieves a user-friendly constellation mapping processing scheme, making astrophotography no longer limited by the lighting environment, and greatly enhancing the adaptability and stability of astrophotography.

[0089] Example 3

[0090] Figure 5 This is a flowchart of the third embodiment of the adaptive constellation mapping method of the present invention. Based on the above embodiment, the step of calculating the first constellation range of the current device under ideal shooting conditions based on the sensor information, time information, position information of the current device, the star catalog information corresponding to the position information, and the field of view information of the camera of the current device further includes:

[0091] S13. Using the center of the celestial sphere model as the observation point, determine the shooting position and shooting orientation of the camera based on the sensor information, the position information, and the field of view information.

[0092] S14. Obtain the model view projection matrix of the celestial sphere model in the rotating state by calling the preset function of the graphics processing interface.

[0093] Optionally, in this embodiment, firstly, taking the center of the sphere as the observation point, the position and orientation of the camera in OpenGL can be obtained based on the information from the accelerometer, magnetometer, GPS, and the FOV of the mobile phone camera, i.e., the parameters of the gluLockAt and gluPerspective functions; then, the gluLockAt and gluPerspective functions are called to obtain an MVP model view projection transformation matrix.

[0094] The beneficial effect of this embodiment is that, by using the center of the celestial sphere model as the observation point, the shooting position and shooting orientation of the camera are determined based on the sensor information, the position information, and the field of view information; and the model view projection matrix of the celestial sphere model in a rotating state is obtained by calling the preset function of the graphics processing interface. This achieves a user-friendly constellation mapping processing scheme, making starry sky photography no longer limited by the lighting environment, greatly enhancing the adaptability and stability of starry sky photography.

[0095] Example 4

[0096] Figure 6 This is a flowchart of the fourth embodiment of the adaptive constellation mapping method of the present invention. Based on the above embodiment, the step of calculating the first constellation range of the current device under ideal shooting conditions based on the sensor information, time information, position information of the current device, the star catalog information corresponding to the position information, and the field of view information of the camera of the current device further includes:

[0097] S15. Perform matrix transformation on the coordinates of all celestial bodies on the celestial sphere model according to the model view projection matrix to obtain the projection positions of all celestial bodies onto the preset view plane.

[0098] S16. On the view plane, retain the stars within a preset range and the constellations to which the stars belong, and use the constellations as the first constellation range.

[0099] Optionally, in this embodiment, the coordinates of the stars on the celestial sphere are transformed using a transformation matrix to obtain their positions projected onto the view plane.

[0100] Optionally, in this embodiment, only the stars projected in the view and the constellations they belong to are retained as the first constellation range under ideal shooting conditions.

[0101] The beneficial effect of this embodiment is that by performing matrix transformation on the coordinates of all celestial bodies on the celestial sphere model through the model view projection matrix, the projection positions of all celestial bodies onto a preset view plane are obtained; celestial bodies within a preset range and their respective constellations are retained on the view plane, and these constellations are used as the first constellation range. This achieves a user-friendly constellation mapping processing scheme, making astrophotography no longer limited by the lighting environment, and greatly enhancing the adaptability and stability of astrophotography.

[0102] Example 5

[0103] Figure 7 This is a flowchart of the fifth embodiment of the adaptive constellation mapping method of the present invention. Based on the above embodiment, the step of acquiring the shooting preview image of the camera and acquiring the sky region in the shooting preview image through a preset sky segmentation algorithm includes:

[0104] S21. The sky mask image is trained using a preset depth model and sky image data to obtain the sky segmentation model.

[0105] S22. Input the captured preview image into the sky segmentation model, and use the sky segmentation algorithm of the sky segmentation model to segment the captured preview image to obtain the sky region.

[0106] Optionally, in this embodiment, an n2netp deep learning model is trained using 100,000 sky images and their corresponding sky mask images. The pixel value for the sky region is 255, and for non-sky regions it is 0, thus obtaining a sky segmentation model. When the model is running, a preview image captured by the camera is input into the model to obtain the corresponding sky mask image. Sky segmentation is then performed on this sky mask image to obtain the sky region.

[0107] The beneficial effect of this embodiment is that the sky segmentation model is obtained by training the sky mask map with a preset depth model and sky image data; the captured preview image is input into the sky segmentation model, and the sky segmentation algorithm of the sky segmentation model is used to segment the sky in the captured preview image to obtain the sky region. This implements a user-friendly constellation mapping processing scheme, making astrophotography no longer limited by the lighting environment, greatly enhancing the adaptability and stability of astrophotography.

[0108] Example 6

[0109] Figure 8This is a flowchart of the sixth embodiment of the adaptive constellation mapping method of the present invention. Based on the above embodiment, the step of determining the celestial bodies falling into the sky region from each constellation within the first constellation range within the sky region, and determining the number of the celestial bodies and their positions in the sky region, includes:

[0110] S31. Within the first constellation range, obtain the number of stars in each constellation.

[0111] S32. If the constellation with the most stars is unique, then the constellation with the most stars shall be the target constellation.

[0112] Optionally, in this embodiment, please refer to Figure 11 The diagram shows a constellation under theoretical shooting conditions. The constellations theoretically visible to the current mobile phone camera are defined as the first constellation range. Please refer to... Figure 12 The diagram shown is a preview of the actual shooting. It illustrates the content of the preview image, but does not show any celestial bodies. Further details can be found in the following references. Figure 13 The sky mask image is shown. The model is input with a preview image captured by a camera to obtain the corresponding sky mask image. Then, the sky mask image is segmented to obtain the sky region.

[0113] Optionally, in this embodiment, please refer to Figure 14 The diagram illustrates constellation selection, where a target constellation is chosen based on the number and position of stars in the sky. For example, if the constellation with the most stars is unique, then that constellation is selected as the target constellation.

[0114] The beneficial effect of this embodiment is that, by obtaining the number of stars within each constellation within the first constellation range, and if the constellation with the most stars is unique, then the constellation with the most stars is taken as the target constellation. This achieves a user-friendly constellation mapping processing scheme, making astrophotography no longer limited by lighting conditions, greatly enhancing the adaptability and stability of astrophotography.

[0115] Example 7

[0116] Figure 9 This is a flowchart of the seventh embodiment of the adaptive constellation mapping method of the present invention. Based on the above embodiment, the step of determining the celestial bodies falling into the sky region from each constellation within the first constellation range within the sky region, and determining the number of the celestial bodies and their positions in the sky region, further includes:

[0117] S33. If there is more than one constellation with the most stars, then the constellations with the most stars shall be included in the second constellation range.

[0118] S34. Within the range of the second constellation, obtain the distance value between the geometric center of each constellation and the geometric center of the sky region, and take the constellation with the smallest distance value as the target constellation.

[0119] Optionally, in this embodiment, the geometric center of the constellation is the average of the projected coordinates of all the stars in the constellation, and the geometric center of the sky region is the average of the coordinates of the sky pixels with a value of 255 in the mask image.

[0120] The beneficial effect of this embodiment is that, if it is determined that there is more than one constellation with the most stars, then the constellations with the most stars are classified into a second constellation range; within the second constellation range, the distance value between the geometric center of each constellation and the geometric center of the sky region is obtained, and the constellation with the smallest distance value is taken as the target constellation. This implements a user-friendly constellation mapping processing scheme, making astrophotography no longer limited by the lighting environment, and greatly enhancing the adaptability and stability of astrophotography.

[0121] Example 8

[0122] Figure 10 This is a flowchart of the eighth embodiment of the adaptive constellation mapping method of the present invention. Based on the above embodiment, the step of determining the target constellation according to the quantity and the position, and fusing the image of the target constellation into the captured preview image, includes:

[0123] S41. Set the star image parameters of the target constellation according to the image parameters of the captured preview image.

[0124] S42. Merge the image of the target constellation into the shooting preview image according to the star image parameters.

[0125] Optionally, in this embodiment, please refer to Figure 15 The diagram illustrates image fusion. The image parameters of the target constellation are set according to the image parameters of the captured preview image, thereby ensuring that the appearance of the stars remains consistent with the background.

[0126] The beneficial effect of this embodiment is that the image parameters of the target constellation are set by the image parameters of the captured preview image; the image of the target constellation is then fused into the captured preview image according to the image parameters. This achieves a user-friendly constellation mapping processing scheme, making astrophotography no longer limited by the lighting environment, and greatly enhancing the adaptability and stability of astrophotography.

[0127] Example 9

[0128] Based on the above embodiments, the present invention also proposes an adaptive constellation mapping device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the adaptive constellation mapping method as described in any of the above embodiments.

[0129] It should be noted that the above-described device embodiments and method embodiments belong to the same concept. The specific implementation process can be found in the method embodiments, and the technical features in the method embodiments are also applicable to the device embodiments, which will not be repeated here.

[0130] Example 10

[0131] Based on the above embodiments, the present invention also proposes a computer-readable storage medium storing an adaptive constellation mapping program, which, when executed by a processor, implements the steps of the adaptive constellation mapping method as described in any of the above embodiments.

[0132] It should be noted that the above-described medium embodiments and method embodiments belong to the same concept. The specific implementation process can be found in the method embodiments, and the technical features in the method embodiments are also applicable to the medium embodiments, which will not be repeated here.

[0133] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0134] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0135] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0136] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. An adaptive constellation mapping method, characterized in that, The method includes: Based on the sensor information, time information, and location information of the current device, as well as the star catalog information corresponding to the location information and the field of view information of the current device's camera, the first constellation range of the current device under ideal shooting conditions is calculated. The camera captures a preview image, and a sky region in the preview image is obtained using a preset sky segmentation algorithm. Within the range of the sky region, identify the celestial bodies that fall into the sky region from each constellation within the range of the first constellation, and determine the number of the celestial bodies and their positions within the sky region; The target constellation is determined based on the quantity and the location, and the image of the target constellation is merged into the captured preview image; The step of calculating the first constellation range of the current device under ideal shooting conditions based on the sensor information, time information, location information of the current device, the star catalog information corresponding to the location information, and the field of view information of the current device's camera includes: Based on the star catalog information, determine the latitude and longitude information of the stars in each constellation on the preset celestial sphere; A celestial sphere model is created based on a preset graphics processing interface and the latitude and longitude information, and the celestial sphere model is rotated according to the time difference between the time information and the star catalog information. Using the center of the celestial sphere model as the observation point, the shooting position and shooting orientation of the camera are determined based on the sensor information, the position information, and the field of view information; The model view projection matrix of the celestial sphere model in a rotating state is obtained by calling the preset function of the graphics processing interface. Based on the model view projection matrix, the coordinates of all celestial bodies on the celestial sphere model are transformed by matrix transformation to obtain the projection positions of all celestial bodies onto the preset view plane. On the view plane, retain the stars within a preset range and the constellations to which the stars belong, and use the constellations as the first constellation range; Within the defined sky region, determining the number of celestial bodies falling within the sky region from each constellation within the first constellation range, and determining the location of these celestial bodies within the sky region, includes: Within the first constellation, obtain the number of stars in each constellation; If the constellation with the most stars is unique, then the constellation with the most stars will be the target constellation. If there is more than one constellation with the most stars, then the constellations with the most stars will be included in the second constellation range. Within the second constellation range, the distance between the geometric center of each constellation and the geometric center of the sky region is obtained, and the constellation with the smallest distance value is taken as the target constellation.

2. The adaptive constellation mapping method according to claim 1, characterized in that, The step of acquiring the camera's preview image and obtaining the sky region in the preview image using a preset sky segmentation algorithm includes: The sky segmentation model is obtained by training the sky mask map using a preset depth model and sky image data. The captured preview image is input into the sky segmentation model, and the sky segmentation algorithm of the sky segmentation model is used to segment the sky in the captured preview image to obtain the sky region.

3. The adaptive constellation mapping method according to claim 2, characterized in that, The step of determining the target constellation based on the quantity and the location, and fusing the image of the target constellation into the captured preview image, includes: Set the star image parameters of the target constellation according to the image parameters of the captured preview image; The image of the target constellation is fused into the captured preview image according to the celestial image parameters.

4. An adaptive constellation mapping device, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the adaptive constellation mapping method as described in any one of claims 1 to 3.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an adaptive constellation mapping program, which, when executed by a processor, implements the steps of the adaptive constellation mapping method as described in any one of claims 1 to 3.

Images