Celestial Observation System for a Vehicle and Celestial Observation Method for a Vehicle

Abstract

The present invention relates to the field of intelligent cockpits. The present invention provides a celestial body observation system for a vehicle, the celestial body observation system comprising: a camera movably mounted on the vehicle and configured to be able to capture an image of the sky outside the vehicle; a vehicle positioning module configured to be able to obtain the azimuth information of the vehicle; a control module configured to be able to determine the azimuth relationship of at least one celestial body to be observed relative to the vehicle according to the azimuth information of the vehicle, and adjust the pose of the camera relative to the vehicle according to the azimuth relationship so that at least one celestial body is within the viewing angle range of the camera; and an output module configured to be able to output a celestial body observation view in the vehicle based at least on the sky image captured by the camera. The present invention also relates to a celestial body observation method for a vehicle and a machine-readable storage medium. By dynamically adapting the celestial body observation angle by adjusting the pose of the camera in consideration of the vehicle azimuth, the celestial body observation experience of vehicle users is improved.

Description

Technical Field

[0001] The present invention relates to a celestial body observation system for a vehicle, and also relates to a celestial body observation method for a vehicle and a machine-readable storage medium. Background Art

[0002] With the progress of modern social technology and the advancement of vehicle intelligence, vehicles are no longer simple means of transportation. On the premise of ensuring safety and comfort, people also put forward higher requirements for vehicle interaction functions. With the triumphant return of the Shenzhou XIII manned spacecraft, a new wave of aerospace knowledge has been set off. Currently, more and more vehicle users expect to obtain an immersive observation experience of the universe and stars within the limited vehicle space.

[0003] At present, a starry sky ceiling is usually used to simulate the effect of stars at night, but this simulated starlight lacks realism and reduces the user experience. In addition, traditional in-vehicle projection solutions are relatively single. Usually, the external camera images of the vehicle are directly connected to the vehicle interior, and the projection content is greatly limited by the vehicle position and attitude, and it is impossible to selectively screen the celestial bodies to be observed, bringing limited enjoyment and popular science to users.

[0004] In this context, there is an expectation to provide a celestial body observation solution for a vehicle, aiming to achieve the tracking and capture of target celestial bodies through dynamic adaptation of the input source, so that vehicle users can obtain a more realistic celestial body observation experience. Summary of the Invention

[0005] The object of the present invention is to provide a celestial body observation system for a vehicle, and the celestial body observation system includes:

[0006] A camera, which is movably installed on the vehicle and configured to be able to capture sky images outside the vehicle;

[0007] A vehicle positioning module, which is configured to be able to obtain the azimuth information of the vehicle;

[0008] A control module, which is configured to be able to determine the azimuth relationship of at least one celestial body to be observed relative to the vehicle according to the azimuth information of the vehicle, and adjust the pose of the camera relative to the vehicle according to the azimuth relationship, so that at least one celestial body is within the viewing range of the camera; and

[0009] An output module, which is configured to be able to output a celestial body observation view in the vehicle based at least on the sky images captured by the camera.

[0010] The present invention particularly includes the following technical concepts: By adjusting the relative pose of the camera, the celestial observation angle is dynamically adapted, so that regardless of the vehicle's orientation, accurate and reliable capture of the target celestial body can be achieved. Thus, real celestial observation conditions as if in nature are created for passengers in the vehicle, allowing users to understand the current celestial body morphology and changes in real time, enriching the user's sensory experience while also expanding their aerospace knowledge reserve.

[0011] Optionally, the celestial observation system further includes a vehicle motion determination module configured to be able to obtain the driving direction and / or driving speed of the vehicle; wherein, the control module is further configured to be able to determine the dynamic change of the azimuth relationship of at least one celestial body relative to the vehicle according to the driving direction and / or driving speed of the vehicle, and additionally adjust the pose of the camera relative to the vehicle according to the dynamic change of the azimuth relationship.

[0012] Optionally, the celestial observation system further includes an image recognition module configured to be able to recognize at least one celestial body to be observed in the sky image captured by the camera; wherein, the control module is further configured to adjust the pose of the camera relative to the vehicle according to the recognition result of the image recognition module, so that the camera performs tracking shooting on the at least one recognized celestial body.

[0013] Thus, the dynamic tracking of the target celestial body is realized by using image recognition technology, improving the observation stability in the driving state.

[0014] Optionally, the celestial observation system further includes an image recognition module configured to be able to recognize at least one celestial body to be observed in the sky image captured by the camera;

[0015] Wherein, the control module is further configured to:

[0016] - Readjust the pose of the camera relative to the vehicle in the case of not recognizing at least one celestial body to be observed, and / or,

[0017] - In the case of not recognizing at least one celestial body to be observed a predetermined number of times, control the output module to output a celestial observation view in the vehicle based on another source image, the another source image being different from the sky image captured in real time by the camera.

[0018] Optionally, the control module is further configured to be able to extract a local section from the sky image captured by the camera, so that the local section only includes at least one celestial body to be observed, and / or, so that the proportion of the at least one celestial body to be observed in the local section exceeds a preset value; and / or, the output module is further configured to output a celestial observation view based on the local section extracted from the sky image.

[0019] Optionally, the control module is further configured to control the output module to output the celestial observation view according to the determined output effect, where outputting the celestial observation view according to the determined output effect includes:

[0020] - Directly output the sky image captured by the camera as the celestial observation view;

[0021] - Mark at least one celestial body in terms of its morphology and / or category in the sky image captured by the camera, and output the marked sky image as the celestial observation view; and / or

[0022] - Output the introductory information about at least one celestial body to be observed in text form and / or voice form synchronously with the output of the sky image.

[0023] Optionally, the control module is further configured to control the output of the celestial observation view in the vehicle at least in a real scene mode and a non-real scene mode. In the real scene mode, the output module is made to output the celestial observation view in the vehicle based on the sky image captured by the camera in real time. In the non-real scene mode, the output module is made to output the celestial observation view in the vehicle based on another source image, where the another source image is different from the sky image captured by the camera in real time. By controlling the output of the celestial observation view in the real scene mode and the non-real scene mode, the free switching of the celestial observation source can be realized for different driving states, user needs, and weather conditions. On the one hand, it can provide passengers with a real-time immersive observation experience. On the other hand, it can also create an atmosphere of being in the starry sea in the cabin and provide passengers with an opportunity to learn aerospace popular science knowledge.

[0024] Optionally, in the non-real scene mode, the another source image includes: the sky image captured or recorded in advance by the camera; and / or, the sky image, starry sky atmosphere image, and / or celestial body popular science image pre-stored in the vehicle locally and / or received from outside the vehicle. It fills the gap in the entertainment and teaching functions of the vehicle, enriches the user experience in the cabin, and enhances the sense of technology of the vehicle.

[0025] Optionally, the control module is further configured to be able to control the output module to output the celestial observation view according to the motion state of the vehicle. Among them, the control module is configured to prohibit the output of the celestial observation view in the vehicle or only allow the output of the celestial observation view in the non-real scene mode when the driving speed of the vehicle is greater than the threshold. The flexible switching of the celestial observation source is realized, which can fully ensure driving safety.

[0026] Optionally, the control module is further configured to be able to control the output module to output the celestial observation view according to the weather conditions. Among them, the control module is configured to only allow the output of the celestial observation view in the non-real scene mode when the weather conditions do not meet the preset requirements.

[0027] Optionally, the control module is further configured to be able to recommend, in a personalized manner according to the identity information of the vehicle user, the output mode of the celestial observation view, the output position of the celestial observation view in the vehicle, and / or the output effect.

[0028] Optionally, the control module is further configured to be able to cooperatively control the output module and at least one cabin component of the vehicle, so as to trigger a change in the state of the at least one cabin component in temporal association with the output of the celestial observation view. The at least one cabin component includes the seats, ambient lights, audio system, and / or air conditioner of the vehicle. Through such cooperative control, the user can observe celestial bodies in the most comfortable posture and angle, and the user's sensory experience is enriched through a multi-modal interaction method.

[0029] Optionally, the celestial observation system further includes a user input module, which is configured to be able to receive:

[0030] - a first specified input from the vehicle user regarding the output mode of the celestial observation view,

[0031] - a second specified input from the vehicle user regarding the output position of the celestial observation view in the vehicle, and / or

[0032] - a third specified input from the vehicle user regarding the category of celestial bodies to be observed;

[0033] Wherein, the control module is further configured to:

[0034] - select to output the celestial observation view in a real-scene mode or a non-real-scene mode according to the first specified input of the vehicle user,

[0035] - control the output module according to the second specified input of the vehicle user to output the celestial observation view at the specified output position in the vehicle, and / or

[0036] - control the pose of the camera relative to the vehicle and / or the output of the output module according to the third specified input of the vehicle user, so that the celestial observation view output in the vehicle includes the category of celestial bodies specified by the vehicle user.

[0037] Optionally, the celestial observation system further includes a user input module, which is configured to be able to receive gesture inputs from the vehicle user for the celestial observation view that has been output in the vehicle;

[0038] Wherein, the control module is further configured to:

[0039] - identify the selected area of the vehicle user in the celestial observation view according to the gesture input of the vehicle user, and control the output module to output the introductory information of the celestial body corresponding to the selected area; and / or

[0040] - Recognize the intention of image transformation based on the gesture input of the vehicle user, and control the movement of the camera and / or the output of the output module, so that the output celestial observation view performs rotation, translation, and / or scaling operations corresponding to the image transformation intention.

[0041] Optionally, the celestial observation system further includes a communication interface configured to be able to upload the sky image captured by the camera to the cloud, transmit it to the mobile terminal device of the vehicle user, and / or share it with another vehicle.

[0042] According to a second aspect of the present invention, there is provided a celestial observation method for a vehicle, wherein the celestial observation method includes the following steps:

[0043] Obtain the azimuth information of the vehicle;

[0044] Determine the azimuth relationship of at least one celestial body to be observed relative to the vehicle according to the azimuth information of the vehicle, and adjust the pose of the camera relative to the vehicle according to the azimuth relationship, so that at least one celestial body is within the viewing range of the camera;

[0045] Capture the sky image outside the vehicle by means of the camera; and

[0046] Output a celestial observation view in the vehicle based at least on the sky image captured by the camera.

[0047] Optionally, the celestial observation method further includes the following steps:

[0048] Control the output of the celestial observation view in the vehicle at least in a real scene mode and a non-real scene mode, wherein in the real scene mode, the output module outputs the celestial observation view in the vehicle based on the sky image captured by the camera in real time, and in the non-real scene mode, the output module outputs the celestial observation view in the vehicle based on another source image different from the sky image captured by the camera in real time.

[0049] According to a third aspect of the present invention, there is provided a machine-readable storage medium, on which a computer program is stored, and the computer program is used to execute the celestial observation method according to the first aspect of the present invention when running on a computer. Description of the Drawings

[0050] Next, the present invention can be better understood by referring to the accompanying drawings in more detail. The principles, features, and advantages of the present invention can be better understood. The accompanying drawings include:

[0051] Figure 1 A block diagram of a celestial observation system for a vehicle according to an exemplary embodiment of the present invention is shown;

[0052] Figures 2a - 2cA schematic diagram showing the output of a celestial observation view in a vehicle in an exemplary scenario;

[0053] Figures 3a - 3f Schematic diagrams showing the interfaces of the celestial observation views output in a vehicle in a real scene mode and a non-real scene mode;

[0054] Figures 4a - 4d Schematic diagrams showing that a user changes the output effect of the celestial observation view through gesture interaction;

[0055] Figure 5 Schematic diagrams showing the interface of a user input module according to an exemplary embodiment of the present invention; and

[0056] Figure 6 Schematic diagrams showing the flowchart of a celestial observation method for a vehicle according to an exemplary embodiment of the present invention. Detailed Description of the Invention

[0057] In order to make the technical problems to be solved, technical solutions and beneficial technical effects of the present invention clearer and more understandable, the present invention will be further described in detail below in conjunction with the drawings and multiple exemplary embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the protection scope of the present invention.

[0058] Figure 1 Schematic diagrams showing the block diagram of a celestial observation system for a vehicle according to an exemplary embodiment of the present invention.

[0059] As Figure 1 shown, the celestial observation system 1 includes a camera 2, an output module 3, a vehicle positioning module 12 and a control module 4, and these modules are connected to each other in terms of communication technology.

[0060] The camera 2 is movably mounted on the vehicle and configured to be able to capture the sky image outside the vehicle. In one example, the camera 2 is a wide-angle camera and is arranged on the outside roof of the vehicle so that its initial viewing angle points to a part of the sky above the vehicle. In another example, the camera 2 can also be an environmental perception camera originally used by the vehicle to support driving assistance functions or autonomous driving functions. At this time, the initial viewing angle of the camera points to the vehicle driving direction, for example. In addition, multiple cameras can also be arranged at different positions on the vehicle body to synthesize the images captured by them and then provide a celestial observation view.

[0061] The output module 3 is configured to be able to output an astronomical observation view in a vehicle. In one example, the output module 3 is configured as a projection device that can project the content to be projected onto a specified projection area in the vehicle cabin or display it in the vehicle using holographic projection technology. Such a projection area can be, for example, the vehicle ceiling, the inner wall of the vehicle cabin, the sunroof or the window. In another example, the output module 3 can also be configured as a vehicle display. Such a display includes, for example, the vehicle head unit screen, the intelligent instrument screen, the front row / rear row multi-functional intelligent tablets, etc. Here, the astronomical observation view can especially exist in the form of images and graphics. This can be, for example, a photo or an astronomical pictogram. It is also possible that the astronomical observation view contains text or graphic annotations or contains voice explanations. Therefore, the astronomical observation view is not necessarily static, but can also change dynamically over time. Thus, the astronomical observation view can also include a video sequence composed of multiple single-frame images.

[0062] The vehicle user can be, for example, the driver, the co-driver or other passengers of the vehicle. However, in some cases, the designated observer can also be located outside the vehicle and observe the astronomical observation view projected onto the window from the outside.

[0063] The vehicle positioning module 12 includes, for example, an in-vehicle GPS sensor, an inertial navigation device, etc., which are configured to be able to obtain the azimuth information of the vehicle. Such azimuth information includes, for example, the geographical coordinates and longitude and latitude information of the vehicle's location. In addition, the vehicle azimuth information also includes, for example, the attitude information of the vehicle (pitch, yaw, etc.). In some cases, the azimuth information can also carry a timestamp.

[0064] The control module 4 is configured to determine the azimuth relationship between at least one celestial body to be observed and the vehicle according to the azimuth information of the vehicle, and adjust the pose of the camera relative to the vehicle according to this azimuth relationship so that at least one celestial body is within the viewing range of the camera. For this purpose, the control module 4 can, for example, access a database storing the astronomical position data of all celestial bodies, and then determine all candidate celestial body information that can be observed from the current position of the vehicle in combination with the vehicle azimuth and the system built-in time. In one example, the control module can directly determine all candidate celestial bodies as the celestial bodies to be observed. In another example, the control module can, according to the system default configuration, use a specific celestial body of a preset category (such as the moon) as the celestial body to be observed. In another example, it is also possible to select the celestial body of interest from all possible observable candidate celestial bodies based on the user input, and use the celestial body of interest as the celestial body to be observed.

[0065] In one example, the control module 4 can be further divided into a calculation unit and a motion execution unit. The calculation unit calculates the azimuth relationship of the celestial body to be observed relative to the vehicle based on the azimuth information of the vehicle (for example, whether a certain celestial body is on the left, right, front, or rear side of the vehicle), and thereby generates a control signal for controlling the movement of the camera 2. This control signal is provided to the motion execution unit coupled to the camera 2, which is, for example, configured as an electrically driven rotary joint that can, for example, drive the camera 2 to rotate 360 degrees in the horizontal direction and 180 degrees in the vertical direction. The motion execution unit controllably adjusts the position and / or angle of the camera 2 relative to the vehicle body so that its viewing range is aligned with the celestial body to be observed.

[0066] To more stably track the celestial body to be observed, the consideration of the vehicle's motion state can also be introduced during the control of the camera's viewing angle transformation. For this purpose, the celestial body observation system 1 also optionally includes a vehicle motion determination module 13. The vehicle motion determination module 13 includes, for example, the wheel speed sensors and gyroscopes of the vehicle, and thereby can determine the driving speed and driving direction of the vehicle. After obtaining this information, the control module 4 can determine the change over time of the azimuth relationship of the celestial body to be observed relative to the vehicle, and additionally adjust the pose of the camera relative to the vehicle according to this dynamic change. For example, the position of the vehicle on the earth is relatively fixed, so the azimuth of the celestial body to be observed relative to the geographical area where the vehicle is located does not change suddenly. However, after the vehicle changes its driving direction, the celestial body to be observed may change from one side of the vehicle to the other side. At this time, if the camera shooting angle is not adjusted in time, the target celestial body may be lost from the in-vehicle projection screen. Therefore, it is necessary to dynamically adjust the camera according to the motion state of the vehicle so that the observation angle of the camera is adapted to the vehicle motion.

[0067] To more precisely track the celestial body to be observed, consideration of the pre-planned driving route of the vehicle can also be introduced during the control of the camera viewing angle transformation. For this purpose, the celestial body observation system 1 also optionally includes a navigation module 14. The control module 4 can, for example, read the pre-planned driving route of the vehicle from the navigation module 14 and estimate the change trend of the azimuth relationship between at least one celestial body and the vehicle within a determined time period. Then, the control module 4 can generate a pose adjustment plan for the camera 2 according to this change trend, and on the premise of following the pose adjustment plan, fine-tune the angle and position of the camera according to the azimuth relationship. The pose adjustment plan can be, for example, a sequence of adjustment steps and a sequence of adjustment parameters (such as rotate left 15° - rotate right 20° - translate 3 cm to the left side of the vehicle) predicted for a determined road section or time period. If the vehicle travels according to the pre-planned driving route, the adjustment of the camera 2 generally will not deviate from this preliminary plan. On this basis, only fine-tuning of the preliminary plan needs to be combined with the specific azimuth and movement conditions of the vehicle, so that not only can more accurate tracking of the celestial body to be observed be achieved, but also the camera can be placed in a ready state before the vehicle undergoes significant direction changes or speed changes to adapt to this upcoming sudden angle switch.

[0068] In addition, the celestial body observation system 1 also optionally includes an image recognition module 21. The image recognition module 21 can be integrated in the camera 2 or the control module 4, for example. The image recognition module 21 is configured to recognize at least one celestial body to be observed in the sky image captured by the camera 2. At this time, the control module 4 is also configured to control the movement of the camera 2 to track and photograph the at least one recognized celestial body. The control module 4 can also be configured to re-adjust the pose of the camera 2 relative to the vehicle in the case where at least one celestial body to be observed is not recognized, or to control the output module 3 to output a celestial body observation view in the vehicle based on another source image in the case where at least one celestial body to be observed is not recognized a predetermined number of times.

[0069] In one example, the control module 4 controls the output of the celestial body observation view in the vehicle at least in a real scene mode and a non-real scene mode. In the real scene mode, the control module 4 causes the output module 3 to output the celestial body observation view in the vehicle based on the sky image captured in real time by the camera 2. In the non-real scene mode, the control module 4 causes the output module 3 to output the celestial body observation view in the vehicle based on another source image, and this another source image is different from the sky image captured in real time by the camera. Specifically, the another source image can be, for example, a sky image captured or recorded in advance (such as a few hours or days ago) by the camera 2, and at this time the another source image is non-real-time. In another example, the another source image can also be image or video data received from outside the vehicle through the communication interface 5, which includes, for example:

[0070] - Sky images taken in real time by another camera within a predetermined range around the vehicle;

[0071] - Sky images shared by the social media friends of the vehicle user;

[0072] - Celestial body images or sky images that can be observed from the current position of the vehicle obtained by accessing astronomical websites;

[0073] - Starry sky atmosphere images; and / or

[0074] - Celestial body popular science images.

[0075] In one example, the control module 4 can control the enabling, disabling, and switching between different output modes of a specific output mode considering various factors. For this purpose, the celestial observation system 1 may also include, for example, a user input module 11, a weather acquisition module 15, a personnel monitoring module 16, and a vehicle function linkage module 17. The control module 4 is respectively connected to these modules to receive conditions that may affect the output of the celestial observation view.

[0076] The user input module 11 is configured, for example, as an independent touch-based interaction interface, or can also be integrally configured with the interaction interface originally installed in the vehicle. The user input module 11 is configured to receive: a first specified input from the user regarding the output mode of the celestial observation view, a second specified input from the user regarding the output position of the celestial observation view in the vehicle, and a third specified input from the user regarding the type of celestial body to be observed. Then, the control module 4 controls the output of the celestial observation view based on the user input detected by the user input module 11. Specifically, the control module 4 is configured, for example, to select to output the celestial observation view in a real scene mode or a non-real scene mode according to the first specified input of the user, and control the output module 3 according to the second specified input of the user to output the celestial observation view at the output position specified by the user. In addition, the control module 4 can also control the predetermined movement of the camera 2 and / or the output of the output module 3 according to the third specified input of the user, so that the celestial observation view output in the vehicle includes the type of celestial body specified by the user. As an example, the types of celestial bodies to be observed include the moon, starry sky, constellations, celestial bodies (satellites, comets, stars, planets), auroras, or a combination of one or more of them.

[0077] The control module 4 can also collect the motion state of the vehicle from the vehicle motion determination module 13, and thereby control the enabling permission of different output modes or recommend a suitable output mode. For example, the control module 4 is configured to prohibit the output of the celestial observation view in the vehicle when the driving speed of the vehicle is greater than 30 km / h, or only allow the output of the celestial observation view in a non-real scene mode.

[0078] The weather acquisition module 15 is configured to be able to acquire the weather conditions at the location where the vehicle is located, which includes not only weather conditions (such as sunny, rainy, cloudy, etc.), but also visibility information (pollution level). Thus, when the control module 4 has the information about the candidate celestial bodies that can be observed from the current position of the vehicle, it can also combine the weather conditions to determine whether these celestial bodies can actually be observed from the current position of the vehicle. For example, although some celestial bodies are theoretically visible to observers in the area where the vehicle is located, if the weather conditions do not meet the preset requirements, these celestial bodies cannot be identified from the real-time captured sky images. Therefore, for example, only the celestial body observation view is allowed to be output in a non-real-scene mode.

[0079] The personnel monitoring module 16 includes, for example, one or more in-vehicle cameras arranged in the vehicle cabin and is configured to be able to monitor the human characteristic information of the passengers, which includes, for example, the age information, gender information, and emotion information of the personnel. In addition, the personnel monitoring module 16 may also include a seat occupancy status sensor to obtain the distribution of passengers in the vehicle. By knowing this information, the control module 4 can personalizedly recommend the output mode, output position, and / or output effect of the celestial body observation view in the vehicle. For example, if there are children among the vehicle passengers, the voice commentary of the celestial body observation view is automatically enabled, and the real sky image to be output is rendered to enrich the animation effect. If the vehicle is in motion and there are only passengers in the rear row except the driver, the celestial body observation view can be projected only to the rear row of the vehicle.

[0080] The vehicle function linkage module 17 is configured to be able to acquire the activation status of at least one predefined function of the vehicle, and this activation status is also provided to the control module 4, so that the control module 4 can control the output of the celestial body observation view in association with the predefined function. Specifically, the control module 4 can automatically trigger the output of the celestial body observation view in response to the activation of the predefined function of the vehicle. For example, there are multiple atmosphere functions preset in the vehicle (such as the proposal function, anniversary reminder function, festival atmosphere function). When a certain one of these preset functions is activated by the user, to further create a romantic atmosphere, the control module 4 can automatically project the celestial body observation view into the vehicle cabin so that the passengers can be in a sea of stars or under the moonlight.

[0081] In addition, the control module 4 can also upload the sky image taken by the camera together with the additional information such as geographic coordinates, time, date, weather, etc. to the cloud, transmit it to the mobile terminal device of the vehicle user and / or share it with another vehicle by means of the communication interface 5. In addition, the control module 4 can also receive a celestial observation sharing request from the outside of the vehicle by means of the communication interface 5, and generate a prompt to enable the non-real scene mode or generate a prompt to switch from the real scene mode to the non-real scene mode in response to receiving such a sharing request. For example, a friend of the vehicle user initiates real-time sharing through social media to share the starry sky image taken during his overseas travel. However, at this time, the vehicle user is enjoying the local night sky in the real scene mode, so the system will push a prompt to the user "XX has initiated a real-time moon viewing request, do you want to switch to the non-real scene mode?" If a positive response to this request from the vehicle user is received, the moon scene picture shared by the friend in real time can be output in the vehicle.

[0082] refer to Figure 1 , the control module 4 can also coordinately control the output module 3 and at least one cabin component 31, 32, 33 of the vehicle, so that the output of the celestial observation view triggers a state change of at least one cabin component in time association with the output of the celestial observation view. The at least one cabin component includes, for example, a seat 31, an audio system 32, an ambient light 33 and / or an air conditioner of the vehicle. For example, synchronously with the output of the celestial observation view or sequentially in time, the control module 4 can control the vehicle seat to move backward / forward as a whole, and pivot the backrest of the vehicle seat backward to a preset position, so that passengers can enjoy celestial observation in a more comfortable state. For another example, during the output of the celestial observation view, the ambient light in the cabin can be dimmed or the color can be changed, the air outlet direction and air volume of the air conditioner can be changed, and the audio system can be controlled to play soothing music to enhance the immersion during celestial observation.

[0083] Figures 2a - 2c A schematic diagram of outputting a celestial observation view in a vehicle is shown in an exemplary scenario.

[0084] refer to Figures 2a - 2c The camera 2 can be movably mounted on the outer roof of the vehicle 100. Driven by the motion execution unit, the camera 2 can not only move up and down along the height direction of the vehicle, move along the horizontal direction, but also move around Figure 2a The x, y, and z axes shown as examples are rotated within a preset angle.

[0085] exist Figure 2aIn the illustrated scene, the celestial body observation function has not yet been triggered in the vehicle 100. At this time, the camera 2 is maintained in an initial position relative to the vehicle 100, and the direction of the viewing angle range 210 of the camera 2 in the initial position is also shown accordingly. At this time, the camera 2 is in a standby or dormant state, for example, and the position and backrest of the seat 31 of the vehicle user 51 are both in the position originally set by the user.

[0086] exist Figure 2b In the illustrated scenario, the celestial observation function is triggered when the vehicle 100 is parked. It should be noted that the triggering of the celestial observation function can be initiated by the vehicle user 51, or can be triggered by the system (for example, in conjunction with the activation status of other predefined functions of the vehicle), or can be automatically triggered when the vehicle configuration meets the preset requirements. At this time, for example, it is known that the vehicle user 51 wants to watch the shape of the moon that night in the cabin, so the control module (not shown for simplicity) calculates the position relationship of the moon 300 relative to the vehicle in combination with the vehicle position information, and controls the camera 2 to view the moon from the vehicle. Figure 2a The initial pose shown is adjusted to Figure 2b The target posture shown. When the camera 2 is in the initial posture, its viewing angle range 210 is basically kept parallel to the roof, and the moon 300 to be observed does not fall into its viewing angle range 210. When the camera 2 is in the target posture relative to the vehicle 100, its viewing angle range 210 is aimed at the area where the moon 300 is located, so that the moon 300 can be captured. In one example, the control module is configured not to turn on the camera 2 when the camera 2 has not yet reached the target posture, and only turn on the camera 2 and control it to shoot after reaching the target posture. After the sky image is shot with the help of the camera 2, it can be projected to a specific area 301 of the ceiling in the car with the help of the output module 3. In addition, after the celestial body observation function is triggered, the seat 31 of the vehicle 2 can also be controlled in a coordinated manner so that the backrest of the seat 31 is pivoted backward by a preset angle, so that the vehicle user 51 can maintain a more comfortable posture to watch the shape of the moon.

[0087] exist Figure 2c In the illustrated scene, the celestial body observation function is triggered when the vehicle 100 is in motion. As the driving direction of the vehicle changes continuously during driving, the position of the moon 300 relative to the vehicle 100 also changes accordingly. Figure 2c In this case, the control module additionally adjusts the position of the camera 2 according to the vehicle's driving speed, driving direction and pre-planned driving route, so that its viewing angle range 210 dynamically adapts to the change in the position relationship of the celestial body 300 to be observed relative to the vehicle 100. Figure 2cIt can also be seen that since the vehicle 100 is in motion, for safety reasons, the backrest of the seat 31 of the front-row vehicle user 51 (such as the driver) is not lowered, but only the backrest inclination of the seat 31' of the rear-row vehicle user 52 is adjusted. At the same time, the output module 30 is also used to adjust the projection area 302 of the celestial observation view in the vehicle 100, so that the projection position is as close as possible to the rear part of the vehicle ceiling, thus not affecting the safe driving of the vehicle driver 51.

[0088] Figures 3a - 3f Schematic diagrams of the interfaces of the celestial observation views output in the vehicle are shown in the real scene mode and the non-real scene mode.

[0089] In Figure 3a and Figure 3b , the sky image captured by the camera in real time is directly output as the celestial observation view. In Figure 3a , the vehicle user specifies to observe the shape of the moon 301 in the vehicle, so the control module controls the camera to capture the moon 301 in the night sky and displays the sky image containing the moon 301 on the interface 41. In Figure 3b , the vehicle user specifies to observe the stars 302 that form the Cancer constellation in the vehicle, so a real-time sky image containing the Cancer constellation stars 302 is shown on the interface 41. To highlight the celestial bodies 301, 302 that the user expects to observe, the control module, for example, extracts a partial section from the sky image captured by the camera, and then only displays the partial section on the interface 41. In this partial section, the celestial bodies 301, 302 to be observed are, for example, located in the center of the screen and the proportion of the screen they occupy exceeds a preset value.

[0090] In Figure 3c and Figure 3d , the celestial observation view is also output in the real scene mode, but at this time, the original captured sky image is no longer directly output, but is pre-processed before output. As Figure 3c shown, geometric figures are superimposed on the moon 301 in the original image, so that the lunar phase shape can be more clearly recognized. In Figure 3d , on the basis of outputting the original sky image containing the Cancer constellation stars 302 on the interface 41, each individual star is connected one by one with a connecting line 303 to outline the constellation contour, and the corresponding constellation name is marked with text 304 beside it.

[0091] Generally speaking, the control module can label the celestial bodies in the celestial observation view in the following aspects:

[0092] - Celestial body categories, such as the moon, starry sky, constellations, celestial bodies (satellites, comets, stars, planets), auroras;

[0093] - Celestial body names (such as the moon, Polaris, etc.);

[0094] - Celestial body outlines or connecting lines;

[0095] - Lunar phases (new moon, first quarter moon, full moon, etc.);

[0096] - Constellation names;

[0097] - Constellation introductions.

[0098] In Figure 3e and Figure 3f the celestial body observation view of the vehicle is output in the non-real scene mode. In the scene shown in Figure 3e , due to cloud cover, the target celestial body cannot be continuously observed in the real scene mode. So, it switches to the non-real scene mode and instead shows a lunar phase diagram 305 divided according to the lunar solar terms on interface 41 with an audio commentary. In Figure 3f , the forms and popular science introductions of multiple constellations 306 are shown on interface 41.

[0099] Figures 4a - 4d Shows a schematic diagram of the user changing the output effect of the celestial body observation view through gesture interaction.

[0100] Refer to Figure 4a , the user completes gesture 61 on the celestial body observation view shown on interface 41 to select the part of the celestial body that they want to learn more about in detail. At this time, the user performs this gesture 61, for example, by pointing a finger at a specific area. The user input module monitors this gesture 61 of the user, and then the control module interprets the corresponding intention. In response to recognizing the user's "selection" intention, the control module changes the output effect of the celestial body observation view so that the celestial body corresponding to the selected area is enhanced and marked with explanatory information, which is correspondingly shown in Figure 4b .

[0101] Refer to Figure 4c , the user completes gesture 62 on the celestial body observation view shown on interface 41. At this time, the user completes this gesture 62 by pinching with two fingers. Similarly, the control module interprets the user's "zoom out the picture" intention, and then controls the movement of the camera or the adjustment of the focal length to perform a zoom operation on the real-time captured picture, which is correspondingly shown in Figure 4d . In addition, by appropriately controlling the output unit, the celestial body observation view to be output can also be rotated, translated, or scaled according to the user's image transformation intention.

[0102] Figure 5 Shows a schematic diagram of the interface of the user input module according to an exemplary embodiment of the present invention.

[0103] In the interface 110 of the user input module 11, input options in the form of multiple virtual keys 111, 112, 113, 114, 115 are shown. By manipulating these virtual keys, vehicle users can activate the celestial observation function in the vehicle and customize their favorite output effects.

[0104] By manipulating the key 111, it is possible to select between the "non-real scene mode" and the "real scene mode", thereby determining the image source for outputting the celestial observation view in the vehicle.

[0105] By manipulating the key 112, it is possible to select the celestial object category to be observed from options such as "observing the moon", "observing the starry sky", "observing constellations", and "observing the aurora". Under the "observing the starry sky" option, for example, all the stars that can be observed from the current position of the vehicle are selected as the celestial objects to be observed. Under the "observing constellations" option, multiple star clusters that form a specific complete constellation are selected as the celestial objects to be observed.

[0106] By manipulating the key 113, it is possible to select the desired output effect from options such as "viewing the scenery", "atmosphere", and "popular science". For example, if the user selects the "viewing the scenery" option, only the originally captured sky image can be output. If the user selects the "atmosphere" option, the originally captured sky image can be rendered or virtualized, or, if the user has previously selected the "non-real scene mode", the starry atmosphere image can be downloaded through the communication interface. If the user selects the "popular science" option, regardless of the output mode, several celestial objects in the celestial observation view can be marked and accompanied by audio or text introductions about the celestial objects to be observed.

[0107] By manipulating the key 114, it is possible to select the desired projection position for outputting the celestial observation view in the vehicle from options such as "roof", "HUD", and "left-side glass of the vehicle".

[0108] By manipulating the key 115, it is possible to control the activation, pause, and exit of the celestial observation function in the vehicle. If the user directly presses the "activate" option in the key 115 without making personalized selections for the keys 111, 112, 113, 114, the output mode, celestial objects to be observed, output effect, and output position can be selected according to the system preset configuration. In addition, as Figure 1 elaborated in detail, it is also possible to intelligently recommend the output mode, output effect, and output position to the user according to various factors such as weather conditions, vehicle motion state, image recognition results, and personnel monitoring results.

[0109] Figure 6 The flowchart of a celestial observation method for a vehicle according to an exemplary embodiment of the present invention is shown. In Figure 6In the illustrated embodiment, the method exemplarily includes steps S01 - S60, and can be implemented, for example, when using Figure 1 the celestial observation system 1 shown.

[0110] In an optional step S01, an output mode for outputting a celestial observation view in a vehicle is obtained. Optionally, in this step, the output effect, output position of the celestial observation view, and the category of the celestial body to be observed can also be obtained according to user input.

[0111] In the following steps, the output of the celestial observation view in the vehicle is controlled at least in a real - scene mode and a non - real - scene mode. In the real - scene mode, the output module outputs the celestial observation view in the vehicle based on the sky image captured in real time by the camera. In the non - real - scene mode, the output module outputs the celestial observation view in the vehicle based on another source image, where the another source image is different from the sky image captured in real time by the camera.

[0112] Taking the real - scene mode as an example, the process of outputting the celestial observation view in the vehicle is introduced in steps S10 - S60.

[0113] In step S10, the orientation information of the vehicle is obtained. In one example, the driving speed, driving direction, and pre - planned driving route of the vehicle can also be obtained in this step.

[0114] In step S20, based on the orientation information of the vehicle, the orientation relationship of at least one celestial body to be observed relative to the vehicle is determined.

[0115] In step S30, according to the orientation relationship, the pose of the camera relative to the vehicle is adjusted so that at least one celestial body is within the viewing range of the camera. For example, the rotation angle of the camera can be calculated, and the camera is controlled to turn through this angle so that the viewing range faces the celestial body to be observed.

[0116] In step S40, the sky image outside the vehicle is captured by the camera. For example, the camera can be turned on and the sky image can be captured after the camera is placed in the target pose relative to the vehicle. It is also possible that the camera remains in the active state all the time and continuously captures the sky image.

[0117] In an optional step S50, it is checked whether at least one celestial body to be observed can be recognized in the sky image captured by the camera. In the case where at least one celestial body to be observed is not recognized, steps S20 - S40 can be repeated to re - adjust the pose of the camera relative to the vehicle according to the orientation information of the vehicle.

[0118] In the case where at least one celestial body to be observed is recognized, in step S60, a celestial body observation view is output in the vehicle based on a sky image captured by means of a camera. For example, the celestial body observation view can be output to a region specified by a user.

[0119] Although specific embodiments of the present invention have been described in detail herein, they are given for illustrative purposes only and should not be considered as limiting the scope of the present invention. Various substitutions, alterations, and modifications can be conceived without departing from the spirit and scope of the present invention.

Claims

1. An astronomical observation system (1) for a vehicle, the astronomical observation system (1) comprising: A camera (2) movably mounted on a vehicle (100) and configured to be able to capture an image of the sky outside the vehicle (100); a vehicle positioning module (12) configured to be able to obtain the orientation information of the vehicle (100); an image recognition module (21) configured to be able to recognize at least one celestial body (300) to be observed in the sky image captured by the camera (2), wherein the category of the at least one celestial body (300) to be observed is obtained according to user input; a control module (4) configured to be able to determine the orientation relationship of the at least one celestial body to be observed relative to the vehicle (100) according to the orientation information of the vehicle (100), and adjust the pose of the camera (2) relative to the vehicle (100) according to the orientation relationship, so that at least one celestial body (300) is within the viewing range (210) of the camera (2), wherein, in the case where at least one celestial body (300) to be observed is not recognized, the pose of the camera (2) relative to the vehicle (100) is readjusted, and the control module (4) is further configured to control the output of the celestial body observation view in the vehicle (100) in at least a real scene mode and a non-real scene mode, wherein, in the real scene mode, the output module (3) outputs the celestial body observation view in the vehicle (100) based on the sky image captured by the camera (2) in real time, and in the non-real scene mode, the output module (3) outputs the celestial body observation view in the vehicle (100) based on another source image different from the sky image captured by the camera (2) in real time, and the control module (4) is further configured to receive a celestial body observation sharing request from outside the vehicle through a communication interface (5), and generate a prompt to enable the non-real scene mode or generate a prompt to switch from the real scene mode to the non-real scene mode in response to receiving the celestial body observation sharing request; and an output module (3) configured to be able to output the celestial body observation view in the vehicle (100) based at least on the sky image captured by the camera (2).

2. The astronomical observation system (1) according to claim 1, wherein, The celestial body observation system (1) further includes a vehicle motion determination module (13) configured to be able to obtain the driving direction and / or driving speed of the vehicle (100); wherein, the control module (4) is further configured to be able to determine the dynamic change of the orientation relationship of the at least one celestial body (300) relative to the vehicle (100) according to the driving direction and / or driving speed of the vehicle (100), and additionally adjust the pose of the camera (2) relative to the vehicle (100) according to the dynamic change of the orientation relationship.

3. The celestial body observation system (1) according to claim 1 or 2, wherein, The celestial body observation system (1) further includes an image recognition module (21) configured to be able to recognize at least one celestial body (300) to be observed in the sky image captured by the camera (2); wherein, the control module (4) is further configured to adjust the pose of the camera (2) relative to the vehicle (100) according to the recognition result of the image recognition module (21) so that the camera (2) tracks and captures the at least one recognized celestial body (300).

4. The celestial body observation system (1) according to any one of claims 1 to 3, wherein, Wherein, The control module (4) is further configured to: - control the output module (3) to output an astronomical object observation view in the vehicle (100) based on another source image when at least one astronomical object (300) to be observed is not recognized a predetermined number of times, the another source image being different from the sky image captured in real time by the camera (2).

5. The astronomical observation system (1) according to any one of claims 1 to 4, wherein, The control module (4) is further configured to be able to extract a partial section from the sky image captured by the camera (2) such that the partial section only includes at least one astronomical object (300) to be observed, and / or such that the proportion of the at least one astronomical object (300) to be observed in the partial section exceeds a preset value; and / or the output module (3) is further configured to output an astronomical object observation view based on the partial section extracted from the sky image.

6. The celestial body observation system (1) according to any one of claims 1 to 5, wherein, The control module (4) is further configured to control the output module (3) to output an astronomical object observation view according to a determined output effect, wherein outputting an astronomical object observation view according to a determined output effect includes: - directly outputting the sky image captured by the camera (2) as the astronomical object observation view; - performing annotation on at least one astronomical object (300) in terms of shape and / or category in the sky image captured by the camera (2), and outputting the annotated sky image as the astronomical object observation view; and / or - synchronously outputting introductory information in text form and / or voice form about at least one astronomical object (300) to be observed along with the output of the sky image.

7. The celestial body observation system (1) according to any one of claims 1 to 6, wherein, In the non-real scene mode, the another source image includes: a sky image captured or recorded in advance by the camera (2); and / or a sky image, a starry sky atmosphere image and / or an astronomical object popular science image pre-stored locally in the vehicle (100) and / or received from outside the vehicle (100).

8. The celestial body observation system (1) according to any one of claims 1 to 7, wherein, The control module (4) is further configured to be able to control the output module (3) to output an astronomical object observation view according to the motion state of the vehicle (100), wherein the control module (4) is configured to prohibit the output of an astronomical object observation view in the vehicle (100) or only allow the output of an astronomical object observation view in the non-real scene mode when the driving speed of the vehicle (100) is greater than a threshold.

9. The celestial body observation system (1) according to any one of claims 1 to 8, wherein, The control module (4) is further configured to be able to control the output module (3) to output an astronomical object observation view according to weather conditions, wherein the control module (4) is configured to only allow the output of an astronomical object observation view in the non-real scene mode when the weather conditions do not meet the preset requirements.

10. The celestial body observation system (1) according to any one of claims 1 to 9, wherein, The control module (4) is further configured to be able to recommend the output mode of the astronomical object observation view, the output position and / or the output effect of the astronomical object observation view in the vehicle (100) personalized according to the identity information of the vehicle user (51).

11. The celestial body observation system (1) according to any one of claims 1 to 10, wherein, The control module (4) is further configured to be able to cooperatively control the output module (3) and at least one cabin component (31) of the vehicle (100) such that a state change of the at least one cabin component (31) is triggered in time association with the output of the astronomical object observation view, the at least one cabin component (31) including the seat, the ambient light, the audio and / or the air conditioner of the vehicle (100).

12. The celestial body observation system (1) according to any one of claims 1 to 11, wherein, The celestial body observation system (1) further includes a user input module (11), which is configured to be capable of receiving: - a first specified input from a vehicle user (51) regarding the output mode of the celestial body observation view, - a second specified input from the vehicle user (51) regarding the output position of the celestial body observation view in the vehicle (100), and / or - a third specified input from the vehicle user (51) regarding the category of celestial bodies to be observed; wherein, the control module (4) is further configured to: - select to output the celestial body observation view in a real scene mode or a non-real scene mode according to the first specified input of the vehicle user (51), - control the output module (3) according to the second specified input of the vehicle user (51) to output the celestial body observation view at the output position specified by the vehicle user (51), and / or - control the pose of the camera (2) relative to the vehicle (100) and / or the output of the output module (3) according to the third specified input of the vehicle user (51), so that the celestial body observation view output in the vehicle (100) includes the category of celestial bodies specified by the vehicle user (51).

13. The celestial body observation system (1) according to any one of claims 1 to 12, wherein, The celestial body observation system (1) further includes a user input module (11), which is configured to be capable of receiving a gesture input (61) from the vehicle user (51) regarding the celestial body observation view that has been output in the vehicle (100); wherein, the control module (4) is further configured to: - identify the selected area of the vehicle user (51) in the celestial body observation view according to the gesture input (61) of the vehicle user (51), and control the output module (3) to output the introductory information of the celestial body corresponding to the selected area; and / or - identify the image transformation intention according to the gesture input (61) of the vehicle user (51), and control the movement of the camera (2) and / or the output of the output module (3), so that the output celestial body observation view performs rotation, translation, and / or scaling operations corresponding to the image transformation intention.

14. The celestial body observation system (1) according to any one of claims 1 to 13, wherein, The celestial body observation system (1) further includes a communication interface (5), and the communication interface (5) is configured to be capable of uploading the sky image captured by the camera (2) to the cloud, transmitting it to the mobile terminal device of the vehicle user (51), and / or sharing it with another vehicle.

15. A method for celestial body observation for a vehicle (100), wherein, The celestial body observation method is performed by using the celestial body observation system according to any one of claims 1 to 14, and the celestial body observation method includes the following steps: obtaining the orientation information of the vehicle (100); determining the orientation relationship of at least one celestial body (300) to be observed relative to the vehicle (100) according to the orientation information of the vehicle (100), and adjusting the pose of the camera (2) relative to the vehicle (100) according to the orientation relationship, so that at least one celestial body (300) is within the viewing range (210) of the camera (2); capturing a sky image outside the vehicle (100) by using the camera (2); and outputting a celestial body observation view in the vehicle (100) at least based on the sky image captured by the camera (2).

16. The celestial body observation method according to claim 15, wherein, The celestial body observation method further includes the following steps: controlling the output of the celestial body observation view in the vehicle (100) at least in a real scene mode and a non-real scene mode, wherein, in the real scene mode, the output module (3) outputs the celestial body observation view in the vehicle (100) based on the sky image captured in real time by the camera (2), and in the non-real scene mode, the output module (3) outputs the celestial body observation view in the vehicle (100) based on another source image, and the another source image is different from the sky image captured in real time by the camera (2).

17. A machine-readable storage medium, on which a computer program is stored, and the computer program is used to execute the celestial body observation method according to claim 15 or 16 when running on a computer.

Images