Vehicle sky screen display method, sky screen system and vehicle

Abstract

The invention relates to a vehicle skyscreen display method, a skyscreen system and a vehicle, and the method comprises the steps: obtaining a vehicle coordinate outputted by a positioning system, and determining the administrative division level of the vehicle according to the vehicle coordinate; acquiring a landmark pattern matched with the landmark in the administrative division hierarchy; according to the user posture in the vehicle, the vehicle coordinates, the vehicle body posture of the vehicle and the current ambient light, the display brightness of the landmark pattern is adjusted; displaying the landmark pattern in the sky screen of the vehicle according to the display brightness; and the scene interactivity of the skyscreen is improved.

Description

Technical Field

[0001] This application relates to the field of vehicle sunroofs, and in particular to a vehicle sunroof display method, sunroof system, and vehicle. Background Technology

[0002] With the development of the automotive industry, more and more cars are equipped with panoramic sunroofs, and ambient lighting is deployed in the sunroofs to project fixed starry sky patterns onto the roof lining, creating a visual effect of stars or constellations.

[0003] Traditional dome displays can achieve starlight brightness adjustment or dynamic flashing effects through simple control modules, but the overall display content (such as star arrangement and constellation shape) is usually a preset template and cannot be dynamically adjusted. Moreover, traditional dome displays sometimes suffer from glare or dimness, resulting in poor user visual comfort.

[0004] Currently, there is no effective solution to the problem of limited scene interactivity in automotive sunroofs in related technologies. Summary of the Invention

[0005] Therefore, it is necessary to provide a vehicle sunroof display method, sunroof system, and vehicle that can improve the scene interactivity of the sunroof, addressing the aforementioned technical problems.

[0006] In a first aspect, this application provides a method for displaying a vehicle sunroof, including:

[0007] Obtain the vehicle coordinates output by the positioning system, and determine the administrative division level of the vehicle based on the vehicle coordinates;

[0008] Obtain landmark patterns that match the landmarks in the administrative division level;

[0009] The display brightness of the landmark pattern is adjusted based on the user's posture in the vehicle, the vehicle's coordinates, the vehicle's body posture, and the current ambient light.

[0010] The landmark pattern is displayed on the vehicle's skylight according to the specified display brightness.

[0011] In some embodiments, the positioning system includes a GPS module and an inertial measurement unit; acquiring the vehicle coordinates output by the positioning system includes:

[0012] The GPS module is subjected to a validity test to determine whether the GPS module is valid;

[0013] If the GPS module is determined to be in a valid state, the GPS coordinates output by the GPS module will be used as the vehicle coordinates.

[0014] If the GPS module is determined to be in a malfunctioning state, the acceleration collected by the inertial measurement unit is read, the vehicle offset is calculated based on the acceleration, and the GPS coordinates last generated by the GPS module are updated based on the vehicle offset to obtain the vehicle coordinates.

[0015] In some embodiments, the positioning system further includes a first camera; if the GPS module is determined to be in a malfunctioning state, the method further includes:

[0016] Read the road image captured by the first camera, and extract road features based on the road image;

[0017] The road features and the vehicle offset are input into a Kalman filter to perform vehicle pose estimation.

[0018] The vehicle coordinates are determined based on the vehicle pose estimation results.

[0019] In some embodiments, the road features and the vehicle offset are input into a Kalman filter to perform vehicle pose estimation, and the vehicle coordinates are determined based on the vehicle pose estimation results, including:

[0020] Determine whether the distance between the vehicle coordinates determined based on the road features and the vehicle coordinates determined based on the vehicle offset is within a preset range;

[0021] If it is determined that the distance exceeds the preset range, then it is determined whether the noise of the road feature is less than the noise threshold.

[0022] If the noise of the road feature is determined to be less than the noise threshold, then the vehicle coordinates determined based on the road feature are used.

[0023] If the noise of the road feature is determined to be no less than the noise threshold, then the vehicle coordinates determined based on the vehicle offset are used.

[0024] In some embodiments, adjusting the display brightness of the landmark pattern based on the user's posture in the vehicle, the vehicle's coordinates, the vehicle's body posture, and the current ambient light includes:

[0025] The user posture data collected by the second camera in the vehicle is obtained, visual feature values ​​are extracted based on the user posture data, and a first weight parameter is determined based on the visual feature values.

[0026] Based on the vehicle coordinates, geographic information feature values ​​are extracted, and a second weighting parameter is determined based on the geographic information feature values.

[0027] The vehicle body attitude data collected by the inertial measurement unit in the vehicle is acquired, and the vehicle body attitude feature values ​​are extracted based on the vehicle body attitude data. A third weighting parameter is then determined based on the vehicle body attitude feature values.

[0028] The system acquires the light intensity data collected by the ambient light sensor in the vehicle, extracts ambient light feature values ​​based on the light intensity data, and determines a fourth weighting parameter based on the ambient light feature values.

[0029] The fusion decision value is obtained by fusing the visual feature value, the geographic information feature value, the vehicle posture feature value and the ambient light feature value according to the first weight parameter, the second weight parameter, the third weight parameter and the fourth weight parameter;

[0030] The display brightness of the landmark pattern is adjusted based on the fusion decision value.

[0031] In some embodiments, after adjusting the display brightness of the landmark pattern based on the fusion decision value, the method further includes:

[0032] Determine whether the driver is present and whether the vehicle is in motion;

[0033] If it is determined that the driver is present and the vehicle is in motion, then it is determined whether the display brightness does not exceed the brightness threshold.

[0034] If it is determined that the display brightness exceeds the brightness threshold, then the display brightness is adjusted to the brightness threshold.

[0035] In some embodiments, after displaying the landmark pattern on the vehicle's skylight according to the display brightness, the method further includes:

[0036] The distance between the vehicle and the landmark is calculated based on the vehicle coordinates, and the size of the landmark pattern displayed in the canopy is adjusted based on the distance.

[0037] In some embodiments, the method further includes:

[0038] Read the ADAS radar signal and the door status collected by the door sensor;

[0039] The ADAS radar signal is used to determine whether there is a risk of oncoming vehicle.

[0040] If a risk of oncoming traffic is detected, a red light pulse will be projected onto the skylight area on the side closest to the opened door when any door is opened.

[0041] Secondly, this application provides a canopy system, comprising: a positioning system, a canopy, and a central controller, wherein the canopy is equipped with an ambient light array, and the positioning system and the ambient light array are respectively connected to the central controller; wherein...

[0042] The positioning system is used to generate vehicle coordinates;

[0043] The central controller is used to receive the vehicle coordinates and execute the vehicle panoramic display method described in the first aspect based on the vehicle coordinates.

[0044] The ambient light array is used to display landmark patterns under the control of the central controller.

[0045] Thirdly, this application provides a vehicle, including: a vehicle body and the panoramic sunroof system described in the second aspect above, wherein the panoramic sunroof system is installed in the vehicle body.

[0046] The aforementioned vehicle sunroof display method, sunroof system, and vehicle determine the landmark pattern to be displayed on the sunroof by responding to the vehicle's location. By combining the user's posture, vehicle coordinates, vehicle body posture, and current ambient light, the display brightness of the landmark pattern is automatically adjusted to reduce glare or darkness. While achieving dynamic adaptation of the sunroof landmark pattern, the user's visual comfort is improved, thereby enhancing the scene interactivity of the sunroof. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of the hardware structure of the canopy system in one embodiment;

[0048] Figure 2 This is a schematic diagram of the architecture of the canopy system in one embodiment;

[0049] Figure 3 This is a flowchart illustrating a vehicle sunroof display method in one embodiment;

[0050] Figure 4 This is a flowchart illustrating a vehicle coordinate calculation method in one embodiment;

[0051] Figure 5 This is a flowchart illustrating a vehicle coordinate calculation method based on IMU and visual SLAM in one embodiment.

[0052] Figure 6 This is a timing diagram of a vehicle sunroof display method in one embodiment;

[0053] Figure 7 This is a flowchart illustrating a method for adjusting the brightness of a dome display in one embodiment;

[0054] Figure 8 This is a timing diagram of a lateral approach vehicle warning triggering method in one embodiment. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0056] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0057] In one embodiment, a vehicle is provided, including: a vehicle body and a panoramic sunroof system, the panoramic sunroof system being installed in the vehicle body. Figure 1 This is a schematic diagram of the hardware structure of the canopy system, such as... Figure 1As shown, the system includes a positioning system 100, a canopy 200, and a central controller 300. The canopy 200 is equipped with an ambient light array 201. The positioning system 100 and the ambient light array 201 are connected to the central controller 300. The positioning system 100 is used to generate vehicle coordinates. In some embodiments, the positioning system 100 includes a GPS module. In some embodiments, the positioning system 100 includes a GPS module and an inertial measurement unit (IMU). In some embodiments, the positioning system 100 includes a GPS module, an inertial measurement unit, and a first camera. The central controller 300 receives the vehicle coordinates and executes a vehicle canopy display method based on the vehicle coordinates. The ambient light array 201 displays landmark patterns under the control of the central controller 300. The ambient light array 201 can be an LED light group or a fiber optic light group.

[0058] In one embodiment, Figure 2 A schematic diagram of the architecture of a canopy system is provided, such as... Figure 2 As shown, the system comprises a perception layer, a decision-making layer, and an execution layer. The following sections describe each architectural layer.

[0059] The perception layer includes a positioning system 100, an ambient light sensor 400, and an ADAS radar 500. The positioning system 100 includes a vision module 101, a GPS module 102, and an IMU 103. The vision module 101 may include a first camera and a second camera. The first camera captures the vehicle's external environment, while the second camera, installed in the cabin, tracks eye contact and detects posture, capturing the driver's or passenger's gaze direction and posture. The GPS module 102 and IMU 103 are used for vehicle positioning and attitude detection, providing accurate position and motion information. The ambient light sensor 400 monitors the light intensity inside and outside the vehicle, providing a basis for subsequent light adjustment. The ADAS radar 500 detects oncoming vehicle signals and driving status to ensure driving safety.

[0060] The decision-making layer includes a central controller 300. The central controller 300 receives various data from the perception layer and performs comprehensive processing and decision-making. In the decision-making layer, the central controller 300 analyzes the data collected by the perception layer to determine the current scene type, including landmark proximity, user behavior analysis, and security risk assessment. Specifically, when a landmark proximity scene is determined, the system loads the corresponding landmark database and prepares to display relevant landmark information. Based on the results of user behavior analysis, the system uses a dynamic dimming algorithm to automatically adjust the brightness and color of the canopy to adapt to different usage needs and environmental changes. Once a security risk is detected, the system immediately activates a red light warning mechanism, issuing a warning to the user through a red light projector to avoid potential dangers.

[0061] The execution layer includes an ambient light array 201, a dimming driver 600, and a red light projector 700. The ambient light array 201 displays landmark images and provides lighting effects to the user. The dimming driver 600 adjusts the light on the canopy as needed to achieve dynamic effects. The red light projector 700 projects red light warnings into specific areas when the system detects a safety risk, alerting the user.

[0062] In one embodiment, Figure 3 A flowchart illustrating a method for displaying a vehicle panoramic sunroof is provided, such as... Figure 3 As shown, the method includes the following steps:

[0063] Step S101: Obtain the vehicle coordinates output by the positioning system, and determine the administrative division level of the vehicle based on the vehicle coordinates.

[0064] The vehicle's current geographical location is obtained by acquiring the vehicle coordinates output by the positioning system. The positioning system can output the vehicle coordinates in the following ways:

[0065] In some embodiments, the positioning system includes a GPS module, which can use the GPS coordinates generated by the GPS module as vehicle coordinates.

[0066] In some embodiments, the positioning system includes a GPS module and an inertial measurement unit, and the GPS module and the inertial measurement unit can be combined to generate vehicle coordinates.

[0067] In some embodiments, the positioning system includes a GPS module, an inertial measurement unit, and a first camera, and can combine the GPS module, the inertial measurement unit, and the first camera to generate vehicle coordinates.

[0068] This embodiment does not restrict the output method of vehicle coordinates.

[0069] Administrative divisions can be cities, urban districts, counties, villages, etc. Taking cities as an example, when a vehicle enters the outer ring road of Shanghai, and its current coordinates are (31.24'N, 121.50'E), which are within the latitude and longitude range of Shanghai (30.40'N~31.53'N, 120.52'E~122.12'E), it can be determined that the vehicle is currently in Shanghai.

[0070] Step S102: Obtain the landmark pattern that matches the landmark in the administrative division level.

[0071] Each city has its own landmarks. A landmark database can be set up in the system to store city geographical locations and landmark patterns. City geographical locations can be represented by latitude and longitude ranges. By matching vehicle coordinates with various latitude and longitude ranges in the landmark database, the system can find the latitude and longitude range in which a vehicle falls and retrieve the landmark pattern corresponding to that range.

[0072] Step S103: Adjust the display brightness of the landmark pattern according to the user's posture in the vehicle, the vehicle's coordinates, the vehicle's body posture, and the current ambient light.

[0073] User posture data can be collected via a second camera deployed in the vehicle, vehicle body posture data can be collected via an inertial measurement unit, and light intensity data can be collected via an ambient light sensor. These data, combined with vehicle coordinates, extract feature information from each sensing data point, and fuse this feature information to obtain a fusion decision value. Based on this fusion decision value, the display brightness of the landmark pattern is calculated. This setup, adjusting display brightness through multimodal sensing data fusion, helps improve user visual comfort.

[0074] Step S104: Display the landmark pattern on the vehicle's skylight according to the display brightness.

[0075] An ambient light array is deployed within the canopy. A two-dimensional array stores the row and column indices of each luminous unit in the array and records the physical distance between adjacent luminous units. An origin is defined within the canopy, with the vehicle's forward direction as the Y-axis, the horizontal direction as the X-axis, and the physical distance between luminous units as the minimum unit distance, establishing a Cartesian coordinate system for the canopy. Landmark patterns are then mapped to this Cartesian coordinate system, and the luminous units at the corresponding positions in the system are illuminated based on the mapping results. The ambient light array can use LED lights or fiber optic lights; this implementation is not limited to either.

[0076] In the initial stage of vehicle movement, if the dome screen does not yet display any landmark patterns, the system will display the preset initial pattern. After the vehicle has been traveling for a period of time, the system obtains the current vehicle coordinates, identifies the corresponding landmark, and automatically switches the initial pattern to the landmark pattern corresponding to the current geographical location. As the vehicle continues to move, the system will automatically switch the landmark pattern based on the updated vehicle coordinates.

[0077] For example, the vehicle matches city landmarks (such as the Oriental Pearl Tower in Shanghai and Tianyi Pavilion in Ningbo) based on high-precision maps and GPS coordinates. Landmark patterns are pre-entered into the landmark database in the form of a coordinate matrix, such as {(0,1),(0,2),…(17,29)}. The LEDs embedded in the canopy correspond to the canopy's Cartesian coordinate system.

[0078] When the vehicle is started, it is located in Ningbo, and the landmark pattern corresponding to Ningbo City—Tianyi Pavilion pattern—is read from the landmark database and displayed on the sky screen.

[0079] When a user drives from Ningbo to Hangzhou and the GPS location changes to Hangzhou, the system reads the landmark image of Hangzhou—the Leifeng Pagoda image—from the landmark database and switches the Tianyi Pavilion image to the Leifeng Pagoda image.

[0080] When a user leaves Hangzhou, their GPS location changes, and no corresponding landmark pattern is found in the landmark database. In this case, a default pattern (such as a starry sky pattern) is displayed.

[0081] In some embodiments, after displaying the landmark pattern on the vehicle's skylight according to the display brightness, the distance between the vehicle and the landmark can be calculated based on the vehicle's coordinates, and the size of the landmark pattern displayed on the skylight can be adjusted according to the distance.

[0082] First, based on vehicle coordinates and pre-set landmark location data, the system calculates the real-time straight-line distance between the vehicle and the landmark using spatial geometric calculations. Then, the system dynamically adjusts the display size of the landmark image on the canopy according to this real-time straight-line distance. Specifically, when the vehicle approaches the landmark, the straight-line distance decreases, and the landmark image is displayed at a larger scale; when the vehicle moves away from the landmark, the straight-line distance increases, and the landmark image is displayed at a smaller scale. For example, assuming the straight-line distance is greater than 10 kilometers, the display scale is 50%; when the straight-line distance is between 5 and 10 kilometers, the display scale is 80%; and when the straight-line distance is within 5 kilometers, the display scale is 100%. This setting enhances the interactive nature of the canopy's scenes.

[0083] In steps S101 to S104 above, the landmark pattern to be displayed on the canopy is determined by responding to the vehicle's location. Combined with the user's posture, vehicle coordinates, vehicle body posture, and current ambient light, the display brightness of the landmark pattern is automatically adjusted to reduce glare or darkness. While achieving dynamic adaptation of the landmark pattern on the canopy, the user's visual comfort is improved, thereby enhancing the scene interactivity of the canopy.

[0084] If the GPS module is functional, it will generate and send GPS signals (i.e., GPS coordinates) to the central controller at a certain frequency. The central controller then controls the canopy to display the corresponding landmark patterns based on the received GPS signals. If the central controller does not receive a GPS signal, it will display a default pattern or no pattern at all. When a vehicle enters a tunnel, it may trigger a network terminal, causing the GPS module to lose its signal and temporarily malfunction. In this case, the landmark pattern display may become discontinuous.

[0085] To address this issue, in one embodiment, the positioning system includes a GPS module and an inertial measurement unit. Accordingly, Figure 4A flowchart illustrating a method for calculating vehicle coordinates is provided, as follows: Figure 4 As shown, obtaining the vehicle coordinates output by the positioning system includes the following steps:

[0086] Step S201: Perform a validity check on the GPS module to determine whether the GPS module is valid.

[0087] The central controller determines the functionality of the GPS module based on whether it receives GPS signals. If the central controller continuously receives GPS signals from the GPS module, it considers the GPS module to be valid; otherwise, it considers the GPS module to be faulty.

[0088] In step S202, if the GPS module is determined to be in a valid state, the GPS coordinates output by the GPS module are used as the vehicle coordinates.

[0089] Step S203: If the GPS module is determined to be in a malfunctioning state, the acceleration collected by the inertial measurement unit is read, the vehicle offset is calculated based on the acceleration, and the GPS coordinates generated by the GPS module in the last time are updated based on the vehicle offset to obtain the vehicle coordinates.

[0090] When the GPS module is in a failed state, record the last GPS coordinates generated by the GPS module, read the acceleration collected by the inertial measurement unit (IMU), calculate the vehicle offset from the time the GPS module failed to the current time based on the acceleration, update the last GPS coordinates generated by the GPS module based on the vehicle offset, and obtain the vehicle coordinates.

[0091] IMU dead reckoning is as follows:

[0092] Record the coordinates (x1, y1, z1) at the location where GPS fails, and obtain the acceleration value a_raw through the IMU, a_raw = [a_x, a_y, a_z]^T, which is the triaxial acceleration value at time T. Define the spatial vehicle speed v, and according to the spatial vector conversion, the vehicle speeds V_x, V_y, and V_z in the x, y, and z directions can be obtained. Therefore, the coordinate offset of the vehicle body after a time t is (V_x×t+x1, V_y×t+y1, V_z×t+z1).

[0093] In this embodiment, in the event of GPS module failure, the vehicle's geographical location is calculated by introducing an inertial measurement unit, thus ensuring the continuity of the landmark pattern display.

[0094] The GPS offset can be roughly determined using the algorithm described above. Under normal circumstances, due to the zero bias of the gyroscope in the IMU (Inertial Measurement Unit), i.e., the deviation caused by sensor accuracy, this bias gradually increases over time. This means the inertial measurement unit has accumulated errors, resulting in a discrepancy between the vehicle offset calculated from acceleration and the actual offset, ultimately leading to inaccurate vehicle coordinates. To solve this problem, in one embodiment, the positioning system further includes a first camera. Figure 5 A flowchart illustrating a vehicle coordinate calculation method based on IMU and visual SLAM is provided, as follows: Figure 5 As shown, if the GPS module is determined to be in a malfunctioning state, the method also includes:

[0095] The system reads road images captured by the first camera and extracts road features based on these images. The road features and vehicle offset are then input into a Kalman filter to estimate the vehicle pose. The vehicle coordinates are determined based on the estimated pose. The road features can be administrative boundaries, such as city boundaries.

[0096] In this embodiment, it can be determined whether the distance between the vehicle coordinates determined based on road features and the vehicle coordinates determined based on vehicle offset is within a preset range; if it is determined that the distance exceeds the preset range, it is determined whether the noise of the road features is less than the noise threshold; if it is determined that the noise of the road features is less than the noise threshold, the vehicle coordinates determined based on the road features are used; if it is determined that the noise of the road features is not less than the noise threshold, the vehicle coordinates determined based on the vehicle offset are used.

[0097] In this embodiment, when the distance between the vehicle coordinates determined based on road features and the vehicle coordinates determined based on vehicle offset exceeds a preset range, there is a possibility that the inertial measurement unit has a large cumulative error, and it is necessary to correct it based on the vehicle coordinates determined based on road features.

[0098] Visual SLAM (Simultaneous Localization and Mapping) feature extraction is as follows:

[0099] For example, when a specific road sign is identified based on road features, such as the Hangzhou boundary, but the actual vehicle coordinates matched by the IMU do not fall within this area, the IMU error needs to be reset and re-accumulated. Another example is when vehicles are identified as passing through the same location based on road features; visual scene matching is used to correct IMU drift.

[0100] However, considering the noise inherent in road feature data, vehicle coordinates determined based on road features may not be reliable. Therefore, when inputting road features and vehicle offsets into a Kalman filter, noise assessment is necessary to optimize the result. For example, if the noise identified as road features in the input data is less than a noise threshold, the vehicle coordinates determined based on road features are reliable and can be used to reset the accumulated error of the inertial measurement unit. Conversely, if the noise identified as road features is greater than the noise threshold, the vehicle coordinates determined based on road features are unreliable, and the vehicle coordinates should still be determined based on the vehicle offset.

[0101] This embodiment improves the accuracy of vehicle coordinates by introducing road features and performing noise assessment on them, and resetting the error of the inertial measurement unit under the premise that the road features are reliable.

[0102] For example, Figure 6 A timing diagram of the vehicle panoramic sunroof display method in this embodiment is provided, such as... Figure 6 As shown in the timeline: when a user enters the Shanghai Outer Ring Road and is located at a designated coordinate position, the GPS module obtains the current location and sends a request to the landmark database to find a landmark pattern that matches the current coordinates. Based on the user's coordinates, the landmark database returns the relevant parameters of the nearest landmark—the Oriental Pearl Tower—including its outline and dynamic lighting effects. To ensure positioning accuracy, visual SLAM is used to verify the characteristics of the Huangpu River, specifically by using image data captured by a camera to verify the features of the Huangpu River. The visual SLAM system confirms that the current environment matches the features of the Oriental Pearl Tower in the landmark database with a 90% match, indicating that the GPS module's recognition is accurate. An image of the Oriental Pearl Tower, accompanied by a red cultural background, is displayed on the car's panoramic sunroof. The projection controller dynamically adjusts the size of the landmark pattern based on the distance between the vehicle and the landmark, adapting it to different viewing distances and providing users with an immersive experience.

[0103] In one embodiment, Figure 7 A flowchart illustrating the method for adjusting the brightness of the dome display is provided, such as... Figure 7 As shown, the display brightness of the landmark pattern is adjusted based on the user's posture, vehicle coordinates, vehicle body posture, and current ambient light, including the following steps:

[0104] Step S301: Obtain user posture data collected by the second camera, extract visual feature values ​​based on the user posture data, and determine the first weight parameter based on the visual feature values.

[0105] The second camera can be deployed inside the cockpit to collect user posture data, specifically head posture and gaze data. Head posture and gaze data from multiple users can be pre-collected and used as sample data to train a deep learning model to extract visual feature values. These visual feature values ​​can be categorized into three scenarios: user looking directly at the sky, strong glare interfering with vision, and clear pupil features at night. The head posture and gaze data collected from users currently in the vehicle are input into the trained deep learning model to predict the visual feature values. The scene types and corresponding first weight parameters for the visual feature values ​​are assigned as follows: user looking directly at the sky: α=0.6; strong glare interfering with vision: α=0.3; clear pupil features at night: α=0.8.

[0106] Step S302: Extract geographic information feature values ​​based on vehicle coordinates, and determine the second weight parameter based on the geographic information feature values.

[0107] A request can be sent to the vehicle navigation interface to obtain geographic information feature values ​​corresponding to the current vehicle coordinates. These geographic information feature values ​​can be categorized into open road, urban canyon, and tunnel scenarios. The corresponding scenario type and the associated second weight parameter allocation for the geographic information feature values ​​are as follows: Open road: β=0.9; Urban canyon: β=0.4; Tunnel scenario: β=0.1.

[0108] Step S303: Obtain vehicle attitude data collected by the inertial measurement unit in the vehicle, extract vehicle attitude feature values ​​based on the vehicle attitude data, and determine the third weight parameter based on the vehicle attitude feature values.

[0109] Vehicle posture data can specifically include the vehicle's offset and attitude angle change per unit time. Based on the vehicle's offset and attitude angle change per unit time, the vehicle posture feature values ​​are determined. The corresponding scene types and the corresponding third weight parameters for the vehicle posture feature values ​​are assigned as follows: Flat road surface: γ=0.8; Off-road bumpy road: γ=0.3; Cornering roll: γ=0.6.

[0110] Step S304: Obtain the light intensity data collected by the ambient light sensor, extract the ambient light feature value based on the light intensity data, and determine the fourth weighting parameter based on the ambient light feature value.

[0111] The ambient light feature values ​​are assigned to the scene type and the corresponding fourth weight parameter as follows: stable illumination: δ=0.9; sudden change in strong light at the tunnel entrance: δ=0.4; alternating flickering of tree shadows: δ=0.2.

[0112] Step S305: Based on each weight parameter, the visual feature value, geographic information feature value, vehicle posture feature value, and ambient light feature value are fused to obtain the fused decision value.

[0113] First, the visual feature values, geographic information feature values, vehicle posture feature values, and ambient light feature values ​​are normalized to unify the scale of different modal data, with values ​​falling within the interval [-1, 1]. Then, a weighted sum is calculated based on each weight parameter. The calculation formula is as follows:

[0114] W fusion =α×W vision +β×W GPS +γ×W IMU +δ×W ligh ;

[0115] Among them, W fusion W represents the fusion decision value. vision W represents visual feature values. GPS W represents the geographic information feature value. IMU W represents the vehicle body attitude characteristic value. ligh This represents the characteristic value of ambient light.

[0116] Step S306: Adjust the display brightness of the landmark pattern according to the fusion decision value.

[0117] The fusion decision value is the value calculated by multimodal perception fusion. It is a dimensionless decision index with a value range of (0,1]. For example, when the value is 0.5, the display brightness is 50%.

[0118] This embodiment combines user line of sight, posture, ambient light and driving status to adjust the brightness of the sunroof, realizing dynamic optimization of multimodal data. Compared with the traditional solution that relies on a single sensor (such as seat pressure or ambient light) to adjust the brightness of the sunroof, it is more conducive to improving the user's driving experience.

[0119] In some embodiments, after adjusting the display brightness of the landmark pattern based on the fusion decision value, the method further includes:

[0120] Determine if the driver is present and if the vehicle is in motion; if the driver is present and the vehicle is in motion, determine if the target brightness does not exceed the brightness threshold; if the display brightness exceeds the brightness threshold, adjust the display brightness to the brightness threshold.

[0121] The brightness threshold can be set to 15%, meaning that when the driver is in the seat and the vehicle is moving, the panoramic sunroof brightness is forced to be ≤15%. In this embodiment, considering that a high-brightness panoramic sunroof may affect the driver's visibility while driving, a partial restriction is implemented to ensure driving safety. In particular, when the panoramic sunroof pattern changes, it may attract the driver's attention; therefore, the function is considered to be linked to entertainment restrictions. When the entertainment restriction switch is turned off in the vehicle, this restriction can be waived.

[0122] In some embodiments, if it is determined that the display brightness does not exceed the brightness threshold, the landmark pattern is displayed at the currently calculated display brightness.

[0123] In some embodiments, the method further includes: reading ADAS radar signals and door status collected by door sensors; determining whether there is a risk of oncoming vehicle based on ADAS radar signals; if it is determined that there is a risk of oncoming vehicle, then when any door is opened, projecting a red light pulse in the skylight area near the side of the opened door.

[0124] In this embodiment, the ADAS radar signal is sent to the central controller at a certain frequency, and the door sensors send the door status to the central controller. When the central controller receives the ADAS radar signal indicating the risk of an approaching vehicle, and also receives the door sensor signal indicating that the door is open, the central controller will generate a command to instruct the ambient lighting array to project a red light pulse onto the sunroof area near the open door. The frequency of the red light pulse can be 2Hz. This embodiment achieves oncoming vehicle alerts through the sunroof, realizing a driving safety linkage.

[0125] For example, Figure 8 A timing diagram of the side approach warning triggering method is provided, such as... Figure 8 As shown, the user manually or remotely opened the vehicle's left rear door. Upon detecting the door opening, the door sensor immediately sent a request to the ADAS radar to monitor the blind spot on the left side of the vehicle to ensure door opening safety. Upon receiving the request, the ADAS radar began scanning the left side of the vehicle and detected an oncoming vehicle at a distance of 1.5 meters and a speed of 30 km / h, indicating a potential collision risk. Based on the detected oncoming vehicle information, the ADAS radar determined that a warning needed to be issued to the user, particularly in the 30% area on the left side of the door, the area most likely to come into contact with the oncoming vehicle. After receiving the warning from the ADAS radar, the central controller sent a command to the panoramic sunroof to display a red warning light in the left area, flashing pulses at a frequency of 2Hz. This high-frequency flashing effectively attracts the user's attention, reminding them of the oncoming vehicle on the left. Ultimately, the panoramic sunroof, following the instructions of the central controller, flashed red light at a frequency of 2Hz in the left area, creating a clear visual warning and helping the user avoid traffic accidents caused by improper door opening.

[0126] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0127] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0128] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0129] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for displaying a vehicle panoramic sunroof, characterized in that, include: Obtain the vehicle coordinates output by the positioning system, and determine the administrative division level of the vehicle based on the vehicle coordinates; Obtain landmark patterns that match the landmarks in the administrative division level; The display brightness of the landmark pattern is adjusted based on the user's posture in the vehicle, the vehicle's coordinates, the vehicle's body posture, and the current ambient light. The landmark pattern is displayed on the vehicle's skylight according to the specified display brightness.

2. The vehicle panoramic sunroof display method according to claim 1, characterized in that, The positioning system includes a GPS module and an inertial measurement unit; acquiring the vehicle coordinates output by the positioning system includes: The GPS module is subjected to a validity test to determine whether the GPS module is valid; If the GPS module is determined to be in a valid state, the GPS coordinates output by the GPS module will be used as the vehicle coordinates. If the GPS module is determined to be in a malfunctioning state, the acceleration collected by the inertial measurement unit is read, the vehicle offset is calculated based on the acceleration, and the GPS coordinates last generated by the GPS module are updated based on the vehicle offset to obtain the vehicle coordinates.

3. The vehicle panoramic sunroof display method according to claim 2, characterized in that, The positioning system further includes a first camera; if the GPS module is determined to be in a malfunctioning state, the method further includes: Read the road image captured by the first camera, and extract road features based on the road image; The road features and the vehicle offset are input into a Kalman filter to perform vehicle pose estimation. The vehicle coordinates are determined based on the vehicle pose estimation results.

4. The vehicle panoramic sunroof display method according to claim 3, characterized in that, The road features and vehicle offset are input into a Kalman filter to perform vehicle pose estimation. Based on the vehicle pose estimation result, the vehicle coordinates are determined, including: Determine whether the distance between the vehicle coordinates determined based on the road features and the vehicle coordinates determined based on the vehicle offset is within a preset range; If it is determined that the distance exceeds the preset range, then it is determined whether the noise of the road feature is less than the noise threshold. If the noise of the road feature is determined to be less than the noise threshold, then the vehicle coordinates determined based on the road feature are used. If the noise of the road feature is determined to be no less than the noise threshold, then the vehicle coordinates determined based on the vehicle offset are used.

5. The vehicle panoramic sunroof display method according to claim 1, characterized in that, Adjusting the display brightness of the landmark pattern based on the user's posture in the vehicle, the vehicle's coordinates, the vehicle's body posture, and the current ambient light includes: The user posture data collected by the second camera in the vehicle is obtained, visual feature values ​​are extracted based on the user posture data, and a first weight parameter is determined based on the visual feature values. Based on the vehicle coordinates, geographic information feature values ​​are extracted, and a second weighting parameter is determined based on the geographic information feature values. The vehicle body attitude data collected by the inertial measurement unit in the vehicle is acquired, and the vehicle body attitude feature values ​​are extracted based on the vehicle body attitude data. A third weighting parameter is then determined based on the vehicle body attitude feature values. The system acquires the light intensity data collected by the ambient light sensor in the vehicle, extracts ambient light feature values ​​based on the light intensity data, and determines a fourth weighting parameter based on the ambient light feature values. The fusion decision value is obtained by fusing the visual feature value, the geographic information feature value, the vehicle posture feature value and the ambient light feature value according to the first weight parameter, the second weight parameter, the third weight parameter and the fourth weight parameter; The display brightness of the landmark pattern is adjusted based on the fusion decision value.

6. The vehicle panoramic sunroof display method according to claim 5, characterized in that, After adjusting the display brightness of the landmark pattern based on the fusion decision value, the method further includes: Determine whether the driver is present and whether the vehicle is in motion; If it is determined that the driver is present and the vehicle is in motion, then it is determined whether the display brightness does not exceed the brightness threshold. If it is determined that the display brightness exceeds the brightness threshold, then the display brightness is adjusted to the brightness threshold.

7. The vehicle panoramic sunroof display method according to claim 1, characterized in that, After displaying the landmark pattern on the vehicle's roof at the specified display brightness, the method further includes: The distance between the vehicle and the landmark is calculated based on the vehicle coordinates, and the size of the landmark pattern displayed in the canopy is adjusted based on the distance.

8. The vehicle panoramic sunroof display method according to claim 1, characterized in that, The method further includes: Read the ADAS radar signal and the door status collected by the door sensor; The ADAS radar signal is used to determine whether there is a risk of oncoming vehicle. If a risk of oncoming traffic is detected, a red light pulse will be projected onto the skylight area on the side closest to the opened door when any door is opened.

9. A canopy system, characterized in that, include: The system includes a positioning system, a canopy, and a central controller. The canopy is equipped with an ambient lighting array, and the positioning system and the ambient lighting array are respectively connected to the central controller. The positioning system is used to generate vehicle coordinates; The central controller is used to receive the vehicle coordinates and execute the vehicle panoramic display method according to any one of claims 1 to 8 above based on the vehicle coordinates; The ambient light array is used to display landmark patterns under the control of the central controller.

10. A vehicle, characterized in that, include: The vehicle body and the skylight system of claim 9, wherein the skylight system is installed in the vehicle body.

Images