Panoramic image display system, method, vehicle, electronic device, and storage medium

Abstract

The present disclosure relates to a panoramic image display system, method, vehicle, electronic device and storage medium, and relates to the field of vehicle design, and the system comprises: in the case of starting a panoramic image function and a UAV image auxiliary function, controlling a UAV to take off from a vehicle and fly to a first target position, and acquiring a UAV image from the UAV, the UAV image being acquired by the UAV from the vehicle at the first target position according to an image mode of the UAV image auxiliary function at present, and according to a display mode of the panoramic image function at present, controlling the display screen to display a panoramic image containing the UAV image, through the UAV image auxiliary function, the target position required to acquire the image is acquired by the UAV, and the UAV image is displayed on the display screen, which can reduce image distortion and image blind area, and can provide effective image assistance for users.

Description

Technical Field

[0001] This disclosure relates to the field of vehicle design, and more specifically, to a panoramic imaging display system, method, vehicle, electronic device, and storage medium. Background Technology

[0002] In existing technologies, vehicle panoramic imaging systems stitch together images captured by cameras. The resulting stitched images may have image distortion and blind spots at the stitching points, making it impossible for users to observe the complete image of the vehicle's surroundings and failing to provide effective image assistance. Summary of the Invention

[0003] The purpose of this disclosure is to provide a panoramic imaging display system, method, vehicle, electronic device, and storage medium to solve the problem that current panoramic imaging systems for vehicles cannot provide complete images of the vehicle's surroundings.

[0004] To achieve the above objectives, a first aspect of this disclosure provides a panoramic image display system for use in a vehicle, the system comprising: a control host, a drone, and a display screen, wherein the display screen is connected to the control host, and the drone communicates with the control host via a wireless connection.

[0005] The control host is used to control the drone to take off from the vehicle and fly to the first target position when the panoramic imaging function and the drone imaging assistance function are activated.

[0006] The drone is used to take pictures of the vehicle at the first target location according to the current image mode of the drone image assistance function to obtain the corresponding drone image, and send the drone image to the control host.

[0007] The control host is also used to control the display screen to display a panoramic image containing the image of the UAV, according to the current display mode of the panoramic image function.

[0008] Optionally, when the current image mode of the drone image assistance function is overhead mode, the first target position is located directly above the vehicle.

[0009] Optionally, the system further includes: multiple radars, which are respectively disposed at multiple locations on the exterior of the vehicle, and the multiple radars are connected to the control host;

[0010] The radar is used to detect obstacles in the target direction, and when an obstacle is detected in the target direction, to send the distance information of the obstacle to the control host. The target direction is determined according to the position of the radar on the vehicle.

[0011] The control host is further configured to, upon receiving distance information of the obstacle sent by any of the radars, generate alarm information corresponding to the target direction based on the distance information, wherein the alarm information includes the location of the first target;

[0012] The control host is also used to control the drone to fly to the first target location to take pictures of the vehicle according to the alarm information, so as to obtain drone images in the target direction, and send the drone images in the target direction to the control host;

[0013] The control host is also configured to control the display screen to display a panoramic image containing the UAV image in the target direction, according to the current display mode of the panoramic image function.

[0014] Optionally, the control host is further configured to, upon receiving alarm information corresponding to two or more target directions, set a preset position directly above the vehicle as the first target position, control the drone to fly to the first target position to take pictures of the vehicle to obtain an overhead image of the vehicle, and send the overhead image to the control host.

[0015] The control host is also used to control the display screen to display a panoramic image containing the overhead view, according to the current display mode of the panoramic image function.

[0016] Optionally, the display screen is a touch screen;

[0017] The display screen is also used to receive the image position selected by the user in the panoramic image and send the image position to the control host;

[0018] The control host is also used to set the image position as the first target position.

[0019] Optionally, the control host is further configured to send a first control command or a second control command to the UAV upon receiving a left turn signal or a right turn signal;

[0020] The drone is also used to fly to a first designated position on the left side of the vehicle according to the first control command, or to fly to a second designated position on the right side of the vehicle according to the second control command, so as to obtain drone images of the first designated position or the second designated position, and send the drone images of the first designated position or the second designated position to the control host.

[0021] The control host is also used to control the display screen to display a panoramic image containing the drone image of the first specified location or the second specified location, according to the current display mode of the panoramic image function.

[0022] Optionally, the control host is further configured to send a lighting activation command to the drone when the auxiliary lighting function is activated;

[0023] The drone is also configured to turn on its lighting device in response to the lighting activation command.

[0024] A second aspect of this disclosure provides a panoramic image display method applied to a vehicle, the vehicle including a display screen and a drone, the method comprising:

[0025] With the panoramic imaging function and the drone imaging assistance function activated, the drone is controlled to take off from the vehicle and fly to the first target position, and drone images are acquired from the drone. The drone images are acquired by the drone taking pictures of the vehicle at the first target position according to the current imaging mode of the drone imaging assistance function.

[0026] Based on the current display mode of the panoramic imaging function, control the display screen to display a panoramic image containing the image of the drone.

[0027] Optionally, when the current image mode of the drone image assistance function is overhead mode, the first target position is located directly above the vehicle.

[0028] Optionally, the vehicle further includes: multiple radars, which are respectively disposed at multiple locations outside the vehicle; the step of controlling the drone to take off from the vehicle and fly to the first target location, and acquiring drone images from the drone when the panoramic imaging function and the drone imaging assistance function are activated, includes:

[0029] If an obstacle is detected in the target direction by any of the radars, the distance information of the obstacle is obtained from the radar, and the target direction is determined according to the position of the radar on the vehicle;

[0030] Based on the distance information, an alarm message corresponding to the target direction is generated, and the alarm message includes the location of the first target.

[0031] Based on the alarm information, the drone is controlled to fly to the first target location to photograph the vehicle in order to obtain drone images in the direction of the target.

[0032] Optionally, controlling the drone to fly to the first target location to photograph the vehicle based on the alarm information, thereby obtaining drone imagery in the target direction, includes:

[0033] Upon receiving alarm messages corresponding to two or more target directions, a preset position directly above the vehicle is set as the first target position, and the drone is controlled to fly to the first target position to photograph the vehicle, thereby obtaining a top-down image of the vehicle. Optionally, the display screen is a touch screen, and the method further includes:

[0034] Receive the image location selected by the user in the panoramic image;

[0035] Set the image position as the first target position.

[0036] Optionally, the method further includes:

[0037] Upon receiving a left turn signal or a right turn signal, a first control command or a second control command is sent to the drone to instruct the drone to fly to a first designated position on the left side of the vehicle according to the first control command, or to fly to a second designated position on the right side of the vehicle according to the second control command, so as to obtain drone images of the first designated position or the second designated position.

[0038] Based on the current display mode of the panoramic imaging function, control the display screen to display a panoramic image containing drone images of the first specified location or the second specified location.

[0039] Optionally, the method further includes:

[0040] When the auxiliary lighting function is activated, a lighting activation command is sent to the drone, which instructs the drone to turn on its lighting equipment.

[0041] A third aspect of this disclosure provides a vehicle comprising: the panoramic imaging display system described in the first aspect of this disclosure.

[0042] A fourth aspect of this disclosure provides a panoramic imaging display device applied to a vehicle, the vehicle including a display screen and a drone, the device comprising:

[0043] The first control module is used to control the drone to take off from the vehicle and fly to the first target position above the vehicle when the panoramic imaging function and the drone imaging assistance function are activated.

[0044] The image acquisition module is used to acquire drone images from the drone, which are captured by the drone at the first target location based on the current image mode of the drone image assistance function.

[0045] The second control module is used to control the display screen to display a panoramic image containing the image of the UAV, according to the current display mode of the panoramic image function.

[0046] A fifth aspect of this disclosure provides an electronic device comprising:

[0047] A memory on which computer programs are stored;

[0048] A processor for executing the computer program in the memory to implement the steps of the method in any of the second aspects.

[0049] A sixth aspect of this disclosure provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the second aspects.

[0050] In the above technical solution, when the panoramic imaging function and the drone imaging assistance function are activated, the drone is controlled to take off from the vehicle and fly to the first target position. The drone image is acquired by the drone, which takes pictures of the vehicle at the first target position according to the current imaging mode of the drone imaging assistance function. According to the current display mode of the panoramic imaging function, the display screen is controlled to display the panoramic image containing the drone image. Through the above-mentioned drone imaging assistance function, the drone takes pictures of the target position where the image needs to be acquired and displays the drone image on the display screen, which can reduce image distortion and image blind spots and provide effective imaging assistance to users.

[0051] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0052] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings:

[0053] Figure 1 This is a schematic diagram illustrating a panoramic image display system according to an exemplary embodiment.

[0054] Figure 2 This is a schematic diagram illustrating another panoramic image display system according to an exemplary embodiment.

[0055] Figure 3 This is a schematic diagram of the system architecture of a panoramic image display system according to an exemplary embodiment.

[0056] Figure 4 This is a schematic diagram illustrating a display screen according to an exemplary embodiment.

[0057] Figure 5 This is a schematic diagram illustrating the location of a drone according to an exemplary embodiment.

[0058] Figure 6 This is a schematic diagram illustrating a drone positioning method according to an exemplary embodiment.

[0059] Figure 7 This is a schematic diagram illustrating another drone location according to an exemplary embodiment.

[0060] Figure 8 This is a schematic diagram illustrating yet another drone location according to an exemplary embodiment.

[0061] Figure 9 This is a flowchart illustrating a panoramic image display method according to an exemplary embodiment.

[0062] Figure 10 This is a flowchart illustrating another panoramic image display method according to an exemplary embodiment.

[0063] Figure 11 This is a flowchart illustrating yet another panoramic image display method according to an exemplary embodiment.

[0064] Figure 12 This is a block diagram illustrating a panoramic image display device according to an exemplary embodiment.

[0065] Figure 13 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0066] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0067] It is understood that in this disclosure, "multiple" refers to two or more, and other quantifiers are similar. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. The singular forms "a," "the," and "the" are also intended to include the plural forms unless the context clearly indicates otherwise.

[0068] It is further understood that the terms "first," "second," etc., are used to describe various types of information, but this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and do not indicate a specific order or degree of importance. In fact, the expressions "first," "second," etc., are completely interchangeable. For example, without departing from the scope of this disclosure, a first target location can also be referred to as a second target location, and similarly, a second target location can also be referred to as a first target location.

[0069] It is further understood that although operations are described in a specific order in the accompanying drawings in the embodiments of this disclosure, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the shown operations to be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0070] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with authorization from the owner of the relevant device.

[0071] Before introducing the panoramic imaging display system provided in this disclosure, the application scenarios involved in the system will first be introduced. It is understood that in existing vehicle panoramic imaging systems, most of them acquire images from multiple cameras set at multiple locations on the vehicle and then stitch the images together. The stitched image is used to observe the environment around the vehicle, helping users to observe the field of vision of narrow roads and congested roads, as well as the situation of other vehicles, pedestrians, and obstacles. However, in existing panoramic imaging systems, because the images are stitched together from four cameras, there are problems such as image distortion and blind spots at the stitching points. This may result in the inability to provide a complete and realistic view of the vehicle's surroundings, thus affecting the user's daily use. Furthermore, in low light conditions, such as at night, the images provided by existing panoramic imaging systems may be dim and unclear, causing inconvenience to users. Therefore, providing users with a complete, clear, and realistic panoramic image of the vehicle has become an urgent problem to be solved. To solve the above problems, this disclosure provides a panoramic imaging display system and a panoramic imaging display method. The panoramic imaging display system will be described below.

[0072] Figure 1 This is a schematic diagram illustrating a panoramic image display system according to an exemplary embodiment, such as... Figure 1As shown, the panoramic imaging display system 10 includes: a control host 101, a drone 102, and a display screen 103. The display screen 103 is connected to the control host 101, and the drone 102 communicates with the control host 101 wirelessly. The panoramic imaging function of the panoramic imaging display system 10 has multiple imaging modes, such as a vehicle-mounted imaging mode and a drone-assisted imaging mode. The drone-assisted imaging mode can include a bird's-eye view mode and an intelligent mode. In the vehicle-mounted imaging mode, i.e., the 360° panoramic imaging mode of the vehicle, multiple panoramic cameras positioned around the vehicle cover the field of view around the vehicle. These cameras simultaneously capture images of the area around the vehicle and transmit the image information to the control host 10. After processing, the images are displayed on the display screen 103. When the drone image assistance function is enabled, the drone image assistance mode is entered. In the overhead view mode, the drone 102 can provide real overhead images of the vehicle 20 and its surroundings and display them on the display screen 103. When the drone image assistance function is enabled, in the intelligent mode, the drone 102 can provide a two-dimensional stitched image of the vehicle 20 from an overhead view and a three-dimensional model of the vehicle 20. In this mode, the user can control the drone 102 to fly to the corresponding position by clicking on different positions of the three-dimensional model and transmit the images to the display screen 103 through the control host 101. In addition, the panoramic image function in the panoramic image display system 10 can also include multiple display modes, such as three-dimensional mode and two-dimensional mode.

[0073] The control host 101 is used to control the drone 102 to take off from the vehicle 20 and fly to the first target position when the panoramic imaging function and the drone imaging assistance function are activated.

[0074] Optionally, when the current image mode of the drone image assistance function is overhead mode, the first target position is located directly above the vehicle 20.

[0075] The drone 102 is used to take pictures of the vehicle 20 at the first target location according to the current image mode of the drone image assistance function to obtain the corresponding drone image, and send the drone image to the control host 101.

[0076] The control host 101 is also used to control the display screen 103 to display a panoramic image containing the image of the drone, according to the current display mode of the panoramic image function.

[0077] Understandably, the vehicle 20 is also equipped with multiple panoramic cameras, which can enable the vehicle imaging mode. In this mode, the panoramic cameras can capture images of the vehicle's surroundings, transmit the image information to the control host 101, process it, and display it on the display screen 103. When the drone imaging assistance function is enabled, in the overhead view mode, the panoramic cameras can obtain single views of various positions of the vehicle 20, which can be stitched together by the control host 101 to obtain an overhead view of the vehicle.

[0078] For example, the drone 102 can be installed inside the vehicle 20 and communicate with the control host 101 via a wireless network. Alternatively, a hangar can be added to the vehicle 20 to house the drone 102. This hangar can be connected to the control host 101 via a CAN (Controller Area Network) bus, allowing the control host 101 to control the opening and closing of the hangar. The drone 102 can be controlled by the control host 101 via the wireless network. For example, the focus of the images acquired by the drone can be adjusted by manipulating the volume knob on the steering wheel, thereby controlling the zoom in or out of the drone images.

[0079] It is understood that the control host 101 can be a smart cockpit host, which can be a host device installed in the driver's cab of a vehicle. It can control the drone 102 by interacting with the user through traditional physical buttons, or by interacting with the user through a screen on the dashboard, a screen between the driver's and passenger's seats, and / or a screen installed at one or more other seats. Alternatively, it can interact with the user through voice or gesture operations, or it can interact with the user through a remote control to control the drone 102.

[0080] In the above technical solution, when the panoramic imaging function and the drone imaging assistance function are activated, the drone is controlled to take off from the vehicle and fly to the first target position. The drone image is acquired from the drone. The drone image is captured by the drone at the first target position according to the current imaging mode of the drone imaging assistance function. According to the current display mode of the panoramic imaging function, the display screen is controlled to display the panoramic image containing the drone image. Through the above-mentioned drone imaging assistance function, the drone is used to capture the target position where the image needs to be acquired, and the drone image is displayed on the display screen. This reduces image distortion and blind spots to a certain extent and can provide effective image assistance to the user.

[0081] Optionally, Figure 2 This is a schematic diagram illustrating another panoramic image display system according to an exemplary embodiment, such as... Figure 2As shown, in one implementation, the panoramic image display system 10 may include: a plurality of radars 104, which are respectively disposed at a plurality of locations outside the vehicle 20, and all of the plurality of radars 104 are connected to the control host 101.

[0082] The radar 104 is used to detect obstacles in the target direction, and when an obstacle is detected in the target direction, to send the distance information of the obstacle to the control host 101. The target direction is determined based on the position of the radar 104 on the vehicle 20. For example, the plurality of radars 104 can be parking radars or other types of radars. The control host 101 includes a control module for controlling the display screen 103, a control module for controlling the drone 102, and / or a receiving module for receiving information from the plurality of radars 104. The control module and the receiving module can be processors such as CPUs and MPUs, or hardware logic circuits or chips capable of implementing the above functions.

[0083] The control host 101 is also used to generate alarm information corresponding to the direction of the target based on the distance information of an obstacle sent by any radar 104 after receiving the distance information of the obstacle. The alarm information includes the location of the first target. The vehicle may include an alarm module that can output an alarm sound as the alarm information. The alarm module, such as a buzzer, can indicate the distance of the obstacle based on the sound frequency of the alarm sound. The higher the frequency, the closer the obstacle is, and the lower the frequency, the farther the obstacle is.

[0084] Optionally, the control host 101 is also used to control the drone 102 to fly to the first target position to take pictures of the vehicle 20 according to the alarm information, so as to obtain drone images in the target direction, and send the drone images in the target direction to the control host 101.

[0085] The control host 101 is also used to control the display screen 103 to display a panoramic image containing the image of the drone in the target direction, according to the current display mode of the panoramic image function.

[0086] Optionally, the control host 101 is also used to set a preset position directly above the vehicle 20 as the first target position when receiving alarm information corresponding to two or more target directions, and control the drone 102 to fly to the first target position to take pictures of the vehicle 20 to obtain an overhead image of the vehicle 20, and send the overhead image to the control host 101.

[0087] The control host 101 is also used to control the display screen 103 to display a panoramic image including a top-down view, according to the current display mode of the panoramic image function.

[0088] Optionally, the display screen 103 is a touch screen;

[0089] The display screen 103 is also used to receive the image position selected by the user in the panoramic image and send the image position to the control host 101.

[0090] The control host 101 is also used to set the image position as the first target position.

[0091] Optionally, the control host 101 is also used to send a first control command or a second control command to the drone 102 upon receiving a left turn signal or a right turn signal.

[0092] The drone 102 is also used to fly to a first designated position on the left side of the vehicle 20 according to the first control command, or to fly to a second designated position on the right side of the vehicle 20 according to the second control command, to obtain drone images of the first designated position or the second designated position, and to send the drone images of the first designated position or the second designated position to the control host 101.

[0093] The control host 101 is also used to control the display screen 103 to display a panoramic image containing the drone image of the first designated location or the second designated location, according to the current display mode of the panoramic image function.

[0094] Optionally, a lighting device may be installed on the drone 102, and the control host 101 is also used to send a lighting command to the drone 102 when the auxiliary lighting function is turned on.

[0095] The drone 102 is also used to turn on its lighting equipment in response to the lighting command.

[0096] The lighting equipment on the drone 102 can include multiple adjustable-brightness LED lights. Because images captured by both the panoramic camera and the drone camera are often unclear in low-light conditions such as at night, leading to a lack of awareness of surrounding road conditions and potential safety hazards, the drone's lighting equipment provides illumination in low-light situations. This ensures clearer images from both the panoramic camera and the drone camera, providing a more comprehensive view of the road conditions.

[0097] Figure 3 This is a schematic diagram of the system architecture of a panoramic image display system according to an exemplary embodiment, such as... Figure 3As shown, in one implementation, in the panoramic image display system, the panoramic camera sends the acquired video signal to the control host 101. The transmission of video signals between the control host 101 and the display screen 103, as well as the transmission of alarm information from the radar 104 to the control host 101, can all be transmitted via LVDS (Low-Voltage Differential Signaling). A hangar for placing the drone 102 can be set up. The hangar can be connected to the control host 101 via a CAN (Controller Area Network) bus, thereby enabling the control host 101 to control the opening or closing of the hangar. The drone 102 can be controlled by the control host 101 through the wireless network.

[0098] Figure 4 This is a schematic diagram illustrating a display screen according to an exemplary embodiment, such as... Figure 4 As shown, in one implementation, after enabling the drone image assistance function, in overhead view mode, the vehicle overhead image acquired by the drone is displayed in area A of the display screen 103; when the 3D mode is enabled, the 3D model of vehicle 20 is displayed in area A of the display screen 103, and the vehicle overhead image acquired by the drone is displayed in area C of the display screen 103; when the 3D mode is enabled, the user can click on various positions of the vehicle 3D model in area A, and the control host 101 controls the drone 102 to fly to the corresponding position according to the received instructions, and provides the vehicle image and surrounding environment image at the corresponding position; when the drone image assistance function is enabled, in intelligent mode, the 2D overhead view of the vehicle captured by the panoramic camera and stitched by the control host 101 is displayed in area A of the display screen 103, and the drone 102 is displayed in area B. 02. With the acquired images, the user can click on various locations in the 2D overhead view of the vehicle in area A. The control host 101 controls the drone 102 to fly to the corresponding location according to the received instructions, providing images of the corresponding location. When the vehicle 20 is driving and turning left or right, the control host 101 controls the drone 102 to fly to the first or second designated location and provides images of the front wheels and surrounding road surface of the vehicle 20, which are displayed in area B. When a radar 104 is triggered, a corresponding alarm sound will be generated. The control host 101 controls the drone 102 to fly to the location with the highest frequency of the radar 104 alarm sound and provides images of the corresponding location, which are displayed in area B. When two or more radars 104 emit alarm frequencies that are consistent, the control host 102 controls the drone 102 to fly directly above the vehicle 20 and provides an overhead view, which is displayed in area B.

[0099] Figure 5 This is a schematic diagram illustrating the location of a drone according to an exemplary embodiment, such as... Figure 5As shown, in one implementation, when the panoramic image display system activates the drone-assisted mode, in two-dimensional mode, the user can click... Figure 4 In the schematic diagram of the central display screen 103, the two-dimensional vehicle model in area A is positioned at different locations. After receiving the instruction, the control host 101 controls the drone 102 to fly to the corresponding position. The position of the drone 102 can be determined by the UWB (UltraWide Band) positioning system.

[0100] Figure 6 This is a schematic diagram illustrating a drone positioning method according to an exemplary embodiment, such as... Figure 6 As shown, in one implementation, the position of the UAV 102 in this panoramic image display system is determined based on the UWB positioning system. The number of positioning anchor points set on the vehicle 20 is greater than or equal to three. The UWB controller 601 obtains the UAV's position by collecting the distances between the UWB anchor points 602 and the UAV 102, using the principle of trilateration. For example, given the positions of three points (x1, y1), (x2, y2), and (x3, y3), and the distances d1, d2, and d3 from the unknown point (x0, y0) to the known three points, the coordinates of the UAV 102 can be resolved using the least squares algorithm, thus obtaining the position of the UAV 102. Therefore, the control host 101 can control the UAV 102 to fly to a designated location and acquire images of that location based on its position.

[0101] Figure 7 This is a schematic diagram illustrating another drone location according to an exemplary embodiment, such as... Figure 7 As shown, in one implementation, the user clicks... Figure 4 When the 3D model of vehicle 20 in area A of the display screen is at different positions, the control host 101 receives the instruction and controls the drone 102 to fly to the corresponding position of vehicle 20. For example, when the user clicks on point P1 of the 3D model of vehicle 20 in area A, the drone 102 flies to the corresponding point P2. At this time, the distance between the drone 102 and vehicle 20 is h1, and the distance between the drone 102 and the ground is h2. At this time, the drone 102 and the horizontal plane where point P1 is located form an angle α. The camera of the drone 102 provides an image of vehicle 20 at point P1.

[0102] Figure 8 This is a schematic diagram illustrating yet another drone location according to an exemplary embodiment, such as... Figure 8As shown, in one implementation, when radar 104 detects an obstacle, it generates an alarm message and sends it to the control host 101. Upon receiving the alarm message, the control host 101 controls the drone 102 to fly above the radar 104 that triggered the alarm and provide an image of the corresponding location. When two or more radars 104 have the same alarm frequency, the drone flies directly above the vehicle 20 and provides a top-down image, which is displayed on... Figure 4 The B area of ​​the display screen shown.

[0103] In the above technical solution, radar can be used to warn of obstacles around the vehicle, and the drone can be controlled to fly to different positions and provide images of the corresponding positions through the operation display screen. The lighting mode can also be turned on, which increases safety, improves the flexibility of displaying vehicle images, reduces blind spots of the image around the vehicle, and provides lighting equipment to assist in providing clearer images in low light conditions.

[0104] Figure 9 This is a flowchart illustrating a panoramic image display method according to an exemplary embodiment of the present disclosure, such as... Figure 9 As shown, this panoramic image display method is applied to a vehicle, and the method may include the following steps:

[0105] In step S901, with the panoramic imaging function and the drone imaging assistance function activated, the drone is controlled to take off from the vehicle and fly to the first target position, and drone images are acquired from the drone. The drone images are acquired by the drone taking pictures of the vehicle at the first target position according to the current imaging mode of the drone imaging assistance function.

[0106] In step S902, the display screen is controlled to display a panoramic image containing the image of the drone, according to the current display mode of the panoramic image function.

[0107] The drone imaging assistance function includes overhead view and intelligent modes. When activated, in overhead view mode, the drone provides a realistic aerial view of the vehicle and its surroundings, displayed on the screen. In intelligent mode, the drone provides a 2D stitched image of the vehicle from an overhead perspective, as well as a 3D model of the vehicle. When the drone imaging assistance function is disabled, multiple panoramic cameras are installed on the vehicle. In overhead view mode, these cameras capture single views of various locations on the vehicle, which are then stitched together by the control unit to obtain an overhead view of the vehicle. Furthermore, the panoramic imaging function has multiple display modes, such as 3D and 2D modes. The acquired panoramic images can be processed by the control unit to produce corresponding 3D and 2D images.

[0108] It is understandable that, such as Figure 1 As shown, when providing auxiliary imagery, the drone moves along with the vehicle. In other words, the initial target position is relative to the vehicle's position; the drone moves with the vehicle while maintaining its relative position, thus providing the imagery the user needs.

[0109] In the above technical solution, when the panoramic imaging function and the drone imaging assistance function are activated, the drone is controlled to take off from the vehicle and fly to the first target position. The drone image is acquired by the drone, which takes pictures of the vehicle at the first target position according to the current imaging mode of the drone imaging assistance function. According to the current display mode of the panoramic imaging function, the display screen is controlled to display the panoramic image containing the drone image. Through the above-mentioned drone imaging assistance function, the drone takes pictures of the target position where the image needs to be acquired and displays the drone image on the display screen. This reduces image distortion and blind spots to a certain extent and can provide effective imaging assistance to the user.

[0110] Optionally, refer to Figure 2 and Figure 8 As shown, the vehicle also includes multiple radars, which are respectively installed at multiple locations on the exterior of the vehicle. Figure 10 This is a flowchart illustrating another panoramic image display method according to an exemplary embodiment of the present disclosure, such as... Figure 10 As shown, step S901 may further include the following steps:

[0111] In step S9011, if an obstacle is detected in the target direction by any radar, the distance information of the obstacle is obtained from the radar, and the target direction is determined according to the position of the radar on the vehicle.

[0112] In step S9012, alarm information corresponding to the target direction is generated based on the distance information, and the alarm information includes the location of the first target.

[0113] In step S9013, based on the alarm information, the drone is controlled to fly to the first target position to take pictures of the vehicle in order to obtain drone images in the direction of the target.

[0114] Optionally, step S9013 may include: upon receiving alarm information corresponding to two or more target directions, setting a preset position directly above the vehicle as the first target position, and controlling the drone to fly to the first target position to take pictures of the vehicle in order to obtain an overhead image of the vehicle.

[0115] Optionally, refer to Figure 3 As shown, the display screen is a touch screen.Figure 11 This is a flowchart illustrating another panoramic image display method according to an exemplary embodiment of the present disclosure, such as... Figure 11 As shown, the method may further include the following steps:

[0116] In step S903, the image location selected by the user in the panoramic image is received.

[0117] In step S904, the image position is set as the first target position.

[0118] Optionally, the method may further include:

[0119] Upon receiving a left-turn signal or a right-turn signal, a first control command or a second control command is sent to the drone to instruct the drone to fly to a first designated position on the left side of the vehicle according to the first control command, or to fly to a second designated position on the right side of the vehicle according to the second control command, so as to obtain drone images of the first designated position or the second designated position.

[0120] Based on the current display mode of the panoramic imaging function, the control display screen displays a panoramic image that includes the drone image of the first designated location or the second designated location.

[0121] Optionally, the method further includes:

[0122] When the auxiliary lighting function is activated, a lighting activation command is sent to the drone, which instructs the drone to turn on its lighting equipment.

[0123] In the above technical solution, radar can be used to warn of obstacles around the vehicle, and the drone can be controlled to fly to different positions and provide images of the corresponding positions through the operation display screen. The lighting mode can also be turned on, which increases safety, improves the flexibility of displaying vehicle images, reduces blind spots of the image around the vehicle, and provides lighting equipment to assist in providing clearer images in low light conditions.

[0124] Figure 12 This is a block diagram illustrating a panoramic image display device according to an exemplary embodiment of the present disclosure. Figure 12 As shown, the panoramic imaging display device can be applied to vehicles, and the device 1200 includes:

[0125] The image acquisition module 1201, when the panoramic image function and the UAV image assistance function are activated, controls the UAV to take off from the vehicle and fly to the first target position, and acquires UAV images from the UAV. The UAV images are acquired by the UAV from the vehicle at the first target position according to the current image mode of the UAV image assistance function.

[0126] The first control module 1202 is also used to control the display screen to display a panoramic image containing the image of the drone, according to the current display mode of the panoramic image function.

[0127] In the above technical solution, when the panoramic imaging function and the drone imaging assistance function are activated, the drone is controlled to take off from the vehicle and fly to the first target position. The drone image is acquired by the drone, which takes pictures of the vehicle at the first target position according to the current imaging mode of the drone imaging assistance function. According to the current display mode of the panoramic imaging function, the display screen is controlled to display the panoramic image containing the drone image. Through the above-mentioned drone imaging assistance function, the drone takes pictures of the target position where the image needs to be acquired and displays the drone image on the display screen. This reduces image distortion and blind spots to a certain extent and can provide effective imaging assistance to the user.

[0128] Optionally, the vehicle further includes: multiple radars, which are respectively disposed at multiple locations on the exterior of the vehicle; the image acquisition module 1201 may also include:

[0129] The distance acquisition module is used to acquire distance information of an obstacle from any radar when an obstacle is detected in the direction of a target. The direction of the target is determined based on the position of the radar on the vehicle.

[0130] The alarm generation module is used to generate alarm information corresponding to the target direction based on the distance information. The alarm information includes the location of the first target.

[0131] The camera module is used to control the drone to fly to the first target location to take pictures of the vehicle based on the alarm information, so as to obtain drone images in the direction of the target.

[0132] Optionally, the image acquisition module 1201 is further configured to: upon receiving alarm information corresponding to two or more target directions, set a preset position directly above the vehicle as the first target position, and control the drone to fly to the first target position to take pictures of the vehicle in order to obtain an overhead image of the vehicle.

[0133] Optionally, the display screen is a touch screen, and the device 1200 may further include:

[0134] The receiving module is used to receive the image location selected by the user in the panoramic image.

[0135] The setting module is used to set the image position as the first target position.

[0136] Optionally, the device 1200 may further include:

[0137] The steering module is used to send a first control command or a second control command to the drone upon receiving a left turn signal or a right turn signal, instructing the drone to fly to a first designated position on the left side of the vehicle according to the first control command, or to fly to a second designated position on the right side of the vehicle according to the second control command, so as to obtain drone images of the first designated position or the second designated position.

[0138] The second control module is used to control the display screen to display a panoramic image containing the drone image of the first designated location or the second designated location, according to the current display mode of the panoramic image function.

[0139] Optionally, the device 1200 further includes:

[0140] The lighting module is used to send a lighting activation command to the drone when the auxiliary lighting function is activated. The lighting activation command is used to instruct the drone to turn on its lighting equipment.

[0141] In the above technical solution, radar can be used to warn of obstacles around the vehicle, and the drone can be controlled to fly to different positions and provide images of the corresponding positions through the operation display screen. The lighting mode can also be turned on, which increases safety, improves the flexibility of displaying vehicle images, reduces blind spots of the image around the vehicle, and provides lighting equipment to assist in providing clearer images in low light conditions.

[0142] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0143] Figure 13 This is a block diagram illustrating an electronic device 1300 according to an exemplary embodiment. For example... Figure 13 As shown, the electronic device 1300 may include a processor 1301 and a memory 1302. The electronic device 1300 may also include one or more of a multimedia component 1303, an input / output (I / O) interface 1304, and a communication component 1305.

[0144] The processor 1301 controls the overall operation of the electronic device 1300 to complete all or part of the steps in the panoramic image display method described above. The memory 1302 stores various types of data to support the operation of the electronic device 1300. This data may include, for example, instructions for any application or method operating on the electronic device 1300, and application-related data such as contact data, sent and received messages, pictures, audio, video, etc. The memory 1302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Multimedia component 1303 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 1302 or transmitted via communication component 1305. The audio component also includes at least one speaker for outputting audio signals. I / O interface 1304 provides an interface between processor 1301 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 1305 is used for wired or wireless communication between the electronic device 1300 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IoT, eMTC, or other 5G, or a combination thereof, is not limited here. Therefore, the corresponding communication component 1305 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0145] In an exemplary embodiment, the electronic device 1300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the panoramic image display method described above.

[0146] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the panoramic image display method described above. For example, the computer-readable storage medium may be the memory 1302 including program instructions, which may be executed by the processor 1301 of the electronic device 1300 to complete the panoramic image display method described above.

[0147] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure. It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this disclosure will not further describe the various possible combinations.

[0148] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A panoramic image display system, characterized in that, The system, applicable to vehicles, includes: a control host, a drone, and a display screen, wherein the display screen is connected to the control host, and the drone communicates with the control host wirelessly. The control host is used to control the drone to take off from the vehicle and fly to the first target position when the panoramic imaging function and the drone imaging assistance function are activated. The drone is used to take pictures of the vehicle at the first target location according to the current image mode of the drone image assistance function to obtain the corresponding drone image, and send the drone image to the control host. The control host is also used to control the display screen to display a panoramic image containing the image of the UAV according to the current display mode of the panoramic image function; The control host is further configured to send a first control command or a second control command to the drone upon receiving a steering signal from the vehicle; the drone is further configured to fly to a first designated position on the left side of the vehicle according to the first control command, or to fly to a second designated position on the right side of the vehicle according to the second control command, to acquire drone images of the first designated position or the second designated position, and to send the drone images of the first designated position or the second designated position to the control host; the control host is further configured to control the display screen to display a panoramic image containing drone images of the first designated position or the second designated position according to the current display mode of the panoramic image function.

2. The system according to claim 1, characterized in that, When the current image mode of the drone image assistance function is overhead mode, the first target position is located directly above the vehicle.

3. The system according to claim 1, characterized in that, Also includes: Multiple radars are respectively installed at multiple locations on the exterior of the vehicle, and the multiple radars are connected to the control host. The radar is used to detect obstacles in the target direction, and when an obstacle is detected in the target direction, to send the distance information of the obstacle to the control host. The target direction is determined according to the position of the radar on the vehicle. The control host is further configured to, upon receiving distance information of the obstacle sent by any of the radars, generate alarm information corresponding to the target direction based on the distance information, wherein the alarm information includes the location of the first target; The control host is also used to control the drone to fly to the first target location to take pictures of the vehicle according to the alarm information, so as to obtain drone images in the target direction, and send the drone images in the target direction to the control host; The control host is also configured to control the display screen to display a panoramic image containing the UAV image in the target direction, according to the current display mode of the panoramic image function.

4. The system according to claim 3, characterized in that, The control host is also used to, upon receiving alarm information corresponding to two or more target directions, set a preset position directly above the vehicle as the first target position, control the drone to fly to the first target position to take pictures of the vehicle, so as to obtain an overhead image of the vehicle, and send the overhead image to the control host. The control host is also used to control the display screen to display a panoramic image containing the overhead view, according to the current display mode of the panoramic image function.

5. The system according to claim 1, characterized in that, The display screen is a touch screen; The display screen is also used to receive the image position selected by the user in the panoramic image and send the image position to the control host; The control host is also used to set the image position as the first target position.

6. The system according to claim 1, characterized in that, The control host is also used to send a lighting command to the drone when the auxiliary lighting function is turned on; The drone is also configured to turn on its lighting device in response to the lighting activation command.

7. A panoramic image display method, characterized in that, Applied to a vehicle, the vehicle including a display screen and a drone, the method includes: With the panoramic imaging function and the drone imaging assistance function activated, the drone is controlled to take off from the vehicle and fly to the first target position, and drone images are acquired from the drone. The drone images are acquired by the drone taking pictures of the vehicle at the first target position according to the current imaging mode of the drone imaging assistance function. Based on the current display mode of the panoramic imaging function, control the display screen to display a panoramic image containing the drone image; The method further includes: upon receiving a steering signal from the vehicle, sending a first control command or a second control command to the drone, wherein the first control command instructs the drone to fly to a first designated position on the left side of the vehicle to obtain a drone image at the first designated position, and the second control command instructs the drone to fly to a second designated position on the right side of the vehicle to obtain a drone image at the second designated position; and controlling the display screen to display a panoramic image containing the drone image at the first designated position or the second designated position, according to the current display mode of the panoramic image function.

8. The method according to claim 7, characterized in that, When the current image mode of the drone image assistance function is overhead mode, the first target position is located directly above the vehicle.

9. The method according to claim 7, characterized in that, The vehicle also includes: multiple radars, which are respectively installed at multiple locations on the exterior of the vehicle; the step of controlling the drone to take off from the vehicle and fly to the first target location when the panoramic imaging function and the drone imaging assistance function are activated, and acquiring drone images from the drone, includes: If an obstacle is detected in the target direction by any of the radars, the distance information of the obstacle is obtained from the radar, and the target direction is determined according to the position of the radar on the vehicle; Based on the distance information, an alarm message corresponding to the target direction is generated, and the alarm message includes the location of the first target. Based on the alarm information, the drone is controlled to fly to the first target location to photograph the vehicle in order to obtain drone images in the direction of the target.

10. The method according to claim 9, characterized in that, The step of controlling the drone to fly to the first target location to photograph the vehicle based on the alarm information, thereby obtaining drone imagery in the target direction, includes: Upon receiving alarm messages corresponding to two or more target directions, the preset position directly above the vehicle is set as the first target position, and the drone is controlled to fly to the first target position to take pictures of the vehicle in order to obtain an overhead image of the vehicle.

11. The method according to claim 7, characterized in that, The display screen is a touch screen, and the method further includes: Receive the image location selected by the user in the panoramic image; Set the image position as the first target position.

12. The method according to claim 7, characterized in that, The method further includes: When the auxiliary lighting function is activated, a lighting activation command is sent to the drone, which instructs the drone to turn on its lighting equipment.

13. A vehicle, characterized in that, include: The panoramic image display system according to any one of claims 1-6.

14. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the panoramic image display method according to any one of claims 7-12.

15. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the panoramic image display method according to any one of claims 7-12.

Images