Vehicle display method, vehicle display device, vehicle and storage medium

By mounting a drone in the car, generating three-dimensional images and adding animation information, the problem of insufficient integration of the car's social interaction function with the vehicle's external environment is solved, and the immersion and rendering effect of the interactive experience is improved.

CN119078648APending Publication Date: 2024-12-06GREAT WALL MOTOR CO LTD
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411353411.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-26
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing car social interaction functions are mainly limited to the interior environment, and lack deep integration with the external environment of the vehicle, resulting in a lack of immersion in the interactive experience between vehicles.

Method used

By a bicycle vehicle equipped with a target drone, an initial three-dimensional image is generated using the first environmental image captured by the bicycle vehicle and the second environmental image captured by the target drone, and the target animation information is determined based on the received target information type, Add animation information to the initial three-dimensional image to realize the superimposed display of the vehicle's external environment and interactive information.

Benefits of technology

It improves the rendering effect and immersion of interactive information between vehicles, enhances the user experience, and ensures the safety of vehicle driving.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119078648A_ABST
    Figure CN119078648A_ABST
Patent Text Reader

Abstract

The invention provides a vehicle display method, a vehicle display device, a vehicle and a storage medium. The method is applied to an own vehicle carrying a target unmanned aerial vehicle. The method comprises the following steps: generating an initial three-dimensional image based on a first environment image shot by an own vehicle and a second environment image shot by a target unmanned aerial vehicle; if the target information sent by the target vehicle is received, determining target animation information corresponding to the target information based on the information type of the target information; wherein the target animation information is used for displaying the target information; adding target animation information at a target position of the initial three-dimensional image to generate a target three-dimensional image; wherein the target position is determined through the current position of the target vehicle; and displaying the target three-dimensional image. According to the method, the vehicle external environment and the interaction information can be displayed in an overlapping manner, so that the immersion of interaction between vehicles is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of vehicles, and more specifically, to a vehicle display method, a vehicle display device, a vehicle, and a storage medium. Background Art

[0002] With the popularization of automobiles, people's expectations for vehicles are no longer limited to basic driving functions, but instead they pursue richer and more convenient interactive experiences.

[0003] In related technologies, the social interaction function of automobiles is mainly concentrated between passengers in the car, and basic functions such as music playback, navigation, and phone calls are realized through the in-vehicle information system. However, the existing interaction is usually limited to the in-car environment and lacks deep integration with the external environment of the vehicle. Therefore, how to superimpose the external environment of the vehicle with the interactive information to improve the immersiveness of the interaction between vehicles has become an urgent problem to be solved. Summary of the invention

[0004] The present application provides a vehicle display method, a vehicle display device, a vehicle and a storage medium. The method can display the interaction information of the vehicle, thereby improving the immersiveness of the interaction between the vehicles.

[0005] In a first aspect, a vehicle display method is provided, the method being applied to a vehicle carrying a target drone; the method comprising:

[0006] Generate an initial three-dimensional image based on the first environment image taken by the ego vehicle and the second environment image taken by the target drone;

[0007] If target information sent by a target vehicle is received, target animation information corresponding to the target information is determined based on the information type of the target information; wherein the target animation information is used to display the target information;

[0008] Adding the target animation information at the target position of the initial three-dimensional image to generate a target three-dimensional image; wherein the target position is determined by the current position of the target vehicle;

[0009] The target three-dimensional image is displayed.

[0010] The above-mentioned technical scheme, when receiving the target information sent by the target vehicle, determines the target animation information corresponding to the target information based on the information type of the target information, and adds the target animation information at the target position of the initial three-dimensional image to generate a target three-dimensional image; compared with the prior art in which information interaction is achieved through an on-board information system, the initial three-dimensional image in this application is generated by a first environment image taken by the vehicle and a second environment image taken by the target drone, and the target image is generated by superimposing the target animation information displaying the target information on the initial three-dimensional image, and the external environment of the vehicle and the interactive information are superimposed and displayed, which can improve the rendering effect of the interactive information display of the vehicle, thereby improving the immersive feeling of interaction between vehicles.

[0011] In combination with the first aspect, in some possible implementations, the method further includes:

[0012] Based on the information type of the target information, the target generation method is obtained by matching the preset generation relationship; wherein the preset generation relationship is used to indicate the corresponding relationship between the preset generation method and the preset information type;

[0013] The target information is processed based on the target generation method to obtain the target animation information.

[0014] The above technical scheme matches the information type of the target information with the preset generation relationship to obtain the target generation method, and processes the target information based on the target generation method to obtain the target animation information; since the preset generation relationship indicates the correspondence between the preset generation method and the preset information type, the target information is processed by the target generation method corresponding to the information type, and target information of different information types can be processed uniformly, thereby improving the readability of the target animation information while displaying the target information in a more vivid way.

[0015] In combination with the first aspect and the above implementation manner, in some possible implementation manners, the information type of the target information includes at least one of an expression type, a text type, and a voice type; and the method further includes:

[0016] If the information type of the target information includes an expression type, determining a target three-dimensional expression in a preset three-dimensional expression that matches the expression information in the target information as the target animation information;

[0017] If the information type of the target information includes the text type and / or the voice type, the text information and / or voice information in the target information output by the preset object is determined as the target animation information.

[0018] In the above technical scheme, if the information type of the target information includes an expression type, the target three-dimensional expression in the preset three-dimensional expression that matches the expression information in the target information is determined as the target animation information; by rendering the expression information as a three-dimensional expression, that is, the target animation information is compared with the expression information, the rendering effect of the interaction can be improved; if the information type of the target information includes a text type and / or a voice type, the text information and / or voice information in the target information output by the preset object is determined as the target animation information; by outputting the text information and / or voice information through the preset object, the user does not need to view the target information, the vehicle driving safety will not be affected, and the output through the preset object can improve the fun of the interaction.

[0019] In combination with the first aspect and the above implementation manner, in some possible implementation manners, if the information type of the target information includes the text type, the method further includes:

[0020] Determining the character length of the text information;

[0021] If the character length is less than the preset character length, the preset animation relationship is matched with the text information to obtain the target animation information; wherein the preset animation relationship is used to indicate the corresponding relationship between the preset text information and the preset animation information.

[0022] In the above technical scheme, if the character length is less than the preset character length, the preset animation relationship is matched with the text information to obtain the target animation information; since the preset animation relationship indicates the correspondence between the preset text information and the preset animation information, when the character length of the text information is less than the preset character length, the target animation information obtained by matching the text information can fully express the text information, thereby improving the interactive rendering effect.

[0023] In combination with the first aspect and the above implementation manner, in some possible implementation manners, the method further includes:

[0024] Performing feature matching on the first environment image and the second environment image to obtain feature point pairs;

[0025] Based on the feature point pairs, the first environment image and the second environment image are stitched together to obtain a stitched environment image;

[0026] The initial three-dimensional image is determined based on information of each feature point in the stitched environment image.

[0027] The above technical scheme performs feature matching on the first environment image and the second environment image to obtain feature point pairs, and based on the feature point pairs, the first environment image and the second environment image are spliced ​​to obtain a spliced ​​environment image, and then the initial three-dimensional image is determined based on the information of each feature point in the spliced ​​environment image; since the spliced ​​environment image is obtained by splicing the first environment image taken by the self-vehicle and the second environment image taken by the target drone, the spliced ​​environment image can be consistent with the environment in which the self-vehicle is located, so that the initial three-dimensional image is generated by the information of each feature point in the spliced ​​environment image, and the authenticity of the generated initial three-dimensional image can be ensured.

[0028] In combination with the first aspect and the above implementations, in some possible implementations, the method further includes: determining coordinate information and depth information of each feature point in the stitched environment image;

[0029] Based on the coordinate information and depth information of each feature point, determine the three-dimensional coordinates of each feature point in the same three-dimensional coordinate system;

[0030] The initial three-dimensional image is generated based on the three-dimensional coordinates of each feature point.

[0031] The above technical solution determines the corresponding three-dimensional coordinates by stitching the coordinate information and depth information of each feature point in the environmental image, thereby improving the accuracy of the three-dimensional coordinates of each feature point in the stitched image, and generating an initial three-dimensional image based on the three-dimensional coordinates of each feature point, thereby ensuring the consistency of the generated initial three-dimensional image with the environment in which the vehicle is located, thereby improving the authenticity of the initial three-dimensional image.

[0032] In combination with the first aspect and the above implementation manner, in some possible implementation manners, the method further includes:

[0033] In response to the three-dimensional image start instruction, sending an image acquisition instruction to the target UAV;

[0034] If the second environment image sent by the target UAV is received, the initial three-dimensional image is generated based on the first environment image and the second environment image;

[0035] The initial three-dimensional image is displayed.

[0036] The above technical solution responds to the three-dimensional image start instruction and sends an image acquisition instruction to the target drone. If the second environment image sent by the target drone is received, an initial three-dimensional image is generated through the first environment image and the second environment image, and the initial three-dimensional image is displayed; through the three-dimensional image switch instruction, the three-dimensional image can be turned on or off in time, thereby displaying the three-dimensional environment image of the vehicle in real time, which can enhance the user's sense of immersion.

[0037] In a second aspect, a vehicle display device is provided, the prompting device comprising:

[0038] A generating module, configured to generate an initial three-dimensional image based on a first environment image taken by the ego vehicle and a second environment image taken by the target UAV;

[0039] A determination module, configured to determine target animation information corresponding to the target information based on the information type of the target information if target information sent by the target vehicle is received; wherein the target animation information is used to display the target information;

[0040] An adding module, used for adding the target animation information at the target position of the initial three-dimensional image to generate a target three-dimensional image; wherein the target position is determined by the current position of the target vehicle;

[0041] A display module is used to display the target three-dimensional image.

[0042] In a third aspect, a vehicle is provided, comprising a memory and a processor, wherein the memory is used to store executable program code; the processor is used to call and run the executable program code from the memory, so that the vehicle executes the vehicle display method in the above-mentioned first aspect or any possible implementation method of the first aspect.

[0043] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the computer executes the vehicle display method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0044] In a fifth aspect, a computer program product is provided, which includes: a computer program code, which, when executed on a computer, enables the computer to execute the vehicle display method in the first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a scene schematic diagram of a vehicle display method provided in an embodiment of the present application;

[0046] Figure 2 is a schematic flow chart of a vehicle display method provided in an embodiment of the present application;

[0047] Figure 3 is a schematic flow chart of another vehicle display method provided in an embodiment of the present application;

[0048] Figure 4is a schematic flow chart of another vehicle display method provided in an embodiment of the present application;

[0049] Figure 5 is a structural schematic diagram of a vehicle display device provided in an embodiment of the present application;

[0050] Figure 6 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The technical solution in the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0052] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0053] It should be noted that in the related art, the social interaction function of the car is mainly concentrated between passengers in the car, and basic functions such as music playback, navigation, and phone calls are realized through the in-vehicle information system. However, the existing interaction is usually limited to the in-car environment and lacks deep integration with the external environment of the vehicle. In view of this, the present application provides a vehicle display method, a vehicle display device, a vehicle, and a storage medium, which can display the interactive information of the vehicle, thereby improving the immersiveness of the interaction between vehicles.

[0054] Figure 1 It is a scene schematic diagram of a vehicle display method provided in an embodiment of the present application.

[0055] For example, Figure 1 As shown, the ego vehicle 100 displays the target three-dimensional image through the display screen 110. Specifically, an initial three-dimensional image (not shown in the figure) is generated through the first environment image taken by the ego vehicle 100 and the second environment image taken by the target drone 130 carried by the ego vehicle 100. The initial three-dimensional image includes the target drone 130, the target vehicle 120 and the ego vehicle 100. When receiving the target information sent by the target vehicle 120, the information type of the target information is determined to be an expression type, and the target animation information 140 corresponding to the target information (expression information) is determined.

[0056] Furthermore, the target position of the target vehicle 120 in the initial three-dimensional image is determined by the current position of the target vehicle 120, and target animation information 140 is added at the target position of the initial three-dimensional image to generate a target three-dimensional image, and the target three-dimensional image is displayed on the display screen 110 of the self-vehicle 100.

[0057] Figure 2 It is a schematic flow chart of a vehicle display method provided in an embodiment of the present application.

[0058] For example, Figure 2 The method shown can be executed by a vehicle controller or chip of the vehicle.

[0059] For example, Figure 2 The method shown is applied to the ego vehicle carrying the target UAV.

[0060] For example, Figure 2 As shown, the method 200 includes the following processes:

[0061] S210, generating an initial three-dimensional image based on a first environment image taken by the ego vehicle and a second environment image taken by the target UAV.

[0062] It should be noted that the self-vehicle is equipped with a target drone, which is equipped with a camera device that can capture images of the external environment of the self-vehicle. In addition, the target drone has a positioning function, and the user can control the target drone to fly autonomously through the self-vehicle. For example, the target drone can fly around the self-vehicle according to user instructions, capture images of the external environment, and transmit the external environment images to the self-vehicle.

[0063] Exemplarily, a first environment image is obtained by photographing a camera installed on the ego vehicle, and a second environment image is obtained by photographing a camera installed on the target drone. It is understandable that the first environment image photographed by the camera of the ego vehicle is used to indicate the external environment image of the ego vehicle, and the second environment image photographed by the camera of the target drone is also used to indicate the external environment image of the ego vehicle. The first environment images can be the same environment image or different environment images; through multiple first environment images photographed by the ego vehicle and multiple second environment images photographed by the target drone mounted on the ego vehicle, an initial three-dimensional image of the ego vehicle can be generated, and the initial three-dimensional image is used to represent the three-dimensional environment image of the ego vehicle.

[0064] Specifically, feature matching is performed on the first environment image and the second environment image to obtain feature point pairs, and based on the feature point pairs, the first environment image and the second environment image are spliced ​​to obtain a spliced ​​environment image; then, based on the information of each feature point in the spliced ​​environment image, an initial three-dimensional image is determined.

[0065] Exemplarily, the feature point pair is used to indicate a set of corresponding feature points in the first environment image and the second environment image.

[0066] Exemplarily, a feature detection algorithm can be used to detect the first environment image and the second environment image, determine the feature points (key points) in the first environment image and the second environment image, and match the feature points in the first environment image and the second ring image through a feature matching algorithm to obtain a feature point pair. Further, based on the feature point pair, for a planar scene, the transformation relationship between the first environment image and the second environment image is determined by a homography matrix (for a non-planar scene, a basic matrix), and the first environment image and the second environment image are mapped to the same plane, and then the first environment image and the second environment image are spliced ​​to obtain a spliced ​​environment image.

[0067] Exemplarily, the stitched environment image is used to represent a panoramic image where the ego vehicle is located; it can be understood that the panoramic image is a two-dimensional image.

[0068] Optionally, the feature detection algorithm may be, but is not limited to: Scale Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), and the like.

[0069] The above technical scheme performs feature matching on the first environment image and the second environment image to obtain feature point pairs, and based on the feature point pairs, the first environment image and the second environment image are spliced ​​to obtain a spliced ​​environment image, and then the initial three-dimensional image is determined based on the information of each feature point in the spliced ​​environment image; since the spliced ​​environment image is obtained by splicing the first environment image taken by the self-vehicle and the second environment image taken by the target drone, the spliced ​​environment image can be consistent with the environment in which the self-vehicle is located, so that the initial three-dimensional image is generated by the information of each feature point in the spliced ​​environment image, and the authenticity of the generated initial three-dimensional image can be ensured.

[0070] Exemplarily, the coordinate information and depth information of each feature point in the stitched environment image are determined, and based on the coordinate information and depth information of each feature point, the three-dimensional coordinates of each feature point in the same three-dimensional coordinate system are determined, and then based on the three-dimensional coordinates of each feature point, an initial three-dimensional image is generated.

[0071] Exemplarily, feature points in an image may be detected by a feature detection algorithm, and feature points in an image may be used to indicate recognizable points in the image. Feature points may correspond to pixel points in the image, and feature points may also be corner points of the image.

[0072] Exemplarily, the depth information of each feature point in the stitched environment image is used to indicate the straight-line distance from the physical point corresponding to the feature point to the shooting device, that is, the depth information indicates the straight-line distance from the actual environment point corresponding to the feature point to the shooting device.

[0073] Exemplarily, the coordinate information of each feature point in the stitched environment image is used to indicate the two-dimensional coordinates of each feature point in the coordinate system.

[0074] Exemplarily, based on the coordinate information (two-dimensional coordinates) and depth information of each feature point in the stitched environment image, the reference point of the vehicle is used as the origin to determine the three-dimensional coordinates corresponding to each feature point in the same three-dimensional coordinate system. For example, the feature point with coordinates (100, 200) in the stitched image has a depth information of 5 meters. The center of mass of the vehicle is used as the origin of the three-dimensional coordinate system, and the three-dimensional coordinates of the feature point in the three-dimensional coordinate system are obtained through the internal and external parameters of the shooting device. Furthermore, the point cloud data is determined based on the three-dimensional coordinates corresponding to all feature points in the stitched environment image, and the data is reconstructed using a point cloud reconstruction algorithm to generate a three-dimensional network model, and the texture of the stitched image is mapped to the three-dimensional network model to generate an initial three-dimensional image.

[0075] Optionally, the reference point of the ego vehicle may be the center of mass of the ego vehicle, or the center of gravity of the ego vehicle, etc. The reference point of the ego vehicle may be determined based on actual conditions and is not specifically limited here.

[0076] The above technical solution determines the corresponding three-dimensional coordinates by stitching the coordinate information and depth information of each feature point in the environmental image, thereby improving the accuracy of the three-dimensional coordinates of each feature point in the stitched image, and generating an initial three-dimensional image based on the three-dimensional coordinates of each feature point, thereby ensuring the consistency of the generated initial three-dimensional image with the environment in which the vehicle is located, thereby improving the authenticity of the initial three-dimensional image.

[0077] Exemplarily, in response to a three-dimensional image start instruction, an image acquisition instruction is sent to the target drone. If a second environment image sent by the target drone is received, an initial three-dimensional image is generated through the first environment image and the second environment image, and the initial three-dimensional image is displayed.

[0078] For example, the 3D image opening instruction can be opened by voice control; for example, "Hello, please open the 3D image, thank you." Alternatively, the 3D image opening instruction can be enabled by opening a control corresponding to the 3D image. The 3D image opening instruction can be determined according to actual conditions and is not specifically limited here.

[0079] Exemplarily, the image acquisition instruction is used to instruct the target drone to acquire an environmental image of a preset area, and the preset area can be determined according to the position of the vehicle. For example, the preset area can be an area within a range of 50 meters from the vehicle.

[0080] Optionally, the method of generating the initial three-dimensional image by using the first environment image and the second environment image may refer to the method in the aforementioned disclosed embodiment, which will not be described in detail here.

[0081] The above technical solution responds to the three-dimensional image start instruction and sends an image acquisition instruction to the target drone. If the second environment image sent by the target drone is received, an initial three-dimensional image is generated through the first environment image and the second environment image, and the initial three-dimensional image is displayed; through the three-dimensional image switch instruction, the three-dimensional image can be turned on or off in time, thereby displaying the three-dimensional environment image of the vehicle in real time, which can enhance the user's sense of immersion.

[0082] Exemplarily, the initial three-dimensional image can be generated by the vehicle; or, the vehicle is connected to a server, the vehicle sends the first environment image and the second environment image to the server, the server generates the initial three-dimensional image, and the initial three-dimensional image is displayed on the display screen of the vehicle; or, the vehicle is communicatively connected to a third-party device, the third-party device has image processing capabilities, the vehicle sends the first environment image and the second environment image to the third-party device, the third-party device generates the initial three-dimensional image, and the initial three-dimensional image is displayed on the display screen of the vehicle.

[0083] It can be understood that when the vehicle is connected to the server (third-party device), the vehicle does not need to occupy resources, and the first environment image and the second environment image are processed by the server (third-party device) to generate an initial three-dimensional image. Since the vehicle resources do not need to be occupied to process the environment image during the image processing process, the computing power of the vehicle can be saved, thereby improving the user experience without affecting other functions of the vehicle.

[0084] S220: If target information sent by the target vehicle is received, target animation information corresponding to the target information is determined based on the information type of the target information.

[0085] It should be noted that the target animation information is used to display the target information, and the user can obtain the target information by viewing the target animation information; that is, the driver and passengers of the own vehicle can determine the target information sent by the target vehicle through the target animation information displayed on the display screen of the own vehicle.

[0086] Optionally, the target information may include expression information, text information or voice information; or, the target information may include expression information and text information. The target information may be determined according to actual conditions and is not specifically limited here. It is understandable that the target information here is used to indicate the information currently being sent, that is, a certain interactive information.

[0087] Exemplarily, the information type of the target information includes at least one of an expression type, a text type, and a voice type. For example, the information type of the target information is an expression type, and may also be a text type or a voice type; or, the information type of the target information includes an expression type and a text type. It is understandable that, in the case where the information type of the target information includes an expression type and a text type, the target information includes expression information and text information; and the target information corresponds to the information type of the target information.

[0088] Exemplarily, based on the information type of the target information, the target animation information corresponding to the target information is determined. Specifically, based on the information type of the target information, the target generation method is obtained by matching the preset generation relationship, and the preset generation relationship is used to indicate the corresponding relationship between the preset generation method and the preset information type; based on the target generation method, the target information is processed to obtain the target animation information.

[0089] Exemplarily, the target generation method is used to indicate a method for processing target information. By processing the target information through the target generation method, target animation information can be obtained.

[0090] For example, the information type of the target information is an expression type, and the expression information (target information) corresponding to the expression type is "flower". The preset generation relationship is matched with the expression type, and the target generation method is determined to be "rendering the expression information as three-dimensional expression information, and voice outputting the meaning of the expression information". The expression information "flower" is rendered to obtain three-dimensional expression information (three-dimensional animation of flowers), and the voice output of the meaning of "flower" is "send you a small flower", and the target animation information is three-dimensional expression information and the voice output is "send you a small flower".

[0091] Exemplarily, the preset generation relationship indicates the correspondence between the preset generation method and the preset information type; for example, the preset information type includes the expression type, and the corresponding preset generation relationship is to perform three-dimensional rendering on the expression information corresponding to the expression type to generate three-dimensional expression information; the preset information type includes the expression type and the text type, and the corresponding preset generation relationship is to generate three-dimensional expression information for the expression information corresponding to the expression type, and output the text information corresponding to the text type through an animated character. The preset generation relationship can be output according to actual conditions, and is not specifically limited here.

[0092] The above technical scheme matches the information type of the target information with the preset generation relationship to obtain the target generation method, and processes the target information based on the target generation method to obtain the target animation information; since the preset generation relationship indicates the correspondence between the preset generation method and the preset information type, the target information is processed by the target generation method corresponding to the information type, and target information of different information types can be processed uniformly, thereby improving the readability of the target animation information while displaying the target information in a more vivid way.

[0093] It should be noted that the information type of the target information includes at least one of an expression type, a text type, and a voice type. For example, the information type of the target information may include only an expression type, or may include an expression type and a text type. In addition, if the information type of the target information includes an expression type (text type or voice type), then the target information includes expression information (text information or voice information).

[0094] In one example, in the case where the target generation mode is determined based on the information type of the target information, the target information is processed based on the target generation mode to obtain the target animation information. Specifically, if the information type of the target information includes the expression type, the target three-dimensional expression in the preset three-dimensional expression that matches the expression information in the target information is determined as the target animation information; if the information type of the target information includes the text type and / or the voice type, the text information and / or the voice information in the preset object output target information is determined as the target animation information.

[0095] Exemplarily, the preset object can be set to a preset character, a preset animal, etc.; or, the preset object can be selected at will. For example, the preset object library includes preset animation characters, preset animals, and preset plants. During the information display process, any preset object in the preset object library can be matched at will to output the target information.

[0096] In the above technical scheme, if the information type of the target information includes an expression type, the target three-dimensional expression in the preset three-dimensional expression that matches the expression information in the target information is determined as the target animation information; by rendering the expression information as a three-dimensional expression, that is, the target animation information is compared with the expression information, the rendering effect of the interaction can be improved; if the information type of the target information includes a text type and / or a voice type, the text information and / or voice information in the target information output by the preset object is determined as the target animation information; by outputting the text information and / or voice information through the preset object, the user does not need to view the target information, the vehicle driving safety will not be affected, and the output through the preset object can improve the fun of the interaction.

[0097] In another example, if the information type of the target information includes a text type, the character length of the text information can also be determined first. If the character length is greater than or equal to the preset character length, the text information in the preset object output target information is determined as the target animation information; if the character length is less than the preset character length, the preset animation relationship is matched with the text information to obtain the target animation information, and the preset animation relationship is used to indicate the correspondence between the preset text information and the preset animation information.

[0098] Exemplarily, the character length is used to indicate the number of characters in the text message, and the character length includes letters, numbers, punctuation marks, words, etc.; for example, if the text message is “Hello, what a coincidence~”, the corresponding character length is 7.

[0099] It can be understood that if the character length in the text information is greater than or equal to the preset character length, the text information contains a large amount of information, and it is difficult to fully express the text information through directly matching animation information; if the character length in the text information is less than the preset character length, the text information can be fully expressed through directly matching animation information.

[0100] Optionally, the preset character length may be 3, 5, 6, etc., which is not specifically limited here.

[0101] Exemplarily, the preset animation relationship indicates the corresponding relationship between the preset text information and the preset animation information; for example, the preset text information is “Thank you”, and the corresponding preset animation information includes the three-dimensional animation information corresponding to “Thank you”.

[0102] In the above technical scheme, if the character length is less than the preset character length, the preset animation relationship is matched with the text information to obtain the target animation information; since the preset animation relationship indicates the correspondence between the preset text information and the preset animation information, when the character length of the text information is less than the preset character length, the target animation information obtained by matching the text information can fully express the text information, thereby improving the interactive rendering effect.

[0103] In another example, if the information type of the target information includes a voice type, the language type of the voice information can also be determined to determine whether the language type is equal to the target language type. If the language type is equal to the target language type, the voice information output by the preset object is determined as the target animation information; if the language type is not equal to the target language type, the voice information is converted into target voice information corresponding to the target language type, and the target voice information output by the preset object is determined as the target animation information.

[0104] Specifically, when the information type of the target information includes a voice type, it is determined that the language type of the voice information is English, and the target language type is Chinese, then the voice information is translated in real time using a natural language processing algorithm and a machine learning algorithm to obtain the target voice information of Chinese language type, and then the target animation information is determined. It can be understood that by performing real-time translation of voice information of a different language type from the target language type, users with different language backgrounds can be supported to communicate seamlessly, thereby improving the user experience.

[0105] It can be understood that the interactive terminals of the target vehicle and the self-vehicle can be installed in the steering wheel, seat armrests and center console of the vehicle, and interactive information can be sent through voice commands, touch operations, etc., which can ensure the security of data interaction during vehicle driving.

[0106] Exemplarily, the interaction information between vehicles (e.g., target information) can be pushed in real time, that is, the interaction information is pushed to the relevant vehicles in real time through the interaction terminal installed in the vehicle to ensure that the information can be received and responded to in time; or, the interaction information between vehicles can be sent to the server and stored by the server. When the user logs in to the interaction terminal, the server synchronizes the interaction information to the vehicle to ensure that no information is lost.

[0107] S230, adding target animation information at the target position of the initial three-dimensional image to generate a target three-dimensional image.

[0108] Exemplarily, the target position is determined by the current position of the target vehicle. Specifically, the current position of the target vehicle may be a positioning coordinate, and the positioning coordinate of the target vehicle is converted into a corresponding three-dimensional coordinate in the initial three-dimensional image, so that the target position corresponding to the target vehicle in the initial three-dimensional image can be determined. That is, the three-dimensional coordinate is used as the target position.

[0109] Exemplarily, the longitude and latitude coordinates of the current position of the target vehicle are obtained, the target depth information of the target vehicle in the stitched environment image is determined, and the corresponding three-dimensional coordinates of the target vehicle in the initial three-dimensional image are determined based on the longitude and latitude coordinates and the target depth information of the target vehicle.

[0110] Optionally, the longitude and latitude coordinates of the current position of the target vehicle may be the longitude and latitude coordinates corresponding to the center of mass of the target vehicle, or the longitude and latitude coordinates corresponding to the center of gravity of the target vehicle, etc., which are not specifically limited here.

[0111] For example, the longitude and latitude coordinates of the current position of the target vehicle are the longitude and latitude coordinates corresponding to the center of gravity of the target vehicle, and the target depth information of the center of gravity of the target vehicle in the stitched environment image is determined. Assuming that the longitude in the longitude and latitude coordinates is λ, the latitude is φ, and the target depth information is h, the longitude and latitude coordinates of the current position of the target vehicle are converted into the initial three-dimensional coordinates (X, Y, Z) with the center of the earth as the origin of the three-dimensional coordinates. The expression of the three-dimensional coordinates (X, Y, Z) is expressed as follows:

[0112] X=R*cos(φ)*cos(λ)

[0113] Y=R*cos(φ)*sin(λ)

[0114] Z=R*sin(φ)+h

[0115] Among them, R represents the radius of the earth, φ represents the latitude in the longitude and latitude coordinates, λ represents the longitude in the longitude and latitude coordinates, and h represents the target depth information corresponding to the current position of the target vehicle.

[0116] Furthermore, the expression for converting the initial three-dimensional coordinates (X, Y, Z) with the center of the earth as the origin of the three-dimensional coordinates into the three-dimensional coordinates (X2, Y2, Z2) with the reference point of the vehicle as the origin (X1, Y1, Z1) is as follows:

[0117] X 2= X-X1

[0118] Y 2= Y-Y1

[0119] Z 2= Z-Z1

[0120] Therefore, by converting the positioning coordinates of the current position of the target vehicle, the positioning coordinates of the target vehicle are converted into corresponding three-dimensional coordinates in the initial three-dimensional image, so that the three-dimensional coordinates can be used as the target position corresponding to the target vehicle in the initial three-dimensional image.

[0121] Exemplarily, the initial three-dimensional image may change as the vehicle moves. At the target position of the initial three-dimensional image, the target animation information is used to cover the screen at the target position of the initial three-dimensional image, thereby generating a target three-dimensional image.

[0122] S240, displaying a target three-dimensional image.

[0123] Exemplarily, the target three-dimensional image is displayed on the display screen of the vehicle. The display screen may be a HUD, a central control screen, a dashboard, or other vehicle-mounted screens, which are not specifically limited here. In addition, the user may also wear an augmented reality device (AR device). Wearing an AR device to view the target three-dimensional image can enhance the user's perception of the environment in which the vehicle is located, thereby improving the interactive rendering effect.

[0124] It is understandable that by superimposing the initial three-dimensional image and the target animation information showing the interactive information, the rendering effect of the information display can be improved, thereby improving the user experience.

[0125] Exemplarily, the target vehicle is equipped with a first drone, and both the first drone and the target drone are equipped with camera devices, so that the driver and passengers of the target vehicle can have a video call with the driver and passengers of the own vehicle. Specifically, the target drone collects voice data and video data of the own vehicle and sends it to the server, which is then transmitted to the target vehicle. The first drone collects voice data and video data of the target vehicle and sends it to the server, which is then transmitted to the own vehicle, thereby realizing a video call between the own vehicle and the target vehicle.

[0126] Exemplarily, the target drone can also capture the facial image of the user of the vehicle, and recognize the user's facial image through the trained emotion recognition model to obtain facial image features, and based on the facial image features, determine the target emotion type corresponding to the facial image, which can be happiness, sadness, anger, etc.

[0127] Furthermore, the target drone adjusts its flight attitude according to the target emotion type, or plays music, etc. For example, if the target emotion type is "happy", the target drone can be controlled to play relaxing and pleasant music.

[0128] The above-mentioned technical scheme, when receiving the target information sent by the target vehicle, determines the target animation information corresponding to the target information based on the information type of the target information, and adds the target animation information at the target position of the initial three-dimensional image to generate a target three-dimensional image; compared with the prior art in which information interaction is achieved through an on-board information system, the initial three-dimensional image in this application is generated by a first environment image taken by the vehicle and a second environment image taken by the target drone, and the target image is generated by superimposing the target animation information displaying the target information on the initial three-dimensional image, and the external environment of the vehicle and the interactive information are superimposed and displayed, which can improve the rendering effect of the interactive information display of the vehicle, thereby improving the immersive feeling of interaction between vehicles.

[0129] Figure 3 It is a schematic flow chart of another vehicle display method provided in an embodiment of the present application.

[0130] For example, Figure 3 The method shown can be executed by a vehicle controller or chip in the vehicle.

[0131] For example, Figure 3 The method shown is applied to the ego vehicle carrying the target UAV.

[0132] For example, Figure 3 As shown, the method 300 includes the following processes:

[0133] S310, in response to the three-dimensional image start instruction, sending an image acquisition instruction to the target UAV.

[0134] For example, when the ego vehicle is driving, the target drone does not collect images of the external environment of the ego vehicle in real time. Instead, when the ego vehicle turns on the display of three-dimensional images, it sends image collection instructions to the target drone, so that the target drone collects images of the external environment where the ego vehicle is located.

[0135] Exemplarily, in response to the start instruction of the three-dimensional image, an image acquisition instruction is sent to the target drone, and the image acquisition instruction is used to instruct the target drone to collect an image of the external environment within a preset range of the vehicle. The preset range can be a range of 50 meters from the vehicle, or a range of 100 meters from the vehicle, etc. The preset range can be determined according to actual conditions and is not specifically limited here.

[0136] For example, the 3D image opening instruction can be opened by voice control; or the 3D image opening instruction can be clicked on the 3D image function control. The 3D image opening instruction can be determined according to actual conditions and is not specifically limited here.

[0137] S320: If a second environment image sent by the target UAV is received, feature matching is performed on the first environment image and the second environment image taken by the vehicle to obtain a feature point pair.

[0138] Exemplarily, when the ego vehicle detects that the three-dimensional image is in the on state, it sends an image acquisition instruction to the target drone. The target drone acquires the external environment image (second environment image) of the ego vehicle and sends the second environment image to the ego vehicle. After receiving the second environment image sent by the target drone, the ego vehicle performs feature matching on the first environment image and the second environment image taken by the ego vehicle to obtain matching feature points.

[0139] Specifically, a feature detection algorithm can be used to detect the first environment image and the second environment image to determine feature points in the first environment image and the second environment image, and a feature matching algorithm can be used to match the feature points in the first environment image and the second ring image to obtain a feature point pair.

[0140] Exemplarily, the feature point pair is used to indicate a set of corresponding feature points in the first environment image and the second environment image.

[0141] S330: Based on the feature point pairs, the first environment image and the second environment image are stitched together to obtain a stitched environment image.

[0142] Exemplarily, the first environment image and the second environment image are spliced ​​according to the feature point pairs to obtain a spliced ​​environment image. Specifically, based on the feature point pairs, the transformation relationship between the first environment image and the second environment image is determined by a homography matrix for a planar scene and a basic matrix for a non-planar scene, and the first environment image and the second environment image are mapped to the same plane, and then the first environment image and the second environment image are spliced ​​to obtain a spliced ​​environment image.

[0143] Exemplarily, the stitched environment image is used to represent a panoramic image where the ego vehicle is located; it can be understood that the panoramic image is a two-dimensional image.

[0144] S340, determining the coordinate information and depth information of each feature point in the stitched environment image.

[0145] Exemplarily, the coordinate information and depth information of each feature point in the stitched environment image are determined. Specifically, the coordinate information of each feature point in the stitched environment image is the coordinate in the same two-dimensional coordinate system, and the depth information is used to indicate the straight-line distance from the physical point corresponding to the feature point to the shooting device, that is, the depth information indicates the straight-line distance from the actual environment point corresponding to the feature point to the shooting device.

[0146] Exemplarily, feature points in the stitched environment image can be detected by a feature detection algorithm. The feature points in the stitched environment image can be used to indicate identifiable points in the stitched environment image. The feature points can correspond to pixel points in the stitched environment image. The feature points can also be corner points of the stitched environment image.

[0147] S350, determining the three-dimensional coordinates of each feature point in the same three-dimensional coordinate system based on the coordinate information and depth information of each feature point.

[0148] Exemplarily, when the coordinate information and depth information of each feature point in the stitched environment image are determined, the three-dimensional coordinates of each feature point in the same three-dimensional coordinate system are determined based on the coordinate information and depth information of each feature point. Specifically, the two-dimensional image coordinates and depth information are converted into three-dimensional world coordinates by a coordinate conversion method, thereby generating the three-dimensional coordinates of each feature point in the same three-dimensional coordinate system.

[0149] For example, the two-dimensional coordinates of a feature point in the stitched image are (100, 200), and the depth information is 5 meters. The three-dimensional coordinates of the feature point in the three-dimensional coordinate system are obtained through the intrinsic and extrinsic parameters of the shooting device.

[0150] S360: Generate an initial three-dimensional image based on the three-dimensional coordinates of each feature point.

[0151] Exemplarily, point cloud data is determined according to the three-dimensional coordinates corresponding to all feature points in the stitched environment image, and the data is reconstructed using a point cloud reconstruction algorithm to generate a three-dimensional network model. On this basis, the texture of the stitched environment image is mapped to the three-dimensional network model to generate an initial three-dimensional image.

[0152] S370, displaying an initial three-dimensional image.

[0153] Exemplarily, in the absence of interactive information, in response to a command to start a three-dimensional image, an initial three-dimensional image is generated through a first environmental image taken by the own vehicle and a second environmental image collected by a target drone carried by the own vehicle, and the initial three-dimensional image is displayed on the display screen of the own vehicle, thereby improving the rendering effect of the image display, enhancing the sense of immersion of the driver and passengers, and improving the user experience.

[0154] The above technical solution responds to the three-dimensional image start instruction and sends an image acquisition instruction to the target drone. If the second environment image sent by the target drone is received, an initial three-dimensional image is generated through the first environment image and the second environment image, and the initial three-dimensional image is displayed; through the three-dimensional image switch instruction, the three-dimensional image can be turned on or off in time, thereby displaying the three-dimensional environment image of the vehicle in real time, which can enhance the user's sense of immersion.

[0155] Figure 4 It is a schematic flow chart of another vehicle display method provided in an embodiment of the present application.

[0156] For example, Figure 4 The method shown can be executed by a vehicle controller or chip in the vehicle.

[0157] For example, Figure 4 The method shown is applied to the ego vehicle carrying the target UAV.

[0158] For example, Figure 4 As shown, the method 400 includes the following processes:

[0159] S410, generating an initial three-dimensional image based on a first environment image taken by the ego vehicle and a second environment image taken by the target UAV.

[0160] Exemplarily, a first environment image is obtained by photographing a camera device configured on the vehicle, and a second environment image is obtained by photographing a target drone mounted on the vehicle, and an initial three-dimensional image is generated based on the first environment image and the second environment image.

[0161] Optionally, the method of generating the initial three-dimensional image according to the first environment image and the second environment image may refer to the aforementioned disclosed embodiment, which will not be described in detail here.

[0162] S420: If target information sent by the target vehicle is received, the target generation method is obtained by matching the target information type with the preset generation relationship.

[0163] Exemplarily, the information type of the target information includes at least one of an expression type, a text type, and a voice type. For example, the information type of the target information is an expression type, and may also be a text type or a voice type; or, the information type of the target information includes an expression type and a text type. It is understandable that, in the case where the information type of the target information includes an expression type and a text type, the target information includes expression information and text information; and the target information corresponds to the information type of the target information.

[0164] Exemplarily, the information type of the target information is matched with the preset generation relationship to obtain the target generation method, and the preset generation relationship is used to indicate the correspondence between the preset generation method and the preset information type; the target information is processed based on the target generation method to obtain the target animation information.

[0165] Exemplarily, the target generation method is used to indicate a method for processing target information. By processing the target information through the target generation method, target animation information can be obtained.

[0166] S430: If the information type of the target information includes an expression type, a target three-dimensional expression in the preset three-dimensional expressions that matches the expression information in the target information is determined as the target animation information.

[0167] It is understandable that, when the information type of the target information includes the expression type, the target generation method is: determining the target three-dimensional expression in the preset three-dimensional expression that matches the expression information in the target information as the target animation information.

[0168] Exemplarily, if the information type of the target information includes an expression type, a target three-dimensional expression in the preset three-dimensional expressions that matches the expression information in the target information is determined as the target animation information. For example, if the expression information is "happy", a three-dimensional expression representing "happy" in the preset three-dimensional expressions is determined as the target animation information.

[0169] S440: If the information type of the target information includes a text type and / or a voice type, the text information and / or the voice information in the preset object output target information is determined as the target animation information.

[0170] It can be understood that, when the information type of the target information includes a text type and / or a voice type, the target generation method is: determining the text information and / or voice information in the preset object output target information as the target animation information.

[0171] Exemplarily, if the information type of the target information includes a text type and / or a voice type, the text information and / or voice information in the preset object output target information is determined as the target animation information.

[0172] Exemplarily, the preset object can be set to a preset character, a preset animal, etc.; or, the preset object can be selected at will. For example, the preset object library includes preset animation characters, preset animals, and preset plants. During the information display process, any preset object in the preset object library can be matched at will to output the target information.

[0173] Exemplarily, the output method of the preset object may be voice broadcast, or a combination of voice broadcast and body movement.

[0174] S450, adding target animation information at the target position of the initial three-dimensional image to generate a target three-dimensional image.

[0175] Exemplarily, the target position is used to indicate the position in the initial three-dimensional image, and the target position is determined by the current position of the target vehicle. Specifically, the current position of the target vehicle can be a positioning coordinate, and the positioning coordinate of the target vehicle is converted into a three-dimensional coordinate with the reference point of the vehicle as the origin of the three-dimensional coordinate, so that the target position corresponding to the positioning coordinate of the target vehicle in the initial three-dimensional image can be determined.

[0176] Exemplarily, when the target position corresponding to the positioning coordinates of the target vehicle in the initial three-dimensional image is determined, target animation information is added at the target position of the initial three-dimensional image, that is, the picture of the target position of the initial three-dimensional image is covered with the target animation information, thereby generating a target three-dimensional image.

[0177] Exemplarily, the initial three-dimensional image will change as the ego vehicle moves; for example, if the initial three-dimensional image indicates a three-dimensional image within a range of 200 meters with the ego vehicle as the center, the initial three-dimensional image will be continuously updated as the ego vehicle moves.

[0178] S460, displaying the target three-dimensional image.

[0179] Exemplarily, the target three-dimensional image is displayed on the display screen of the vehicle. The display screen may be a HUD, a central control screen, a dashboard, or other vehicle-mounted screens, which are not specifically limited here. In addition, the user may also wear an augmented reality device (AR device). Wearing an AR device to view the target three-dimensional image can enhance the user's perception of the environment in which the vehicle is located, thereby improving the interactive rendering effect.

[0180] It is understandable that by superimposing the initial three-dimensional image and the target animation information showing the interactive information, the rendering effect of the information display can be improved, thereby improving the user experience.

[0181] The above-mentioned technical scheme, when receiving the target information sent by the target vehicle, determines the target animation information corresponding to the target information based on the information type of the target information, and adds the target animation information at the target position of the initial three-dimensional image to generate a target three-dimensional image; compared with the prior art in which information interaction is achieved through an on-board information system, the initial three-dimensional image in this application is generated by a first environment image taken by the vehicle and a second environment image taken by the target drone, and the target image is generated by superimposing the target animation information displaying the target information on the initial three-dimensional image, and the external environment of the vehicle and the interactive information are superimposed and displayed, which can improve the rendering effect of the interactive information display of the vehicle, thereby improving the immersive feeling of interaction between vehicles.

[0182] It should be understood that the above examples are intended to help those skilled in the art understand the embodiments of the present application, rather than to limit the embodiments of the present application to the specific numerical values ​​or specific scenarios illustrated. Those skilled in the art can obviously make various equivalent modifications or changes based on the above examples, and such modifications or changes also fall within the scope of the embodiments of the present application.

[0183] Combination of the above Figures 1 to 4 The vehicle display method provided by the embodiment of the present application is described in detail; Figure 5 and Figure 6 The device embodiments of the present application are described in detail. It should be understood that the device in the embodiments of the present application can execute the various methods of the aforementioned embodiments of the present application, that is, the specific working processes of the following various products can refer to the corresponding processes in the aforementioned method embodiments.

[0184] Figure 5It is a structural schematic diagram of a vehicle display device provided in an embodiment of the present application.

[0185] Among them, the vehicle display device is configured on the vehicle carrying the target drone.

[0186] For example, Figure 5 As shown, the vehicle display device 500 includes:

[0187] Generating module 510: used to generate an initial three-dimensional image based on a first environment image taken by the ego vehicle and a second environment image taken by the target UAV;

[0188] Determination module 520: for determining target animation information corresponding to the target information based on the information type of the target information if target information sent by the target vehicle is received; wherein the target animation information is used to display the target information;

[0189] Adding module 530: used to add target animation information at the target position of the initial three-dimensional image to generate a target three-dimensional image; wherein the target position is determined by the current position of the target vehicle;

[0190] Display module 540: used to display the target three-dimensional image.

[0191] Optionally, as an embodiment, the determining module 520 is specifically configured to:

[0192] The target generation method is obtained by matching the information type of the target information with the preset generation relationship; wherein the preset generation relationship is used to indicate the corresponding relationship between the preset generation method and the preset information type;

[0193] The target information is processed based on the target generation method to obtain target animation information.

[0194] Optionally, as an embodiment, the information type of the target information includes at least one of an expression type, a text type, and a voice type; the determination module 520 is specifically configured to:

[0195] If the information type of the target information includes an expression type, determining a target three-dimensional expression in the preset three-dimensional expressions that matches the expression information in the target information as the target animation information;

[0196] If the information type of the target information includes a text type and / or a voice type, the text information and / or the voice information in the preset object output target information is determined as the target animation information.

[0197] Optionally, as an embodiment, if the information type of the target information includes a text type, the determination module 520 is specifically configured to:

[0198] Determine the character length of the text message;

[0199] If the character length is less than the preset character length, the preset animation relationship is matched with the text information to obtain the target animation information; wherein the preset animation relationship is used to indicate the corresponding relationship between the preset text information and the preset animation information.

[0200] Optionally, as an embodiment, the generating module 510 is specifically used for:

[0201] Performing feature matching on the first environment image and the second environment image to obtain feature point pairs;

[0202] Based on the feature point pairs, the first environment image and the second environment image are stitched together to obtain a stitched environment image;

[0203] An initial three-dimensional image is determined based on information of each feature point in the stitched environment image.

[0204] Optionally, as an embodiment, the generating module 510 is specifically used for:

[0205] Determine the coordinate information and depth information of each feature point in the stitched environment image;

[0206] Based on the coordinate information and depth information of each feature point, determine the three-dimensional coordinates of each feature point in the same three-dimensional coordinate system;

[0207] Based on the 3D coordinates of each feature point, an initial 3D image is generated.

[0208] Optionally, as an embodiment, the vehicle display device 500 further includes a sending module, and the sending module is specifically used for:

[0209] In response to the three-dimensional image opening instruction, an image acquisition instruction is sent to the target UAV;

[0210] The generation module 510 is specifically used for:

[0211] If a second environment image sent by the target UAV is received, an initial three-dimensional image is generated based on the first environment image and the second environment image;

[0212] Display the initial 3D image.

[0213] It should be noted that the above-mentioned vehicle display device 500 is embodied in the form of a functional unit. The term "module" here can be implemented in the form of software and / or hardware, and is not specifically limited to this.

[0214] For example, a "module" may be a software program, a hardware circuit, or a combination of the two that implements the above functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a dedicated processor, or a group processor, etc.) and a memory for executing one or more software or firmware programs, a combined logic circuit, and / or other suitable components that support the described functions.

[0215] Therefore, the units of each example described in the embodiments of the present application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0216] Figure 6 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application.

[0217] For example, the vehicle 600 and Figure 1 The own vehicle 100 in FIG. 1 represents the same vehicle.

[0218] For example, Figure 6 As shown, the vehicle 600 includes: a memory 610 and a processor 620, wherein the memory 610 stores an executable program code 630, and the processor 620 is used to call and execute the executable program code 630 to perform a vehicle display method.

[0219] Exemplarily, the memory 610 can be used to store relevant programs of the vehicle display method provided in the embodiments of the present application; the processor 620 can call the relevant programs of the vehicle display method stored in the memory 610 to execute the vehicle display method of the embodiments of the present application; for example, based on the first environment image taken by the self-vehicle and the second environment image taken by the target drone, an initial three-dimensional image is generated; if target information sent by the target vehicle is received, based on the information type of the target information, target animation information corresponding to the target information is determined; wherein the target animation information is used to display the target information; the target animation information is added at the target position of the initial three-dimensional image to generate a target three-dimensional image; wherein the target position is determined by the current position of the target vehicle; and the target three-dimensional image is displayed.

[0220] In this embodiment, the functional modules of the device can be divided according to the above method example. For example, each functional module can be corresponded, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0221] In the case of dividing each functional module according to each function, the device may also include a generating module, a determining module, an adding module and a displaying module, etc. It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.

[0222] It should be understood that the device provided in this embodiment is used to execute the above-mentioned vehicle display method, and thus can achieve the same effect as the above-mentioned implementation method.

[0223] In the case of an integrated unit, the device may include a processing module and a storage module. When the device is applied to a vehicle, the processing module may be used to control and manage the actions of the vehicle. The storage module may be used to support the vehicle in executing related program codes, etc.

[0224] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits shown in conjunction with the disclosure of the present application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0225] In addition, the device provided in the embodiments of the present application may specifically be a chip, component or module, and the chip may include a connected processor and memory; wherein the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute a vehicle display method provided in the above embodiments.

[0226] The present application also provides a computer-readable storage medium, in which a computer program code is stored. When the computer program code is executed on a computer, the computer executes the above-mentioned related method steps to implement a vehicle display method provided in the above-mentioned embodiment. Among them, computer-readable storage media may include but are not limited to any type of disk, including floppy disks, optical disks, digital versatile disks (DVD), compact disc read-only memory (CD-ROM), microdrives and magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read only memory (EEPROM), dynamic random access memory (DRAM), video random access memory (VRAM), flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0227] The present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement a vehicle display method provided in the above-mentioned embodiment.

[0228] Among them, the vehicle, computer-readable storage medium, computer program product or chip provided in this application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0229] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0230] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0231] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A vehicle display method, characterized in that: The method is applied to a self-vehicle carrying a target UAV; the method comprises: Generate an initial three-dimensional image based on the first environment image taken by the ego vehicle and the second environment image taken by the target drone; If target information sent by a target vehicle is received, target animation information corresponding to the target information is determined based on the information type of the target information; wherein the target animation information is used to display the target information; Adding the target animation information at the target position of the initial three-dimensional image to generate a target three-dimensional image; wherein the target position is determined by the current position of the target vehicle; The target three-dimensional image is displayed.

2. The method according to claim 1, characterized in that The determining, based on the information type of the target information, target animation information corresponding to the target information includes: Based on the information type of the target information, the target generation method is obtained by matching the preset generation relationship; wherein the preset generation relationship is used to indicate the corresponding relationship between the preset generation method and the preset information type; The target information is processed based on the target generation method to obtain the target animation information.

3. The method according to claim 2, characterized in that The information type of the target information includes at least one of an expression type, a text type, and a voice type; and the processing of the target information based on the target generation method to obtain the target animation information includes: If the information type of the target information includes the expression type, determining a target three-dimensional expression in the preset three-dimensional expressions that matches the expression information in the target information as the target animation information; If the information type of the target information includes the text type and / or the voice type, the text information and / or voice information in the target information output by the preset object is determined as the target animation information.

4. The method according to claim 3, characterized in that: If the information type of the target information includes the text type, the method further includes: Determining the character length of the text information; If the character length is less than the preset character length, the preset animation relationship is matched with the text information to obtain the target animation information; wherein the preset animation relationship is used to indicate the corresponding relationship between the preset text information and the preset animation information.

5. The method according to claim 1, characterized in that The generating of an initial three-dimensional image based on the first environment image taken by the ego vehicle and the second environment image taken by the target UAV includes: Performing feature matching on the first environment image and the second environment image to obtain feature point pairs; Based on the feature point pairs, the first environment image and the second environment image are stitched together to obtain a stitched environment image; The initial three-dimensional image is determined based on information of each feature point in the stitched environment image.

6. The method according to claim 5, characterized in that The determining the initial three-dimensional image based on the information of each feature point in the stitched environment image includes: Determine the coordinate information and depth information of each feature point in the stitched environment image; Based on the coordinate information and depth information of each feature point, determine the three-dimensional coordinates of each feature point in the same three-dimensional coordinate system; The initial three-dimensional image is generated based on the three-dimensional coordinates of each feature point.

7. The method according to claim 1, characterized in that The method further comprises: In response to the three-dimensional image start instruction, sending an image acquisition instruction to the target UAV; The generating of an initial three-dimensional image based on the first environment image taken by the ego vehicle and the second environment image taken by the target UAV includes: If the second environment image sent by the target UAV is received, the initial three-dimensional image is generated based on the first environment image and the second environment image; The initial three-dimensional image is displayed.

8. A vehicle display device, characterized in that: The device is configured on a vehicle carrying a target UAV; the device comprises: A generating module, configured to generate an initial three-dimensional image based on a first environment image taken by the ego vehicle and a second environment image taken by the target UAV; A determination module, configured to determine target animation information corresponding to the target information based on the information type of the target information if target information sent by the target vehicle is received; wherein the target animation information is used to display the target information; An adding module, used for adding the target animation information at the target position of the initial three-dimensional image to generate a target three-dimensional image; wherein the target position is determined by the current position of the target vehicle; A display module is used to display the target three-dimensional image.

9. A vehicle, characterized in that: The vehicle comprises: A memory for storing executable program codes; A processor is used to call and run the executable program code from the memory so that the vehicle executes the vehicle display method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the vehicle display method according to any one of claims 1 to 7 is implemented.

Citation Information

Cited By

  • Image display method and device, vehicle and computer readable storage medium

    CN119348427A

  • Vehicle model display method and device, vehicle and storage medium

    CN120156309A

  • Vehicle interaction method and device, vehicle-mounted equipment, storage medium and program product

    CN120560550A