Video generation method, vehicle control method, device, equipment, vehicle and medium

Abstract

The invention relates to a video generation method and device, a vehicle control method and device, electronic equipment, a vehicle and a storage medium, and a virtual reality video of the vehicle is generated according to target environment information and vehicle state information of the vehicle. According to the invention, the VR video linked with the vehicle running state can be generated, and the vehicle use experience of the user is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a video generation method, a vehicle control method, a device, an electronic device, a vehicle, a storage medium and a product. Background Art

[0002] VR (Virtual Reality) technology is a comprehensive integrated technology that can realize the interaction between people and the virtual world through smart devices, bringing immersive experience to the audience. The application of VR technology in the automotive field is becoming more and more extensive. However, the existing in-vehicle VR images can only be used to play pre-stored 2D or 3D images, which cannot meet user needs and make the user experience poor. Summary of the invention

[0003] The embodiments of the present application provide a video generation method, a vehicle control method, an apparatus, an electronic device, a vehicle, a storage medium and a program product, which are intended to generate a virtual reality video of the operating status of a linked vehicle, so as to at least partially solve the above-mentioned technical problems.

[0004] In order to achieve the above object, according to a first aspect of the present application, a vehicle control method is provided, the method comprising:

[0005] Generate a virtual reality video of the vehicle based on the vehicle's target environment information and vehicle status information.

[0006] Optionally, the vehicle status information includes: at least one of suspension status information, steering wheel status information, body status information, steering status information, seat status information and vehicle air conditioning status information of the vehicle.

[0007] Optionally, generating a virtual reality video of the vehicle according to the target environment information and the vehicle state information of the vehicle includes:

[0008] Generate a virtual reality video of the vehicle based on the target environment information and vehicle status information at the same time.

[0009] Optionally, a virtual reality video of the vehicle is generated according to the target environment information and the vehicle state information at the same time, including:

[0010] Determining vehicle state information according to the first time mark of the target environment information of the vehicle; or,

[0011] The target environment information of the vehicle is determined according to the second time identifier of the vehicle state information of the vehicle.

[0012] Optionally, the target environment information includes a road condition image, and generating a virtual reality video of the vehicle according to the target environment information and vehicle state information of the vehicle includes:

[0013] Performing three-dimensional transformation on the terrain information in the road condition image to obtain a three-dimensional terrain map;

[0014] A virtual reality video is generated according to the three-dimensional terrain map and the vehicle status information.

[0015] Optionally, the three-dimensional transformation of the terrain information in the road condition image to obtain a three-dimensional terrain map includes:

[0016] Rendering is performed based on the road surface stereo matching image and the road surface depth map in the road condition image to obtain a three-dimensional terrain map.

[0017] Optionally, the environmental information includes three-dimensional environmental information, and generating a virtual reality video of the vehicle according to the target environmental information and vehicle state information of the vehicle includes:

[0018] A virtual reality video of the vehicle is generated according to the three-dimensional environment information and the vehicle state information.

[0019] Optionally, the method further comprises:

[0020] The target environmental information is determined from at least two pre-stored environmental information, wherein the at least two environmental information are environmental information in a driving video of a target road section.

[0021] Optionally, the method further comprises:

[0022] In response to the operation of starting the video production function, the target environment information is collected through the road condition perception module in the vehicle.

[0023] Optionally, the method further comprises:

[0024] The virtual reality video is imported into a preset video library, wherein the preset video library includes a local video library and / or a cloud video library, and the preset video library includes at least one virtual reality video.

[0025] Optionally, the method further comprises:

[0026] In response to the virtual reality video playing operation, the virtual reality video is played, and the vehicle is controlled according to the vehicle state information corresponding to the virtual reality video.

[0027] According to a second aspect of the present application, a vehicle control method is provided, comprising:

[0028] In response to the virtual reality video playing operation, the virtual reality video is played, and the vehicle is controlled according to the vehicle state information corresponding to the virtual reality video.

[0029] Optionally, before playing the virtual reality video and controlling the vehicle according to the vehicle state information corresponding to the virtual reality video, the method includes:

[0030] Generate a virtual reality video of the vehicle based on the target environment information and vehicle status information of the vehicle;

[0031] Or, obtain the VR video to be played from the preset video library.

[0032] Optionally, generating a virtual reality video of the vehicle according to the target environment information and the vehicle state information of the vehicle includes:

[0033] Generate a virtual reality video of the vehicle based on the target environment information and vehicle status information at the same time.

[0034] Optionally, a virtual reality video of the vehicle is generated according to the target environment information and the vehicle state information at the same time, including:

[0035] Determining vehicle status information according to the first identifier of the target environment information of the vehicle; or,

[0036] The target environment information of the vehicle is determined according to the second identifier of the vehicle state information of the vehicle.

[0037] Optionally, the target environment information includes a road condition image, and generating a virtual reality video of the vehicle according to the target environment information and vehicle state information of the vehicle includes:

[0038] Performing three-dimensional transformation on the terrain information in the road condition image to obtain a three-dimensional terrain map;

[0039] A virtual reality video is generated according to the three-dimensional terrain map and the vehicle status information.

[0040] Optionally, the three-dimensional transformation of the terrain information in the road condition image to obtain a three-dimensional terrain map includes:

[0041] Rendering is performed based on the road surface stereo matching image and the road surface depth map in the road condition image to obtain a three-dimensional terrain map.

[0042] Optionally, the environmental information includes three-dimensional environmental information, and generating a virtual reality video of the vehicle according to the target environmental information and vehicle state information of the vehicle includes:

[0043] A virtual reality video of the vehicle is generated according to the three-dimensional environment information and the vehicle state information.

[0044] Optionally, the method comprises:

[0045] The generated virtual reality video of the vehicle is stored in a preset video library.

[0046] Optionally, controlling the vehicle according to the vehicle state information corresponding to the virtual reality video includes:

[0047] The vehicle is controlled according to the video viewing angle and the vehicle status information; wherein the video viewing angle represents the vehicle control authority possessed by the target user.

[0048] Optionally, controlling the vehicle according to the video viewing angle and the vehicle state information includes:

[0049] If the video viewing perspective is the driver's perspective, the vehicle is controlled in response to a control operation triggered by a decoupled pedal and / or steering wheel of the vehicle and according to the control operation and the vehicle status information.

[0050] Optionally, controlling the vehicle according to the video viewing angle and the vehicle state information includes:

[0051] If the video viewing perspective is a passenger perspective, the vehicle is controlled according to the vehicle status information.

[0052] According to a third aspect of the present application, a video generating device is provided, including:

[0053] The video generation module is used to generate a virtual reality video of the vehicle based on the target environment information and vehicle status information of the vehicle.

[0054] According to a fourth aspect of the present application, a vehicle control device is provided, comprising:

[0055] The vehicle control module is used to play the virtual reality video in response to the virtual reality video playing operation, and to control the vehicle according to the vehicle state information corresponding to the virtual reality video.

[0056] According to the fifth aspect of the present application, this embodiment also provides an electronic device, which includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the above method.

[0057] According to the sixth aspect of the present application, this embodiment also provides a vehicle, which includes the above-mentioned electronic device.

[0058] According to the seventh aspect of the present application, this embodiment also provides a computer-readable storage medium, which includes a computer program. When the computer program is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of the above method.

[0059] According to the eighth aspect of the present application, this embodiment also provides a computer program product, including a computer program, which is stored in a computer-readable storage medium; when the processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device performs the steps of the above method.

[0060] To summarize, the embodiment of the present application uses the above-mentioned technical solution to combine the target environment information and vehicle status information of the vehicle when the vehicle is in driving state to generate a virtual reality video of the vehicle. Compared with generating a virtual reality video based only on environmental information, the virtual reality video generated by the present application can include both environmental information and vehicle status information, so that the content of the generated virtual reality video is richer and the purpose of the virtual reality video is expanded, so that users can not only experience virtual reality road conditions, but also experience the feedback of road conditions on the driving status of the vehicle, providing users with a more realistic car experience.

[0061] Other features and advantages of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can obtain other drawings based on these drawings without creative work.

[0063] In order to more completely understand the present application and its beneficial effects, the following description will be given in conjunction with the accompanying drawings, wherein the same figure numbers represent the same parts in the following description.

[0064] Figure 1 is a first schematic diagram of a video production process provided in an exemplary embodiment of the present application;

[0065] Figure 2 is a schematic diagram of a vehicle status information collection process provided in an exemplary embodiment of the present application;

[0066] Figure 3 is a second schematic diagram of a video production process provided in an exemplary embodiment of the present application;

[0067] Figure 4 is a schematic diagram of a user experience process provided in an exemplary embodiment of the present application;

[0068] Figure 5 is a schematic diagram of a first video playback process provided in an exemplary embodiment of the present application;

[0069] Figure 6 is a schematic diagram of a second video playback process provided in an exemplary embodiment of the present application;

[0070] Figure 7 is a schematic diagram of a video production device provided in an exemplary embodiment of the present application;

[0071] Figure 8 is a schematic diagram of a vehicle control device provided in an exemplary embodiment of the present application;

[0072] Fig. 9 It is a schematic diagram of the architecture of an electronic device provided in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0073] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0074] Based on the above background technology, VR technology is increasingly being used in the automotive field. For example, a virtual reality-based cloud driving training system allows trainees to use real car control devices and virtual reality helmets to conduct vehicle driving simulation learning. For another example, VR technology can be used to simulate vehicle performance tests under different road conditions, including wind tunnel tests and collision tests, making the analysis of simulation results more intuitive and convenient. For another example, VR technology can be used to create a virtual driving environment and simulate real driving operations.

[0075] However, existing in-vehicle VR images are generally generated directly based on driving conditions and cannot be linked with the vehicle, resulting in a poor user experience. For example, users cannot feel the bumps of the vehicle when driving on the road.

[0076] In order to solve the above problems, the present application proposes a video generation method, a vehicle control method, an apparatus, an electronic device, a vehicle, a computer-readable storage medium and a computer program product, which aim to generate a virtual reality video (hereinafter referred to as VR video) in combination with vehicle status information, so that the virtual reality video can be aligned with the vehicle status information in time, so that the vehicle can be controlled based on the corresponding vehicle status information during the playback of the VR video. For example, if the vehicle is driving on a bumpy road in the VR video, the vehicle can control the body to tilt and adjust the height of the rear suspension of the vehicle, so that the user can feel a more realistic road condition.

[0077] In one embodiment, the video generation method in the present application is as follows: Figure 1 As shown, it may include:

[0078] S10, generating a virtual reality video of the vehicle according to the target environment information and the vehicle state information of the vehicle.

[0079] It should be noted that, in the present embodiment, the user can turn on the vehicle's VR video production function while driving the vehicle. For example, the user turns on the vehicle's VR video production function by setting a VR video production function button on the vehicle's display screen, or turns on the vehicle's VR video production function through a terminal device connected to the vehicle (such as a mobile phone).

[0080] On this basis, the vehicle can generate a VR video of the vehicle driving according to the target environment information and vehicle status information.

[0081] It is worth noting that the target environmental information in this embodiment can be the environmental information of the environment in which the vehicle is located during driving, or the environmental information of the environment in which the vehicle is located when the vehicle is parked. That is, the vehicle in this embodiment can generate VR video in real time during driving, and can also generate VR video when parked. There is no specific limitation on this. Environmental information that is not specially explained in subsequent embodiments can be understood as target environmental information.

[0082] For example, the vehicle can collect vehicle driving environment information and vehicle status information in real time during driving, and generate a VR video of the vehicle driving in real time; or, the vehicle can collect vehicle driving environment information and vehicle status information during driving, and then generate a VR video of the vehicle driving based on the collected environment information and vehicle status information when parking.

[0083] It can be understood that, in this embodiment, since the VR video generated in this embodiment not only includes environmental information but also vehicle status information, the user can not only watch the 3D driving environment when playing the VR video, but also feel the changes in vehicle status (such as vehicle body tilt, vehicle turning, etc.). Therefore, this embodiment defines the generated VR video as a 5D video. For example, this embodiment can add the impact of road conditions on a moving vehicle on the basis of the traditional 3D stereoscopic effect, allowing users to experience real road surface information, and combined with the acceleration and deceleration of the vehicle speed, steering, relative swing of the seat, relative wind speed of the air conditioner and other information, to provide users with an immersive simulated driving experience or a simulated riding experience.

[0084] In a specific embodiment, the vehicle can generate a VR video based on environmental information and vehicle status information corresponding to the time stamp of each environmental information, wherein the environmental information can specifically be a 2D road condition image of the vehicle's driving condition, and can also include 3D environmental information, such as 3D point cloud data collected by devices such as lidar and cameras.

[0085] It is worth noting that in the present embodiment, the time stamp of the environmental information may be the timestamp of the environmental information collection, or the time stamp may also be understood as the timestamp of the environmental information in the video in which it is located, and the timestamp is used to characterize the location information of the environmental information in the video. For example, the multi-frame road condition image in the generated VR video may include a first road condition image and a second road condition image, the timestamp of the first road condition image is T1, and the timestamp of the second road condition image is T2.

[0086] The vehicle status information corresponding to the timestamp can be understood as follows: if the vehicle turns right in the first road condition image, the vehicle status information corresponding to the timestamp may include wheel steering information, so that the user can feel the real wheel steering of the vehicle when watching the VR video. The generated VR video is actually a 5D video, that is, it not only includes driving road condition information, but also includes vehicle status information, so that when playing the VR video, it provides users with visual experience, dynamic experience, environmental special effects, interactivity, etc.

[0087] It can be understood that in this embodiment, the vehicle collects environmental information and the vehicle's current vehicle status information in real time during its driving process. If there are multiple environmental information, then each environmental information has corresponding vehicle status information. In this way, when playing the VR video, the corresponding vehicle status information can be called according to the video process to control the vehicle.

[0088] In a specific embodiment, a vehicle is provided with a Yunni system, which is a system composed of three levels: a perception layer, a decision layer, and an execution layer. The perception layer of the Yunni system may include a preview camera for recording the road conditions ahead and the surroundings; the decision layer may include a Yunni intelligent computing center for rendering the video taken by the preview camera to generate VR video. Therefore, the video generation method in the embodiment of the present application can be specifically applied to the Yunni intelligent computing center in the Yunni system of the vehicle.

[0089] Therefore, in an embodiment of the present application, a virtual reality video of the vehicle is generated by combining the environmental information and vehicle status information of the vehicle when the vehicle is in driving state. Compared with generating a virtual reality video based only on environmental information, the virtual reality video generated by the present application can include environmental information and vehicle status information, so that the user can not only experience the virtual reality road conditions, but also experience the status of the vehicle when driving, thereby effectively optimizing the user's car experience.

[0090] In one embodiment, the video generation method in the present application may further include:

[0091] The vehicle status information includes: at least one of suspension status information, steering wheel status information, body status information, steering status information, seat status information and vehicle air conditioning status information of the vehicle.

[0092] In this embodiment, the vehicle status information of the vehicle may include at least one of the vehicle's suspension status information, steering wheel status information, body status information, steering status information, seat status information, and vehicle air conditioning status information.

[0093] For example, suspension position height, suspension current information, steering wheel angle, tire angle, body roll angle, seat roll angle, body pitch angle, air-conditioning wind speed, and air-conditioning current signal, etc.

[0094] In this embodiment, after the vehicle collects environmental information during driving, the environmental information can be rendered. At the same time, the vehicle can open the corresponding timestamp and location stamp for the environmental information, and record the vehicle status information in real time at the timestamp, and import the vehicle status information into the corresponding timestamp, so that when the VR video is played to the timestamp T, according to the vehicle status information recorded in the timestamp T, the intelligent computing center can issue instructions to drive the vehicle to act according to the recorded vehicle status information, such as adjusting the suspension position height.

[0095] The vehicle status information in this embodiment is as follows: Figure 2 As shown, including but not limited to: suspension position height H1 / H2..., suspension current signal A1 / A2..., steering wheel angle value ∠1 / ∠2..., tire angle value ∠1 / ∠2..., body roll angle ∠a1 / ∠a2..., seat roll angle ∠a1 / ∠a2..., body pitch angle ∠A1 / ∠A2..., seat roll angle ∠A1 / ∠A2..., air conditioning corresponding wind speed m1 / m2..., air conditioning current signal A1 / A2...

[0096] Among them, the mark "1" represents the vehicle status information collected at the time stamp T1, and the mark "2" represents the vehicle status information collected at the time stamp T2.

[0097] In this way, this embodiment can control the vehicle according to the above-mentioned vehicle status information synchronously during the playback of VR video, providing users with a more realistic user experience, so that users can feel the feedback effect of road conditions on the vehicle's driving status.

[0098] In one embodiment, in the above S10, “generating a virtual reality video of the vehicle according to the target environment information and the vehicle state information of the vehicle” may include:

[0099] S101, generating a virtual reality video of the vehicle according to target environment information and vehicle status information at the same time.

[0100] Combined with the above-mentioned embodiment, after the vehicle collects environmental information during driving, it can render the environmental information. At the same time, the vehicle can open the corresponding timestamp for the rendered environmental information, and then synchronously collect the current vehicle status information. Then, a virtual reality video can be generated based on the target environmental information and vehicle status information at the same time.

[0101] Among them, the VR video is generated by the environmental information and vehicle status information at the same time, so that the environmental information and the corresponding vehicle status information in the generated VR video are aligned. That is, this embodiment can ensure that when the generated VR video is played later, the vehicle operation is controlled according to the vehicle status information recorded in the VR video at the same time as the environmental information, so that the user can not only watch the VR driving scene, but also can synchronously feel the impact of the environmental information (such as road conditions) on the driving vehicle, providing the user with an immersive car simulation experience.

[0102] For example, Figure 3 As shown, the vehicle can import the vehicle status information into the timestamp of the environmental information to establish an association between the timestamp and the vehicle status information. The VR video generated in this way can not only include road condition information, but also include the vehicle status information corresponding to the timestamp, so that when the VR video is played, when the timestamp is played, according to the vehicle status information recorded in the timestamp, the intelligent computing center can issue instructions to drive the vehicle to act according to the recorded vehicle status information.

[0103] In this way, the vehicle status information in the VR video generated in this embodiment is actually synchronized with the video playback process, so that when the VR video is played, the vehicle can be controlled according to the vehicle status information recorded in the VR video at the same time as the environmental information, providing users with an immersive car simulation experience.

[0104] In one embodiment, in the above S101, “generating a virtual reality video of the vehicle according to the target environment information and the vehicle state information at the same time” may include:

[0105] S1011, determining vehicle state information according to the first time identifier of the target environment information of the vehicle; or,

[0106] S1012: Determine target environment information of the vehicle according to the second time identifier of the vehicle state information of the vehicle.

[0107] In this embodiment, according to the above description, when the vehicle collects environmental information, it can record the first time mark corresponding to the environmental information, and then determine the corresponding vehicle state information according to the first time mark. For example, in this embodiment, after the vehicle obtains the environmental information, it can determine the timestamp of the environmental information (i.e., the first time mark in this embodiment).

[0108] The first time identifier of the environmental information may be a timestamp synchronously recorded when the vehicle collects the environmental information.

[0109] In addition, the vehicle can also calculate the timestamp (i.e., the first time identifier) ​​of the environmental information based on the format parameters of the generated VR video (such as the time base of the VR video, the video duration, etc.). For example, if there are multiple environmental information, the vehicle can label the environmental information according to the order in which the environmental information is collected (such as environmental information 20), and then when the time base is 1 / 50 (indicating that the duration of each environmental information is 0.02 seconds), the timestamp can be 0.4 (i.e., 0.02*20), indicating that the environmental information is located at 0.4 seconds of the media stream.

[0110] Furthermore, when the vehicle collects environmental information and determines the timestamp of the environmental information, it can simultaneously collect the vehicle's current actual vehicle status information. The vehicle status information can characterize the impact of the road conditions corresponding to the environmental information on driving and the impact on the users in the vehicle. For example, when there is an obstacle on the road, the vehicle turns to avoid the obstacle. At this time, the vehicle status information may include vehicle steering information, steering wheel angle information, etc. For another example, when the vehicle passes over a speed bump, the vehicle status information may include the vehicle body tilt angle, vehicle suspension position height, etc.

[0111] In this way, the timestamp of each environmental information contains the corresponding vehicle status information, so that when the generated VR video is played later, the vehicle operation is controlled according to the vehicle status information recorded in the timestamp of each environmental information in the VR video, so that the user can not only watch the VR driving scene, but also feel the impact of the road conditions on the moving vehicle at the same time, providing the user with an immersive riding simulation experience.

[0112] In this embodiment, the vehicle status information collection step is as follows: Figure 2 As shown, it may specifically include:

[0113] (1) Record multiple sections in sections: Section 1 / Section 2…;

[0114] (2) Marking the time T (i.e., the timestamp in this embodiment) on the corresponding road section: time T1 / time T2...;

[0115] (3) At the corresponding time T1 / T2, record the suspension position height H1 / H2..., suspension current signal A1 / A2..., steering wheel angle value ∠1 / ∠2..., tire angle value ∠1 / ∠2..., body roll angle ∠a1 / ∠a2..., seat roll angle ∠a1 / ∠a2..., body pitch angle ∠A1 / ∠A2..., seat roll angle ∠A1 / ∠A2..., air conditioner corresponding wind speed m1 / m2..., air conditioner current signal A1 / A2...

[0116] In addition, the vehicle can also determine the driving environment information of the vehicle according to the second time mark of the collected vehicle status information.

[0117] For example, when collecting vehicle status information, the vehicle can record the second time stamp corresponding to each vehicle status information, and simultaneously collect the vehicle's driving environment information, so that the vehicle status information and the corresponding environmental information can be associated through the second time stamp. In this way, the vehicle can generate a VR video based on the vehicle status information and the corresponding environmental information at the same time.

[0118] In general, in this embodiment, vehicle status information can be determined based on environmental information, or environmental information can be determined based on vehicle status information, so that the environmental information and vehicle status information in the generated VR video are aligned in time, so that users can not only watch the VR driving scene, but also can simultaneously feel the impact of environmental information (such as road conditions) on the driving vehicle, providing users with an immersive car simulation experience. For example, if the environmental information is that there is a speed bump on the road, then the corresponding vehicle status information at this time can be the vehicle suspension height information, so that users can feel the bumps of the vehicle when the vehicle passes over the speed bump when watching the VR video.

[0119] It can be seen that the present application can record the vehicle status and information at the corresponding time, completely getting rid of the time-consuming, cumbersome and error-prone problem of loading vehicle interface information according to time in the middle and late stages of vehicle VR film production. It also solves the dizziness caused by the lack of synchronization between vision and body sensation due to the inability to accurately segment the time points of video nodes and vehicle linkage, and reduces labor costs, so that users do not need to book vehicle VR films in advance, providing convenience for passengers to personalize and independently produce vehicle VR films after purchasing a car.

[0120] In one embodiment, in the above S10, “generating a virtual reality video according to the environmental information and the vehicle state information” may include:

[0121] S102, performing three-dimensional transformation on the terrain information in the road condition image to obtain a three-dimensional terrain map;

[0122] S103: Generate a virtual reality video according to the three-dimensional terrain map and the vehicle status information.

[0123] In this embodiment, since the environmental information in this embodiment can be a 2D road condition image of the vehicle's driving road condition, it can also include 3D environmental information. For the 2D road condition image, the vehicle can perform three-dimensional transformation on the terrain information in the road condition image to obtain a three-dimensional terrain map, and then generate a virtual reality video based on the three-dimensional terrain map and the vehicle status information. Among them, the terrain information in this embodiment can include road surface depth information and road surface parallax information (parallax refers to the horizontal position difference of the same object in two different perspective images), and can also include mathematical elements (such as scale, coordinate grid, etc.) and terrain elements (such as landforms) and other road condition information.

[0124] Combined with the above-mentioned embodiment, after the vehicle obtains the road condition image, it can determine the timestamp of the road condition image. It can be understood that even if the terrain information in the road condition image is converted into three dimensions, the timestamp of the three-dimensional terrain map is consistent with the timestamp of the road condition image. Therefore, the specific method of generating a virtual reality video based on the three-dimensional terrain map and the vehicle status information is as follows: according to the format parameters of the generated VR video, the timestamp of the three-dimensional terrain map is calculated (or the timestamp of the road condition image corresponding to the three-dimensional terrain map is used as the timestamp of the three-dimensional terrain map). Furthermore, after the vehicle determines the timestamp of the three-dimensional terrain map, the vehicle status information collected synchronously can be imported into the timestamp, so that the three-dimensional terrain map has the corresponding vehicle status information, and the VR video is obtained.

[0125] In a specific embodiment, in the above S102, “converting the terrain information in the environmental information into three dimensions to obtain a three-dimensional terrain map” may include:

[0126] Rendering is performed based on the road surface stereo matching image and the road surface depth map in the road condition image to obtain a three-dimensional terrain map.

[0127] In this embodiment, the intelligent computing center of the vehicle can obtain the road surface stereo matching information and road surface depth map information through the preview camera, and perform 3D terrain rendering based on the road surface stereo matching information and road surface depth map information to obtain a three-dimensional terrain map, so that the corresponding vehicle status information can be filled in the timestamp of the three-dimensional terrain map later, and a 5D VR video is generated, and the VR video is named and stored. The specific rendering method of the terrain in this embodiment can refer to the existing rendering method, and no specific limitation is made to this.

[0128] In one embodiment, in the above S10, “generating a virtual reality video according to the environmental information and the vehicle state information” may include:

[0129] S104: Generate a virtual reality video of the vehicle according to the three-dimensional environment information and the vehicle state information.

[0130] In this embodiment, if the environmental information is 3D environmental information, such as 3D point cloud data collected by devices such as lidar and cameras, then a virtual reality video of the vehicle can be generated directly based on the 3D environmental information and vehicle status information. It will not be repeated here and you can refer to the description of the above embodiment.

[0131] In this way, in this embodiment, the vehicle can collect both 2D and 3D environmental information, and can generate VR videos based on the environmental information, which is suitable for different driving conditions. For example, if the image quality of the currently collected 2D road condition image is poor and cannot be used for rendering, then the 3D point cloud data can be directly used to construct a VR video, thereby ensuring the quality of the generated VR video.

[0132] In one embodiment, the video generation method in the present application may include:

[0133] S20, determining target environment information from at least two pre-stored environment information.

[0134] In this embodiment, the vehicle can be divided into multiple target driving videos according to different road sections, for example, the driving video corresponding to road section 1 and the driving video corresponding to road section 2, wherein each driving video can contain multiple environmental information. In this way, the target environmental information can be obtained from the multiple environmental information, so that the vehicle status information can be filled in the timestamp of the environmental information to generate a 5D VR video.

[0135] It is worth noting that in this embodiment, the vehicle's intelligent computing center can also set a corresponding location stamp for each piece of environmental information, and can import the road section information corresponding to the environmental information into the location stamp, so that when the VR is played subsequently, the VR video corresponding to the road section information can be played continuously according to the road section information in the location stamp, and the vehicle can be controlled according to the vehicle status information in the timestamp of the VR video. In addition, the user can choose the VR video of the corresponding road section to experience it, which also improves the user's car simulation experience and meets the user's car needs.

[0136] In one embodiment, the video generation method of the present application may further include:

[0137] S40, in response to the start operation of the video production function, the target environment information is collected through the road condition perception module in the vehicle.

[0138] In this embodiment, combined with the above description, the user can turn on the vehicle's VR video production function while driving the vehicle. For example, the user turns on the vehicle's VR video production function by setting the VR video video production function button on the vehicle's display screen, or turns on the vehicle's VR video production function through a terminal device connected to the vehicle (such as a mobile phone).

[0139] The vehicle can respond to the operation of starting the video production function to start the video production function, and then turn on the vehicle's preview function, such as turning on the vehicle's camera, collecting road stereo matching images and road depth maps for terrain rendering, generating a 3D terrain map, and synchronously collecting vehicle status information, and recording the vehicle status information in the corresponding timestamp of the 3D terrain map to generate a 5D scene image (i.e., the VR image in this embodiment).

[0140] In one embodiment, the video generation method in the present application may further include:

[0141] S50: Import the virtual reality video into a preset video library, wherein the preset video library includes a local video library and / or a cloud video library, and the preset video library includes at least one virtual reality video.

[0142] In this embodiment, after the vehicle generates a VR video, it can import the VR video into a preset video library, wherein the preset video library is a local video library and / or a cloud video library, and the preset video library can contain at least one virtual reality video, so that subsequent users can select a target virtual reality video from the preset video library for playback.

[0143] It can be seen that in this embodiment, users can actively create VR videos, and can also upload their own VR videos to the preset video library in the cloud (such as the 5D image library for real-life viewing), so that players who share the cloud 5D image library can also experience the beautiful scenery of the sharer, which can not only enrich the VR videos in the preset video library, but also meet the users' diverse test drive simulation needs.

[0144] In a specific embodiment, Figure 3 As shown, the video generation method in the present application may specifically include:

[0145] (1) Turn on the vehicle and enable the preview function;

[0146] (2) Start the 5D scene image production function;

[0147] (3) The vehicle's intelligent computing center obtains road surface stereo matching information, obtains road surface depth map information, and performs 3D terrain rendering;

[0148] In 3D terrain rendering, the vehicle status and information are recorded in real time at the corresponding time T, and a 5D scene image is generated and named (318 Beautiful Scenery). While rendering the image, the intelligent computing center will open the timestamp and location stamp, and the vehicle's driving data can be implanted one by one according to these timestamps and location stamps during the vehicle's movement.

[0149] (4) After rendering is completed, it can be saved locally and in the cloud 5D image library.

[0150] In one embodiment, the video generation method in the present application may include:

[0151] S60: In response to the virtual reality video playing operation, the virtual reality video is played, and the vehicle is controlled according to the vehicle state information corresponding to the virtual reality video.

[0152] In this embodiment, when the vehicle generates a VR video, the VR video can be imported into a preset video library, so that the preset video library can contain at least one virtual reality video.

[0153] If the user wants to simulate driving or riding in a car, he can select the virtual reality video he wants to experience from the preset video library to trigger the virtual reality video playback operation.

[0154] The vehicle can respond to the virtual reality video playback operation, play the virtual reality video selected by the user, and control the vehicle according to the vehicle status information corresponding to the virtual reality video.

[0155] For example, when the vehicle is playing a VR video, when it reaches timestamp T, it can issue instructions through the intelligent computing center based on the vehicle status information recorded in timestamp T to drive the vehicle to act according to the recorded vehicle status information, such as controlling the relative movement of the Yunniang chassis, allowing passengers to experience real road information, and then combining with information such as vehicle acceleration and deceleration, steering, relative swing of seats, relative wind speed of air conditioner, etc., to provide users with an immersive car experience.

[0156] Therefore, in this embodiment, the depth map, point cloud information, GRB image and other data obtained by the preview camera are integrated by utilizing the integration and intelligent rendering capabilities of the Yunni Intelligent Computing Center, and a 3D image is obtained after rendering, which is closer to the real scene. The vehicle status information is automatically recorded at the corresponding timestamp, and the suspension height information and suspension current information corresponding to the timestamp, the corresponding vehicle speed and acceleration and deceleration, the steering wheel and wheel angle information, the relative swing of the seat, the relative wind speed of the air conditioner and other vehicle states are recorded in the video to generate a 5D video. This eliminates the problem of loading the vehicle interface information according to time in the middle and late stages of the production of in-vehicle VR films, which results in a long time, cumbersome tasks and prone to errors. It also solves the problem of dizziness caused by the lack of synchronization between vision and body sensation due to the inability to accurately segment the video nodes and the time points of the vehicle linkage, and reduces labor costs, so that users do not need to book in-vehicle VR films in advance, providing convenience for passengers to personalize and independently produce in-vehicle VR films after purchasing a car.

[0157] Accordingly, the present application also proposes a vehicle control method, which can be applied to a vehicle, and the method may include the following steps:

[0158] Step 1: In response to a virtual reality video playback operation, the virtual reality video is played, and the vehicle is controlled according to vehicle state information corresponding to the virtual reality video.

[0159] Combined with the above-mentioned embodiment, after the vehicle obtains the environmental information, it can determine the timestamp of the environmental information. Then, when the vehicle collects the environmental information and determines the timestamp of the environmental information, it can synchronously collect the current actual vehicle status information of the vehicle, so that the timestamp of each environmental information contains the corresponding vehicle status information, and obtains the VR video, and then the VR video can be imported into the preset video library. If the user wants to simulate driving or riding, he can select the virtual reality video he wants to experience from the preset video library, trigger the virtual reality video playback operation, and the vehicle can respond to the virtual reality video playback operation, play the virtual reality video selected by the user, and control the vehicle according to the vehicle status information corresponding to the virtual reality video, so that the user can not only watch the VR driving scene, but also can synchronously feel the impact of road conditions on the driving vehicle, providing the user with an immersive car simulation experience.

[0160] In one embodiment, before playing the virtual reality video and controlling the vehicle according to the vehicle state information corresponding to the virtual reality video, the method includes:

[0161] Step 2, generating a virtual reality video of the vehicle based on the target environment information and vehicle status information of the vehicle; or,

[0162] Step 3: Obtain the virtual reality video to be played from the preset video library.

[0163] In this embodiment, the vehicle can generate a VR video of the vehicle driving according to the driving environment information and the vehicle state information. In addition, the vehicle can also directly obtain the virtual reality video to be played from the preset video library.

[0164] It is understandable that the application scenario of the vehicle control method in this embodiment can be that the vehicle is in a parking state, that is, since the vehicle is controlled according to the vehicle state information while playing the VR video in this embodiment, if this embodiment is executed while the vehicle is in a driving state, it will affect the normal control of the vehicle. Therefore, in this embodiment, the vehicle can respond to the video viewing operation when parked and obtain the virtual reality video to be played from the preset video library. Specific reference is made to the various embodiments of the above-mentioned video generation method, which will not be repeated here.

[0165] In this way, the embodiment of the present application combines the vehicle's driving environment information and vehicle status information when the vehicle is in driving state to generate a virtual reality video of the vehicle, so that the virtual reality video can include environmental information and vehicle status information. The user can not only experience the virtual reality road conditions, but also experience the vehicle's driving status, thereby effectively optimizing the user's car simulation experience.

[0166] In one embodiment, the above “generating a virtual reality video of the vehicle according to the target environment information and the vehicle state information of the vehicle” may include:

[0167] Generate a virtual reality video of the vehicle based on the target environment information and vehicle status information at the same time.

[0168] In this embodiment, the vehicle can generate a virtual reality video based on the target environment information and vehicle status information at the same time.

[0169] Among them, the VR video is generated by the environmental information and vehicle status information at the same time, so that the environmental information and the corresponding vehicle status information in the generated VR video are aligned. That is, this embodiment can ensure that when the generated VR video is played later, the vehicle operation is controlled according to the vehicle status information recorded in the VR video at the same time as the environmental information, so that the user can not only watch the VR driving scene, but also can synchronously feel the impact of the environmental information (such as road conditions) on the driving vehicle, providing the user with an immersive car simulation experience.

[0170] In one embodiment, the above “generating a virtual reality video of the vehicle according to the target environment information and the vehicle state information at the same time” may include:

[0171] Determining vehicle status information according to the first identifier of the target environment information of the vehicle; or,

[0172] The target environment information of the vehicle is determined according to the second identifier of the vehicle state information of the vehicle.

[0173] In this embodiment, according to the above description, when collecting environmental information, the vehicle may record the first time identifier corresponding to the environmental information, and then determine the corresponding vehicle status information according to the first time identifier.

[0174] Alternatively, the vehicle may also determine the driving environment information of the vehicle according to the second time mark of the collected vehicle status information.

[0175] That is, in this embodiment, the vehicle status information can be determined based on the environmental information, or the environmental information can be determined based on the vehicle status information, so that the environmental information and the vehicle status information in the generated VR video are aligned in time, so that the user can not only watch the VR driving scene, but also can synchronously feel the impact of the environmental information (such as road conditions) on the driving vehicle, providing the user with an immersive car simulation experience. Please refer to the various embodiments of the above-mentioned video generation method, which will not be repeated here.

[0176] In one embodiment, the target environment information includes a road condition image, and the above “generating a virtual reality video of the vehicle according to the target environment information and the vehicle state information of the vehicle” may include:

[0177] Performing three-dimensional transformation on the terrain information in the road condition image to obtain a three-dimensional terrain map;

[0178] A virtual reality video is generated according to the three-dimensional terrain map and the vehicle status information.

[0179] In this embodiment, since the environmental information in this embodiment can be a 2D road condition image of the vehicle's driving road conditions, and can also include 3D environmental information, for the 2D road condition image, the vehicle can perform three-dimensional transformation on the terrain information in the road condition image to obtain a three-dimensional terrain map, and then generate a virtual reality video based on the three-dimensional terrain map and the vehicle status information.

[0180] For example, the intelligent computing center of the vehicle can obtain the road surface stereo matching information and road surface depth map information in the road condition image through the preview camera, and perform 3D terrain rendering based on the road surface stereo matching information and road surface depth map information to obtain a three-dimensional terrain map, so that the corresponding vehicle status information can be filled in the timestamp of the three-dimensional terrain map later, and a 5D VR video is generated, and the VR video is named and stored. The specific rendering method of the terrain in this embodiment can refer to the existing rendering method, and no specific limitation is made to this.

[0181] In one embodiment, the above “generating a virtual reality video of the vehicle according to the target environment information and the vehicle state information of the vehicle” may include:

[0182] A virtual reality video of the vehicle is generated according to the three-dimensional environment information and the vehicle state information.

[0183] In this embodiment, if the environmental information is 3D environmental information, such as 3D point cloud data collected by devices such as lidar and cameras, then a virtual reality video of the vehicle can be generated directly based on the 3D environmental information and vehicle status information. It will not be repeated here and you can refer to the description of the above embodiment.

[0184] In this way, in this embodiment, the vehicle can collect both 2D and 3D environmental information, and can generate VR videos based on the environmental information, which is suitable for different driving conditions. For example, if the image quality of the currently collected 2D road condition image is poor and cannot be used for rendering, then the 3D point cloud data can be directly used to construct a VR video, thereby ensuring the quality of the generated VR video.

[0185] In one embodiment, in the above step 1, “controlling the vehicle according to the vehicle state information corresponding to the virtual reality video” may include:

[0186] During the playing of the virtual reality video, vehicle status information in each time mark of the virtual reality video is obtained, and the vehicle is controlled according to the vehicle status information.

[0187] In combination with the above-mentioned embodiment, in this embodiment, since the timestamp of each environmental information contains the corresponding vehicle status information when making the VR video, when the generated VR video is played later, the vehicle operation can be controlled according to the vehicle status information recorded in the timestamp of each environmental information in the VR video, so that the user can not only watch the VR driving scene, but also feel the impact of the road conditions on the moving vehicle while watching the VR video, thereby providing the user with an immersive riding simulation experience.

[0188] In one embodiment, in the above step 1, “controlling the vehicle according to the vehicle state information corresponding to the virtual reality video” may include:

[0189] Step 101, controlling the vehicle according to the video viewing angle and the vehicle status information; wherein the video viewing angle represents the vehicle control authority possessed by the target user.

[0190] It should be noted that, in this embodiment, Figure 4As shown, the user can start the electronic parking to put the vehicle in a parked state, and then connect the in-car VR glasses or 3D in-car TV, etc., and select car-machine linkage, enter the cloud 5D video library to select the VR video to be played (if the user does not select car-machine linkage, the vehicle returns to the parking state and can play 2D road condition videos for the user). After the user accesses the cloud 5D video library to confirm the VR video to be played, he can determine whether to use the vehicle's decoupled pedals and steering wheel.

[0191] If the vehicle detects that the user is using the pedals and steering wheel, it can be determined that the current video viewing perspective is the driver's perspective, and the vehicle can enter driver mode; if the vehicle does not detect that the user is using the pedals and steering wheel, it can be determined that the current video viewing perspective is the passenger's perspective, and the vehicle can enter passenger mode.

[0192] It can be understood that, in this embodiment, virtual pedals and steering wheels may also be provided for user operation.

[0193] The vehicle can then be controlled based on the video viewing angle and vehicle status information.

[0194] In a specific embodiment, in the above step 101, "controlling the vehicle according to the video viewing angle and the vehicle state information" may include:

[0195] Step a: If the video viewing angle is the driver's angle of view, respond to a control operation triggered by the decoupled pedals and / or steering wheel of the vehicle, and control the vehicle according to the control operation and the vehicle status information.

[0196] Step b: If the video viewing perspective is a passenger perspective, then controlling the vehicle according to the vehicle status information is executed.

[0197] In this embodiment, if Figure 5 As shown, if the current role is the driver, the vehicle can start the pedal steering wheel, and during the VR video playback, according to the timestamp in the VR video, perform autonomous suspension adjustment (including calling the suspension current value at the corresponding moment, and adjusting the suspension according to the current value), seat adjustment (including calling the steering wheel angle and body roll at the corresponding moment to adjust the seat swing), air conditioning adjustment (including calling the air conditioning current signal at the corresponding moment to adjust the air conditioning wind speed), steering wheel self-control, etc., and the experience is completed after the video playback ends, and exit the current function.

[0198] In this embodiment, if Figure 6As shown, if the current role is a passenger, the vehicle can call the vehicle status information recorded in real time at the corresponding time during the VR video playback, and perform corresponding operations such as suspension adjustment, vehicle speed adjustment, acceleration and deceleration adjustment, seat adjustment, and air conditioning adjustment. After the video playback is over, the experience is completed and the current function is exited. That is, at this time, only the 5D viewing function is provided to the user, and the user does not have the vehicle operation authority.

[0199] In another embodiment, the vehicle may also classify the generated VR videos when generating them. For example, the VR videos may be divided into: VR videos played for drivers and VR videos played for passengers. In this way, when playing VR videos, corresponding VR videos may be played for different users to meet the vehicle experience requirements of different users.

[0200] Therefore, in this embodiment, the viewing can be divided into the driver's perspective and the passenger's perspective. If the external virtual pedals and steering wheel are selected, the pedals and steering wheel can be operated from the driver's perspective; if the external virtual pedals and steering wheel are abandoned, it will be from the passenger's perspective, and when playing the 5D video, the relative movement of the Yunyan chassis is controlled according to the loaded position stamps and the vehicle status information in the position stamps, allowing passengers to experience the real road information, and then combined with the acceleration and deceleration of the speed, steering, relative swing of the seats, relative wind speed of the air conditioner and other information, to provide users with a more realistic user simulation experience.

[0201] Accordingly, the embodiment of the present application also provides a video generating device, such as Figure 7 As shown, the device may include:

[0202] The video generation module 1001 is used to generate a virtual reality video of the vehicle based on the target environment information and vehicle status information of the vehicle.

[0203] Optionally, the vehicle status information includes: at least one of suspension status information, steering wheel status information, body status information, steering status information, seat status information and vehicle air conditioning status information of the vehicle.

[0204] Optionally, the video generation module 1001 is further used for:

[0205] Generate a virtual reality video of the vehicle based on the target environment information and vehicle status information at the same time.

[0206] Optionally, the video generation module 1001 is further used for:

[0207] Determining vehicle state information according to the first time mark of the target environment information of the vehicle; or,

[0208] The target environment information of the vehicle is determined according to the second time identifier of the vehicle state information of the vehicle.

[0209] Optionally, the target environment information includes a road condition image, and the video generation module 1001 is further used to:

[0210] Performing three-dimensional transformation on the terrain information in the road condition image to obtain a three-dimensional terrain map;

[0211] A virtual reality video is generated according to the three-dimensional terrain map and the vehicle status information.

[0212] Optionally, the video generation module 1001 is further used for:

[0213] Rendering is performed based on the road surface stereo matching image and the road surface depth map in the road condition image to obtain a three-dimensional terrain map.

[0214] Optionally, the environmental information includes three-dimensional environmental information, and the video generation module 1001 is further used for:

[0215] A virtual reality video of the vehicle is generated according to the three-dimensional environment information and the vehicle state information.

[0216] Optionally, the video generation method in the present application may further include:

[0217] The road segment division module is used to divide the initial driving video into driving videos of different road segments according to its road segment information; and obtain at least two environmental information of the vehicle driving from the driving video.

[0218] Optionally, the video generation method in the present application may further include:

[0219] The data acquisition module is used to respond to the start operation of the video production function and collect the target environment information through the road condition perception module in the vehicle.

[0220] Optionally, the video generation method in the present application may further include:

[0221] The video import module is used to import the virtual reality video into a preset video library, wherein the preset video library includes a local video library and / or a cloud video library, and the preset video library contains at least one virtual reality video.

[0222] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0223] Accordingly, the embodiment of the present application also provides a vehicle control device, such as Figure 8 As shown, the device may include:

[0224] The vehicle control module 1002 is used to respond to the virtual reality video playback operation, play the virtual reality video, and control the vehicle according to the vehicle state information corresponding to the virtual reality video.

[0225] Optionally, the vehicle control device in the present application further includes:

[0226] A video generation module, used for generating a virtual reality video of the vehicle according to target environment information and vehicle status information of the vehicle;

[0227] The video acquisition module is used to acquire the virtual reality video to be played from the preset video library.

[0228] Optionally, the video generation module is further used to:

[0229] Generate a virtual reality video of the vehicle based on the target environment information and vehicle status information at the same time.

[0230] Optionally, the video generation module is further used to:

[0231] Determining vehicle status information according to the first identifier of the target environment information of the vehicle; or,

[0232] The target environment information of the vehicle is determined according to the second identifier of the vehicle state information of the vehicle.

[0233] Optionally, the target environment information includes a road condition image, and the video generation module is further used to:

[0234] Performing three-dimensional transformation on the terrain information in the road condition image to obtain a three-dimensional terrain map;

[0235] A virtual reality video is generated according to the three-dimensional terrain map and the vehicle status information.

[0236] Optionally, the video generation module is further used to:

[0237] Rendering is performed based on the road surface stereo matching image and the road surface depth map in the road condition image to obtain a three-dimensional terrain map.

[0238] Optionally, the environmental information includes three-dimensional environmental information, and the video generation module is further used to:

[0239] A virtual reality video of the vehicle is generated according to the three-dimensional environment information and the vehicle state information.

[0240] Optionally, the vehicle control device in the present application further includes:

[0241] The video storage module is used to store the generated virtual reality video of the vehicle in a preset video library.

[0242] Optionally, the vehicle control module 1002 is further configured to:

[0243] The vehicle is controlled according to the video viewing angle and the vehicle status information; wherein the video viewing angle represents the vehicle control authority possessed by the target user.

[0244] Optionally, the vehicle control module 1002 is further configured to:

[0245] If the video viewing perspective is the driver's perspective, the vehicle is controlled in response to a control operation triggered by a decoupled pedal and / or steering wheel of the vehicle and according to the control operation and the vehicle status information.

[0246] Optionally, the vehicle control module 1002 is further configured to:

[0247] If the video viewing perspective is a passenger perspective, the vehicle is controlled according to the vehicle status information.

[0248] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0249] Accordingly, the present application also provides an electronic device, such as Fig. 9 As shown, Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 1100 includes a processor 1101 having one or more processing cores, a memory 1102 having one or more computer-readable storage media, and a computer program stored in the memory 1102 and executable on the processor. The processor 1101 is electrically connected to the memory 1102. It will be understood by those skilled in the art that the vehicle structure shown in the figure does not constitute a limitation on the vehicle, and may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0250] The processor 1101 is the control center of the electronic device 1100, and uses various interfaces and lines to connect various parts of the entire electronic device 1100. By running or loading software programs and / or units stored in the memory 1102, and calling data stored in the memory 1102, the processor 1101 executes various functions of the electronic device 1100 and processes data, thereby monitoring the electronic device 1100 as a whole. The processor 1101 can be a processor (Central Processing Unit, CPU), a graphics processing unit (graphics processing unit, GPU), a network processor (Network Processor, NP), etc., and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application.

[0251] In the embodiment of the present application, the processor 1101 in the electronic device 1100 will load instructions corresponding to the processes of one or more application programs into the memory 1102 according to the following steps, and the processor 1101 will run the application programs stored in the memory 1102 to implement various functions, such as:

[0252] Generate a virtual reality video of the vehicle based on the vehicle's target environment information and vehicle status information.

[0253] Also for example:

[0254] In response to the virtual reality video playing operation, the virtual reality video is played, and the vehicle is controlled according to the vehicle state information corresponding to the virtual reality video.

[0255] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0256] Optional, such as Fig. 9 As shown, the electronic device 1100 further includes: a touch screen 1103, a radio frequency circuit 1104, an audio circuit 1105, an input unit 1106, and a power supply 1107. The processor 1101 is electrically connected to the touch screen 1103, the radio frequency circuit 1104, the audio circuit 1105, the input unit 1106, and the power supply 1107, respectively. Those skilled in the art can understand that Fig. 9 The vehicle structure shown in the figure does not constitute a limitation on the vehicle, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0257] The touch display screen 1103 can be used to display a graphical user interface and receive operation instructions generated by the user acting on the graphical user interface. The touch display screen 1103 may include a display panel and a touch panel. Among them, the display panel can be used to display information input by the user or information provided to the user and various graphical user interfaces of the vehicle, which can be composed of graphics, text, icons, videos and any combination thereof. Optionally, the display panel can be configured in the form of a liquid crystal display (LCD, Liquid Crystal Display), an organic light-emitting diode (OLED, Organic Light-Emitting Diode), etc. The touch panel can be used to collect the user's touch operation on or near it (such as the user uses any suitable object or accessory such as a finger, stylus, etc. on the touch panel or near the touch panel), and generate corresponding operation instructions, and the operation instructions execute the corresponding program. Optionally, the touch panel may include two parts: a touch display system and a touch controller. Among them, the touch display system detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch display system, converts it into the touch point coordinates, and then sends it to the processor 1101, and can receive the command sent by the processor 1101 and execute it. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it is transmitted to the processor 1101 to determine the type of touch event, and then the processor 1101 provides a corresponding visual output on the display panel according to the type of touch event. In an embodiment of the present application, the touch panel and the display panel can be integrated into the touch display screen 1103 to realize the input and output functions. However, in some embodiments, the touch panel and the touch panel can be used as two independent components to realize the input and output functions. That is, the touch display screen 1103 can also be used as a part of the input unit 1106 to realize the input function.

[0258] The RF circuit 1104 may be used to send and receive RF signals to establish wireless communication with network devices or other vehicles through wireless communication, and to send and receive signals with network devices or other vehicles.

[0259] The audio circuit 1105 can be used to provide an audio interface between the user and the vehicle through a speaker and a microphone. The audio circuit 1105 can transmit the electrical signal converted from the received audio data to the speaker, which is converted into a sound signal for output; on the other hand, the microphone converts the collected sound signal into an electrical signal, which is received by the audio circuit 1105 and converted into audio data, and then the audio data is output to the processor 1101 for processing, and then sent to another vehicle through the radio frequency circuit 1104, or the audio data is output to the memory 1102 for further processing. The audio circuit 1105 may also include an earplug jack to provide communication between an external headset and the vehicle.

[0260] The input unit 1106 may be used to receive input numbers, character information or user feature information (such as fingerprint, iris, facial information, etc.), and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0261] The power supply 1107 is used to supply power to various components of the electronic device 1100. Optionally, the power supply 1107 can be logically connected to the processor 1101 through a power management device, so that the power management device can manage charging, discharging, power consumption and other functions. The power supply 1107 can also include one or more DC or AC power supplies, recharging devices, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0262] although Fig. 9 Not shown, the electronic device 1100 may also include a camera, a sensor, a wireless fidelity module, a Bluetooth module, etc., which will not be described in detail here.

[0263] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0264] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0265] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a plurality of computer programs are stored. The computer program can be loaded by a processor to execute any one of the video generation methods or vehicle control methods provided in the embodiments of the present application. The computer program can execute the following steps of the video generation method:

[0266] Generate a virtual reality video of the vehicle based on the vehicle's target environment information and vehicle status information.

[0267] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0268] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0269] Since the computer program stored in the computer-readable storage medium can execute any one of the video generation methods or vehicle control methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the video generation methods or vehicle control methods provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0270] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0271] The present application is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0272] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0273] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0274] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0275] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0276] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated communication signals and carrier waves.

[0277] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0278] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0279] The embodiments, implementation methods and related technical features of the present application can be combined and replaced with each other without conflict.

[0280] The above are only preferred embodiments of the present application and do not constitute any form of limitation to the present application. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application are still within the scope of the technical solution of the present application.

Claims

1. A video generation method, characterized in that: The method is applied to a vehicle, and comprises: Generate a virtual reality video of the vehicle based on the vehicle's target environment information and vehicle status information.

2. The method according to claim 1, characterized in that The vehicle status information includes: at least one of suspension status information, steering wheel status information, body status information, steering status information, seat status information and vehicle air conditioning status information of the vehicle.

3. The method according to claim 1, characterized in that The method of generating a virtual reality video of the vehicle according to the target environment information and the vehicle state information of the vehicle includes: A virtual reality video of the vehicle is generated based on the target environment information and the vehicle status information at the same moment.

4. The method according to claim 3, characterized in that Generate a virtual reality video of the vehicle based on the target environment information and vehicle status information at the same time, including: Determining vehicle state information according to the first time mark of the target environment information of the vehicle; or, The target environment information of the vehicle is determined according to the second time identifier of the vehicle state information of the vehicle.

5. The method according to any one of claims 1 to 4, characterized in that: The target environment information includes a road condition image, and generating a virtual reality video of the vehicle according to the target environment information and vehicle state information of the vehicle includes: Performing three-dimensional transformation on the terrain information in the road condition image to obtain a three-dimensional terrain map; A virtual reality video is generated according to the three-dimensional terrain map and the vehicle status information.

6. The method according to claim 5, characterized in that The three-dimensional transformation of the terrain information in the road condition image to obtain a three-dimensional terrain map includes: Rendering is performed based on the road surface stereo matching image and the road surface depth map in the road condition image to obtain a three-dimensional terrain map.

7. The method according to claim 1, characterized in that The environmental information includes three-dimensional environmental information, and generating a virtual reality video of the vehicle according to the target environmental information and the vehicle state information includes: A virtual reality video of the vehicle is generated according to the three-dimensional environment information and the vehicle state information.

8. The method according to claim 1, characterized in that The method further comprises: The target environmental information is determined from at least two pre-stored environmental information, wherein the at least two environmental information are environmental information in a driving video of a target road section.

9. The method according to claim 1, characterized in that: The method further comprises: In response to the operation of starting the video production function, the target environment information is collected through the road condition perception module in the vehicle.

10. The method according to claim 1, characterized in that The method further comprises: The virtual reality video is imported into a preset video library, wherein the preset video library includes a local video library and / or a cloud video library, and the preset video library includes at least one virtual reality video.

11. The method according to claim 1, characterized in that: The method further comprises: In response to the virtual reality video playing operation, the virtual reality video is played, and the vehicle is controlled according to the vehicle state information corresponding to the virtual reality video.

12. A vehicle control method, characterized in that: include: In response to the virtual reality video playing operation, the virtual reality video is played, and the vehicle is controlled according to the vehicle state information corresponding to the virtual reality video.

13. The vehicle control method according to claim 12, characterized in that: Before playing the virtual reality video and controlling the vehicle according to the vehicle state information corresponding to the virtual reality video, the method includes: Generate a virtual reality video of the vehicle based on the target environment information and vehicle status information of the vehicle; Or, obtain the VR video to be played from the preset video library.

14. The vehicle control method according to claim 13, characterized in that: The method of generating a virtual reality video of the vehicle according to the target environment information and the vehicle state information of the vehicle includes: Generate a virtual reality video of the vehicle based on the target environment information and vehicle status information at the same time.

15. The method according to claim 14, characterized in that Generate a virtual reality video of the vehicle based on the target environment information and vehicle status information at the same time, including: Determining vehicle status information according to the first identifier of the target environment information of the vehicle; or, The target environment information of the vehicle is determined according to the second identifier of the vehicle state information of the vehicle.

16. The method according to claim 13, characterized in that The target environment information includes a road condition image, and generating a virtual reality video of the vehicle according to the target environment information and vehicle state information of the vehicle includes: Performing three-dimensional transformation on the terrain information in the road condition image to obtain a three-dimensional terrain map; A virtual reality video is generated according to the three-dimensional terrain map and the vehicle status information.

17. The method according to claim 16, characterized in that The three-dimensional transformation of the terrain information in the road condition image to obtain a three-dimensional terrain map includes: Rendering is performed based on the road surface stereo matching image and the road surface depth map in the road condition image to obtain a three-dimensional terrain map.

18. The method according to claim 13, characterized in that The environmental information includes three-dimensional environmental information, and generating a virtual reality video of the vehicle according to the target environmental information and the vehicle state information includes: A virtual reality video of the vehicle is generated according to the three-dimensional environment information and the vehicle state information.

19. The vehicle control method according to any one of claims 13 to 18, characterized in that: The method comprises: The virtual reality video of the vehicle is stored in a preset video library.

20. The vehicle control method according to claim 12, characterized in that: The controlling the vehicle according to the vehicle state information corresponding to the virtual reality video includes: The vehicle is controlled according to the video viewing angle and the vehicle status information; wherein the video viewing angle represents the vehicle control authority possessed by the target user.

21. The vehicle control method according to claim 20, characterized in that: The controlling the vehicle according to the video viewing angle and the vehicle state information includes: If the video viewing perspective is the driver's perspective, the vehicle is controlled in response to a control operation triggered by a decoupled pedal and / or steering wheel of the vehicle and according to the control operation and the vehicle status information.

22. The vehicle control method according to claim 20, characterized in that: The controlling the vehicle according to the video viewing angle and the vehicle state information includes: If the video viewing perspective is a passenger perspective, the vehicle is controlled according to the vehicle status information.

23. A video generating device, characterized in that: The device comprises: The video generation module is used to generate a virtual reality video of the vehicle based on the target environment information and vehicle status information of the vehicle.

24. A vehicle control device, characterized in that: The device comprises: The vehicle control module is used to play the virtual reality video in response to the virtual reality video playing operation, and to control the vehicle according to the vehicle state information corresponding to the virtual reality video.

25. An electronic device, characterized in that: It includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes any method described in claims 1 to 11, or executes any method described in claims 12 to 22.

26. A vehicle, characterized in that: The vehicle is provided with the electronic device according to claim 25.

27. A computer-readable storage medium, characterized in that: It includes a computer program, which, when executed on an electronic device, is used to cause the electronic device to execute any one of the methods described in claims 1 to 11, or to execute any one of the methods described in claims 12 to 22.

28. A computer program product, characterized in that It includes a computer program, which is stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device executes any one of the methods described in claims 1 to 11, or executes any one of the methods described in claims 12 to 22.

Images