Video playback method, device and vehicle

Abstract

The present invention provides a video playback method, device and vehicle, the method comprising: in response to an AR viewing start instruction, collecting in-vehicle environment data, and determining the first audio and video currently selected; obtaining a second audio and video; wherein the second audio and video is synthesized by the vehicle or the cloud by the in-vehicle environment data and the first audio and video; sending the second audio and video to AR glasses for playback, and / or sending the second audio and video to the display end of the vehicle for playback; the present invention provides a live effect for users watching a performance by synthesizing the image and sound of the in-vehicle environment into the live video.

Description

Technical Field

[0001] The present invention relates to the field of vehicle technology, and in particular to a video playing method, device and vehicle. Background Art

[0002] Currently, users begin to pay attention to the entertainment of vehicles. Therefore, current vehicles are usually equipped with AR devices to realize activities such as watching movies, games, and meetings. However, for live singing and performances, it is difficult to provide users with on-site effects and feelings when watching through current AR devices. Summary of the invention

[0003] In view of this, the present invention aims to propose a video playback method, device and vehicle to solve the problem that the on-site effect and feeling of watching live performances using vehicle-mounted AR are not strong, which affects the user experience.

[0004] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0005] A video playback method, applied to a vehicle, comprising:

[0006] In response to an instruction to start AR viewing, collect in-vehicle environment data and determine a first audio or video currently selected;

[0007] Acquire a second audio and video; wherein the second audio and video is synthesized by the vehicle or the cloud by combining the in-vehicle environment data and the first audio and video;

[0008] The second audio and video are sent to the AR glasses for playback, and / or the second audio and video are sent to the display end of the vehicle for playback.

[0009] Furthermore, the synthesizing the in-vehicle environment data and the first audio and video includes:

[0010] Extracting data from the first audio and video according to the time sequence of audio and video acquisition to obtain multiple frames of audio and video data carrying a first timestamp;

[0011] Extracting the in-vehicle environment data according to the time sequence of data collection to obtain user data carrying a second timestamp;

[0012] Based on the first timestamp and the second timestamp, multiple frames of the audio and video data and the user data are synthesized to obtain the second audio and video; wherein the audio and video data in the second audio and video are time-aligned with the user data.

[0013] Furthermore, before synthesizing the multiple frames of the audio and video data and the user data based on the first timestamp and the second timestamp to obtain the second audio and video, the method further includes:

[0014] Labeling the user data to filter out target user data having target user characteristics;

[0015] The synthesizing, based on the first timestamp and the second timestamp, multiple frames of the audio and video data and the user data to obtain the second audio and video includes:

[0016] Based on the first timestamp and the second timestamp, multiple frames of the audio and video data and the target user data are synthesized to obtain the second audio and video.

[0017] Furthermore, the method further comprises:

[0018] When playing the second audio and video, the audio system of the vehicle is adjusted to an on state, and the ambient light of the vehicle is adjusted to a rhythmic state to simulate the on-site atmosphere.

[0019] Furthermore, in the case where the first audio and video are synthesized by the cloud, the method further includes:

[0020] In response to the AR viewing start instruction, the in-vehicle environment data is sent to the cloud through the AR glasses, and the identifier of the first audio and video data is indicated to the cloud, so that the cloud obtains the first audio and video based on the identifier.

[0021] Compared with the prior art, the video playback method of the present invention has the following advantages:

[0022] The present invention collects in-vehicle environmental data and determines the currently selected first audio and video in response to an AR viewing start instruction; obtains the second audio and video; wherein the second audio and video is obtained by the vehicle or the cloud by synthesizing the in-vehicle environmental data and the first audio and video; sends the second audio and video to the AR glasses for playback, and / or sends the second audio and video to the display end of the vehicle for playback; thus, the present invention synthesizes the in-vehicle environmental data and the video data of the on-site meeting, superimposes the sound and image in the car on the video of the on-site meeting, thereby creating an effect of watching the scene for the user, and because the user's voice and actions can also be superimposed on the video of the on-site meeting, while creating the effect of watching the scene, it can also combine the user's interaction in the car with the on-site video, to achieve a personalized interactive experience for the user.

[0023] Another object of the present invention is to propose a video playback method to solve the problem that the on-site effect and feeling of watching live performances using vehicle-mounted AR are not strong, which affects the user experience.

[0024] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0025] A video playback method, applied in the cloud, comprising:

[0026] Receiving in-vehicle environment data and an identifier of the first audio and video; wherein the in-vehicle environment data is sent by the vehicle in response to an AR viewing start instruction;

[0027] Based on the identifier, obtaining the first audio and video;

[0028] Synthesizing the in-vehicle environment data and the first audio and video to obtain a second audio and video;

[0029] The second audio and video is sent to the vehicle and / or to the AR glasses for playback.

[0030] Further, the receiving of the in-vehicle environment data and the identification of the first audio and video includes:

[0031] Receiving the in-vehicle environment data and the identification of the first audio and video sent by the vehicle through the AR glasses;

[0032] The acquiring the first audio and video based on the identifier includes:

[0033] According to the identifier of the first audio and video, the first audio and video is obtained from the database in the cloud.

[0034] The advantages of the video playback method described are the same as those of the above-mentioned video playback method over the prior art, which will not be described in detail here.

[0035] Another object of the present invention is to provide a device to solve the problem that the on-site effect and feeling of watching live performances using vehicle-mounted AR are not strong, which affects the user experience.

[0036] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0037] A video playback device, applied to a vehicle, comprising:

[0038] A collection module, used to respond to an AR viewing start instruction, collect in-vehicle environment data, and determine the first audio and video currently selected;

[0039] An acquisition module, used to acquire a second audio and video; wherein the second audio and video is synthesized by the vehicle or the cloud by combining the in-vehicle environment data and the first audio and video;

[0040] A sending module is used to send the second audio and video to the AR glasses for playback, and / or to send the second audio and video to the display end of the vehicle for playback.

[0041] The advantages of the device and the above method over the prior art are the same, which will not be described in detail here.

[0042] Another object of the present invention is to provide a device to solve the problem that the on-site effect and feeling of watching live performances using vehicle-mounted AR are not strong, which affects the user experience.

[0043] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0044] A video playback device, applied to the cloud, comprising:

[0045] A receiving module, configured to receive in-vehicle environment data and an identifier of a first audio or video; wherein the in-vehicle environment data is sent by the vehicle in response to an instruction to start the AR viewing;

[0046] An acquisition module, used for acquiring the first audio and video based on the identifier;

[0047] A synthesis module, used for synthesizing the in-vehicle environment data and the first audio and video to obtain a second audio and video;

[0048] A sending module is used to send the second audio and video to the vehicle and / or to the AR glasses for playback.

[0049] The advantages of the device and the above method over the prior art are the same, which will not be described in detail here.

[0050] Another object of the present invention is to propose a vehicle to solve the problem that the on-site effect and feeling of watching live performances using vehicle-mounted AR are not strong, which affects the user experience.

[0051] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0052] A vehicle comprises: a video playing unit, wherein the video playing unit is used to execute the above-mentioned video playing method.

[0053] The advantages of the vehicle and the above-mentioned video playback method over the prior art are the same, which will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0055] Figure 1 A schematic diagram of an implementation environment of a video playback method provided by Embodiment 1 of the present invention is shown;

[0056] Figure 2 A flowchart of a video playback method provided by Embodiment 1 of the present invention is shown;

[0057] Figure 3 A flowchart showing a video playback method according to another embodiment of the present invention is shown;

[0058] Figure 4 A flowchart of a video playback method provided by Embodiment 2 of the present invention is shown;

[0059] Figure 5 A schematic diagram showing the structure of a video playback device provided by Embodiment 3 of the present invention is shown;

[0060] Figure 6 A schematic structural diagram of a video playback device provided in a fourth embodiment of the present invention is shown. DETAILED DESCRIPTION

[0061] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0062] A video playing method, device and vehicle of the present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0063] Embodiment 1

[0064] This application embodiment provides a video playback method, please refer to Figure 1 , which shows a schematic diagram of the implementation environment of the video playback method provided in the embodiment of the present application. The implementation environment may include: a vehicle, AR glasses, and the cloud.

[0065] The vehicle controller is connected to the AR glasses for data transmission. For example, when the user selects the corresponding video to play, the video data can be sent to the AR glasses through the vehicle controller, and the AR glasses can play it. The vehicle controller and the AR glasses can be connected by wire or wirelessly. In the case of a wired connection between the vehicle controller and the AR glasses, the data cable connected to the AR glasses is connected to the interface of the vehicle controller to enable the vehicle controller and the AR glasses to transmit data; in the case of a wireless connection between the vehicle controller and the AR glasses, the vehicle body's own Bluetooth and WIFI are turned on. After the AR glasses are turned on, they are actively paired with the vehicle controller through Bluetooth / WIFI; in some embodiments, when the AR glasses and the vehicle body Bluetooth / WIFI are paired for the first time or the AR glasses and the vehicle body Bluetooth are not matched, the user can actively pair them.

[0066] The AR glasses are connected to the cloud, where the cloud is used to connect to the external network to obtain video data. The cloud also has a data processing function to process the video and then send it to the AR glasses for playback.

[0067] Reference Figure 2 , Figure 2 A flowchart of a video playback method provided by an embodiment of the present invention is shown. Figure 2 As shown, the video playback method of the embodiment of the present invention is applied to a vehicle, and the method includes:

[0068] S101, in response to an AR viewing start instruction, collecting in-vehicle environment data and determining the first audio and video currently selected.

[0069] In an embodiment of the present invention, a button or control may be configured on the display end of the vehicle to indicate the user's intention to start AR viewing. For example, in the map navigation interface of the vehicle, when the user selects a video, he or she may choose whether to start AR viewing. When the user clicks the "AR viewing" button or control, it indicates that the user indicates to watch through AR, thereby collecting environmental data inside the vehicle based on the user's instructions. For example, the image data inside the vehicle is collected through the camera, and the sound data inside the vehicle is collected through the microphone.

[0070] Specifically, when the user selects "AR viewing", the video currently selected by the user is determined, wherein the video can be selected from multiple videos on the interface, or selected after searching on the interface, wherein the selected video can be connected to the external network through the vehicle's external T-BOX (Telematics BOX, a vehicle networking intelligent terminal) to obtain the first audio and video selected by the user.

[0071] S102, obtaining a second audio and video; wherein the second audio and video is obtained by synthesizing the vehicle environment data and the first audio and video by the vehicle or the cloud.

[0072] Specifically, when data synthesis is performed by the vehicle, the synthesis is performed directly based on the first audio and video and vehicle environment data acquired by the vehicle.

[0073] When data synthesis is performed by the cloud, the vehicle controller sends the vehicle environment data to the cloud through the AR glasses, and also sends the selected first audio and video identifier to the cloud through the AR glasses, so that the cloud obtains the first audio and video according to the first audio and video identifier. As a result, the cloud performs data synthesis based on the obtained vehicle environment data and the first audio and video to obtain the second audio and video.

[0074] S103: Send the second audio and video to the AR glasses for playback, and / or send the second audio and video to the display end of the vehicle for playback.

[0075] After synthesizing the vehicle environment data and the first audio and video to obtain the second audio and video, the second audio and video can be sent to the AR glasses for playback, or sent to the vehicle display end for playback, or played simultaneously on the AR glasses and the vehicle display end according to the user's choice. The vehicle display end can be a commonly used vehicle display screen, including a rearview mirror display, a vehicle DVD, and a vehicle navigation display.

[0076] Among them, while playing the second audio and video to the AR glasses or sending the display end of the vehicle, the vehicle audio can also be turned on through the vehicle controller, and the vehicle atmosphere lights can be adjusted to a rhythmic state. In this way, high-quality audio output is provided to users through the in-car audio, and the lighting atmosphere of the scene is simulated through the vehicle atmosphere lights, further enhancing the on-site effect of watching live performances in the car.

[0077] In the embodiment of the present invention, when receiving an AR viewing instruction, the vehicle or the vehicle environment data sent by the vehicle to the cloud and the first audio and video are synthesized to obtain a second audio and video, and then the obtained second audio and video are sent to the AR glasses or the vehicle display terminal for playback. Thus, the user can watch the live performance video including the image and sound in the car through the AR glasses or the vehicle display terminal, creating a live atmosphere for the user. In addition, by controlling the state of the components in the car, some live effects are simulated for the user in the car, creating a live atmosphere, and further enhancing the user's sense of presence when watching the live performance video in the car.

[0078] Reference Figure 3 , Figure 3 A flowchart of a video playback method according to another embodiment of the present invention is shown. Figure 3 As shown, the video playback method of the embodiment of the present invention is applied to a vehicle, comprising:

[0079] S201, in response to an AR viewing start instruction, collecting in-vehicle environment data and determining the first audio and video currently selected.

[0080] The first audio and video of the embodiment of the present invention can be selected through the vehicle APP, or through the mobile phone APP connected to the vehicle. For example, the user selects the video he wants to watch through the mobile phone APP, and then sends it to the vehicle according to the identifier of the first audio and video selected by the mobile phone APP. The vehicle accesses the content end through the vehicle TBOX to obtain the corresponding audio and video, and obtains the first audio and video.

[0081] S202: extract data of the first audio and video according to the time sequence of audio and video acquisition to obtain multiple frames of audio and video data carrying a first timestamp.

[0082] Specifically, data of the first audio and video is extracted according to the video stream to obtain multiple frames of audio and video data carrying a first timestamp, wherein the data extraction of the first audio and video may be performed frame by frame or in an interval frame manner.

[0083] S203, extracting the in-vehicle environment data according to the time sequence of data collection to obtain user data carrying a second timestamp.

[0084] Among them, the data extraction of the in-vehicle environment data is to obtain the image data and sound data of the AR eye user, thereby directly synthesizing the user's image and sound into the live video to enhance the live viewing effect of the user.

[0085] In some embodiments, cameras and sound sensors are set up in different areas of the vehicle to collect image data and sound data only for the area where the user of the AR glasses is located. Therefore, when extracting data, it is only necessary to perform processing such as clarifying the image and sound in order to process the user's image and sound.

[0086] The data extraction of user data mainly includes: labeling the action data in the user data and clarifying the sound data in the user data. Specifically, the user's actions in the image data are identified, and the user data is labeled according to the action recognition results, and then the user data related to the video viewing is screened out, thereby synthesizing the user's related actions into the video, achieving the synthetic effect of the user interacting with the live performance in the car.

[0087] S204: Based on the first timestamp and the second timestamp, synthesize multiple frames of the audio and video data and the user data to obtain the second audio and video; wherein the audio and video data in the second audio and video are time-aligned with the user data.

[0088] After obtaining multiple frames of audio and video data carrying a first timestamp and user data carrying a second timestamp, the audio and video data and user data in the same time are synthesized according to the first timestamp and the second timestamp, so that the audio and video data and user data in the synthesized second audio and video are time-aligned.

[0089] S205, sending the second audio and video to the AR glasses for playback, and / or sending the second audio and video to the display end of the vehicle for playback.

[0090] Among them, while playing the second audio and video to the AR glasses or sending the display end of the vehicle, the vehicle audio can also be controlled to turn on through the vehicle controller, and the vehicle atmosphere light can be adjusted to a rhythmic state, thereby providing users with high-quality audio output through the in-car audio, and simulating the lighting atmosphere of the scene through the vehicle atmosphere light, further enhancing the on-site effect of watching live performances in the car. For example, when playing a live concert, live music can be provided through the audio, and the on-site lighting effect can be simulated through the atmosphere light, thereby simulating the on-site atmosphere so that users can experience better on-site effects.

[0091] In the embodiment of the present invention, when receiving an AR viewing instruction, the vehicle collects vehicle environment data and the first audio and video acquired, then extracts data from the in-vehicle environment data based on the time sequence, and extracts data from the first audio and video, obtains user data and audio and video data with timestamps, and synthesizes the user data and audio and video data based on the timestamp to obtain the second audio and video, and then sends the obtained second audio and video to the AR glasses or the vehicle display end for playback, so that the user can watch the live performance video including the user himself through the AR glasses or the vehicle display end, and create a live atmosphere for the user by integrating the user into the video. At the same time, by controlling the state of the in-vehicle components, some live effects are simulated for the user in the car, creating a live atmosphere, and further enhancing the user's sense of presence when watching the live performance video in the car. In addition, since the user data is also labeled, when the data is synthesized, the filtered labeled data and audio and video data are synthesized, the user's actions can be displayed in the video, realizing the effect of the user interacting with the scene in the car, and improving the user's viewing experience.

[0092] Embodiment 2

[0093] Reference Figure 4 , Figure 4 The flowchart of the video playback method provided by the embodiment of the present invention is shown as follows: Figure 4 As shown, the video playback method of the embodiment of the present invention is applied to the cloud, including:

[0094] S301, receiving in-vehicle environment data and an identifier of a first audio or video; wherein the in-vehicle environment data is sent by the vehicle in response to an AR viewing start instruction.

[0095] Specifically, when the second audio and video are synthesized through the cloud, when the user confirms to turn on AR viewing, the vehicle sends the collected in-vehicle environment data and the first audio and video identifier selected by the user to the cloud through the AR glasses according to the instructions.

[0096] S302: Acquire the first audio and video based on the identifier.

[0097] Specifically, the cloud obtains the first audio and video from the cloud connection database according to the received first audio and video identifier.

[0098] In some embodiments, the cloud and the vehicle-mounted TBOX are connected via a wireless network, so that after obtaining the first audio and video identifier, the cloud can obtain the first audio and video through the vehicle-mounted TBOX according to the first audio and video identifier.

[0099] S303: synthesize the in-vehicle environment data and the first audio and video to obtain second audio and video.

[0100] Specifically, the sound data and image data of the AR glasses user in the in-car environment data are extracted to obtain user data; among them, the user's image data can be recognized and the user's action label can be determined to synthesize the user's actions for the live performance into the video, such as the interactive actions between the user and the scene. As a result, the user can see the live meeting with a 3D effect, and can see the effect of his own interaction in the car and the synthesis of the live meeting (such as hugging, blowing kisses, shaking hands with the singer, etc.), to achieve a personalized and private experience.

[0101] In some embodiments, data extraction and audio and video data extraction are performed according to the time sequence of data collection, so that the extracted user data and audio and video data are timestamped, and the user data and audio and video data are synthesized based on the timestamp, thereby aligning the image and sound of the user in the second audio and video with the video time to create a live effect for the user.

[0102] S304, sending the second audio and video to the vehicle, and / or sending to the AR glasses for playback.

[0103] In the embodiment of the present invention, when receiving an AR viewing instruction, the first audio and video obtained by the identification sent by the vehicle and the in-vehicle environment data sent by the vehicle are synthesized to obtain the second audio and video, and then the obtained second audio and video are sent to the AR glasses or the vehicle display terminal for playback, so that the user can watch the live performance video including the in-vehicle image and sound through the AR glasses or the vehicle display terminal, creating a live atmosphere for the user. In addition, by controlling the state of the in-vehicle components, some live effects are simulated for the user in the car, creating a live atmosphere, and further enhancing the user's sense of presence when watching the live performance video in the car.

[0104] The above process is explained below with a specific example:

[0105] First, when the user selects the live video to be viewed on the vehicle operation terminal, such as the vehicle APP or the mobile phone APP, and chooses to turn on the AR viewing mode, the vehicle controller collects the vehicle environment data through the vehicle camera, sound sensor, etc., and obtains the identifier of the first audio and video selected by the user, or the vehicle controller sends the vehicle environment data and the identifier of the selected first audio and video to the cloud through the AR glasses;

[0106] Afterwards, the vehicle controller connects to the external network through the vehicle-mounted TBOX according to the video identifier of the first audio and video, accesses the content end to obtain the first audio and video; or the cloud obtains the first audio and video from the cloud database according to the identifier of the first audio and video.

[0107] After obtaining the first audio and video and in-vehicle environment data, the vehicle controller or the cloud extracts data from the first audio and video according to the time sequence of audio and video acquisition to obtain multiple frames of audio and video data carrying a first timestamp; and extracts data from the in-vehicle environment data according to the time sequence of data acquisition to obtain user data carrying a second timestamp, wherein user actions can also be identified during data extraction to obtain labeled user data, wherein the label is used to describe user actions.

[0108] After data extraction, the vehicle controller or the cloud aligns the time of the user data and the audio and video data according to the first timestamp and the second timestamp when synthesizing the user data and the audio and video data, thereby obtaining the second audio and video.

[0109] Finally, the vehicle controller or the cloud will send the second audio and video to the AR glasses and / or to the vehicle display for playback, so as to display the synthesized second audio and video to the user. As a result, the user can watch the live video with 3D effects through the AR glasses or the vehicle display, and can see the interaction between himself and the scene. While playing the second audio and video, the vehicle controller can also control the car audio to turn on, providing users with higher-quality audio playback effects, and control the vehicle's ambient light to enter a rhythmic state, so that the ambient light moves with the sound, creating visual and sound effects similar to the scene, and simulating the scene atmosphere for the user.

[0110] Embodiment 3

[0111] Reference Figure 5 , Figure 5 FIG. 4 shows a schematic diagram of the structure of a video playback device provided by an embodiment of the present invention. Figure 5 As shown, the video playback device of the embodiment of the present invention is applied to a vehicle, and the device includes:

[0112] The collection module 401 is used to collect in-vehicle environment data in response to the AR viewing start instruction, and determine the first audio and video currently selected;

[0113] An acquisition module 402 is used to acquire a second audio and video; wherein the second audio and video is synthesized by the vehicle or the cloud by combining the in-vehicle environment data and the first audio and video;

[0114] The sending module 403 is used to send the second audio and video to the AR glasses for playback, and / or send the second audio and video to the display end of the vehicle for playback.

[0115] In some embodiments, the acquisition module 402 further includes:

[0116] A data extraction module, configured to extract data from the first audio and video according to the time sequence of audio and video acquisition to obtain multiple frames of audio and video data carrying a first timestamp; and to extract data from the in-vehicle environment data according to the time sequence of data acquisition to obtain user data carrying a second timestamp;

[0117] A synthesis module is used to synthesize multiple frames of the audio and video data and the user data based on the first timestamp and the second timestamp to obtain the second audio and video; wherein the audio and video data in the second audio and video are time-aligned with the user data.

[0118] In some embodiments, the data extraction module further includes:

[0119] A screening module, used to perform labeling processing on the user data and screen out target user data having target user characteristics;

[0120] The synthesis module also includes:

[0121] A synthesis submodule is used to synthesize multiple frames of the audio and video data and the target user data based on the first timestamp and the second timestamp to obtain the second audio and video.

[0122] In some embodiments, the apparatus further comprises:

[0123] The control module is used to adjust the audio system of the vehicle to an on state and the atmosphere light of the vehicle to a rhythmic state when playing the second audio and video, so as to simulate the atmosphere of the scene.

[0124] Embodiment 4

[0125] Reference Figure 6 , Figure 6 FIG. 4 shows a schematic diagram of the structure of a video playback device provided by an embodiment of the present invention. Figure 6 As shown, the video playback device of the embodiment of the present invention is applied to the cloud, and the device includes:

[0126] The receiving module 501 is used to receive in-vehicle environment data and an identifier of a first audio or video; wherein the in-vehicle environment data is sent by the vehicle in response to the AR viewing start instruction;

[0127] An acquisition module 502, configured to acquire the first audio and video based on the identifier;

[0128] A synthesis module 503, used for synthesizing the in-vehicle environment data and the first audio and video to obtain a second audio and video;

[0129] The sending module 504 is used to send the second audio and video to the vehicle and / or to the AR glasses for playback.

[0130] In some embodiments, the acquisition module 502 further includes:

[0131] The acquisition submodule is used to acquire the first audio and video from the database in the cloud according to the identifier of the first audio and video.

[0132] Based on the same inventive concept, an embodiment of the present invention further provides a system, including: a vehicle, AR glasses, and a cloud; wherein the vehicle and the cloud are connected via the AR glasses;

[0133] The vehicle includes a vehicle controller, the vehicle controller is used to obtain in-vehicle environment data and determine the identifier of the currently selected first audio and video;

[0134] The cloud is used to obtain the first audio and video according to the first audio and video identifier and send it to the vehicle controller through the AR glasses;

[0135] In the case where the second audio and video are synthesized by the vehicle, the vehicle controller is further used to synthesize the in-vehicle environment data and the first audio and video to obtain the second audio and video;

[0136] The vehicle also includes a display terminal for receiving and playing a second audio and video sent by the vehicle.

[0137] The AR glasses are connected to the vehicle and are used to receive and play the second audio and video sent by the vehicle.

[0138] In the case where the second audio and video are synthesized by the cloud, the cloud is used to receive the in-vehicle environment data and the identifier of the first audio and video sent by the vehicle control, and obtain the first audio and video according to the first audio and video identifier;

[0139] The cloud is also used to synthesize the in-vehicle environment data and the first audio and video to obtain the second audio and video, and send the second audio and video to the AR glasses and / or send it to the vehicle through the AR glasses.

[0140] Based on the same inventive concept, an embodiment of the present invention further provides a vehicle, the vehicle comprising a video playback unit, and the video playback unit executes the video playback method as described in any of the above embodiments.

[0141] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0142] For the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the order of the actions described, because according to the present invention, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and components involved are not necessarily required by the present invention.

[0143] The video playback method, device and vehicle provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A video playback method, characterized in that: Applied to a vehicle, the method comprises: In response to an instruction to start AR viewing, collect in-vehicle environment data and determine a first audio or video currently selected; Acquire a second audio and video; wherein the second audio and video is synthesized by the vehicle or the cloud by combining the in-vehicle environment data and the first audio and video; Sending the second audio and video to the AR glasses for playback, and / or sending the second audio and video to the display end of the vehicle for playback; The synthesizing the in-vehicle environment data and the first audio and video includes: Extracting data from the first audio and video according to the time sequence of audio and video acquisition to obtain multiple frames of audio and video data carrying a first timestamp; Extracting the in-vehicle environment data according to the time sequence of data collection to obtain user data carrying a second timestamp; Performing labeling processing on the user data to filter out target user data having target user characteristics, wherein the label is used to describe the user's actions; Based on the first timestamp and the second timestamp, multiple frames of the audio and video data and the target user data are synthesized to obtain the second audio and video; wherein the audio and video data in the second audio and video are time-aligned with the user data.

2. The video playback method according to claim 1, characterized in that: The method further comprises: When playing the second audio and video, the audio system of the vehicle is adjusted to an on state, and the ambient light of the vehicle is adjusted to a rhythmic state to simulate the on-site atmosphere.

3. The video playback method according to claim 1, characterized in that: In the case where the first audio and video are synthesized by the cloud, the method further includes: In response to the AR viewing start instruction, the in-vehicle environment data is sent to the cloud through the AR glasses, and the identifier of the first audio and video data is indicated to the cloud, so that the cloud obtains the first audio and video based on the identifier.

4. A video playback method, characterized in that: Applied to the cloud, the method includes: Receiving in-vehicle environment data and an identifier of a first audio or video; wherein the in-vehicle environment data is sent by the vehicle in response to an AR viewing start instruction; Based on the identifier, obtaining the first audio and video; Synthesizing the in-vehicle environment data and the first audio and video to obtain a second audio and video; Sending the second audio and video to the vehicle, and / or sending it to the AR glasses for playback; The synthesizing the in-vehicle environment data and the first audio and video includes: Extracting data from the first audio and video according to the time sequence of audio and video acquisition to obtain multiple frames of audio and video data carrying a first timestamp; Extracting the in-vehicle environment data according to the time sequence of data collection to obtain user data carrying a second timestamp; Performing labeling processing on the user data to filter out target user data having target user characteristics, wherein the label is used to describe the user's actions; Based on the first timestamp and the second timestamp, multiple frames of the audio and video data and the target user data are synthesized to obtain the second audio and video; wherein the audio and video data in the second audio and video are time-aligned with the user data.

5. The video playback method according to claim 4, characterized in that: The receiving of the in-vehicle environment data and the identification of the first audio and video includes: Receiving the in-vehicle environment data and the identification of the first audio and video sent by the vehicle through the AR glasses; The acquiring the first audio and video based on the identifier includes: According to the identifier of the first audio and video, the first audio and video is obtained from the database in the cloud.

6. A video playback device, characterized in that: Applied to a vehicle, the device comprises: A collection module, used to respond to an AR viewing start instruction, collect in-vehicle environment data, and determine the first audio and video currently selected; An acquisition module, used to acquire a second audio and video; wherein the second audio and video is synthesized by the vehicle or the cloud by combining the in-vehicle environment data and the first audio and video; A sending module, used to send the second audio and video to the AR glasses for playback, and / or send the second audio and video to the display end of the vehicle for playback; The synthesizing the in-vehicle environment data and the first audio and video includes: Extracting data from the first audio and video according to the time sequence of audio and video acquisition to obtain multiple frames of audio and video data carrying a first timestamp; Extracting the in-vehicle environment data according to the time sequence of data collection to obtain user data carrying a second timestamp; Performing labeling processing on the user data to filter out target user data having target user characteristics, wherein the label is used to describe the user's actions; Based on the first timestamp and the second timestamp, multiple frames of the audio and video data and the target user data are synthesized to obtain the second audio and video; wherein the audio and video data in the second audio and video are time-aligned with the user data.

7. A video playback device, characterized in that: Applied to the cloud, the device comprises: A receiving module, configured to receive in-vehicle environment data and an identifier of the first audio and video; wherein the in-vehicle environment data is sent by the vehicle in response to an AR viewing start instruction; An acquisition module, used for acquiring the first audio and video based on the identifier; A synthesis module is used to extract data from the first audio and video according to the time sequence of audio and video acquisition to obtain multiple frames of audio and video data carrying a first timestamp; extract data from the in-vehicle environment data according to the time sequence of data acquisition to obtain user data carrying a second timestamp; label the user data to filter out target user data with target user characteristics, wherein the label is used to describe the user's actions; based on the first timestamp and the second timestamp, synthesize multiple frames of the audio and video data and the target user data to obtain a second audio and video; wherein the audio and video data in the second audio and video are time-aligned with the user data; A sending module is used to send the second audio and video to the vehicle and / or to the AR glasses for playback.

8. A vehicle, characterized in that: The vehicle comprises: a video playing unit, and the video playing unit is used to execute the video playing method as described in any one of claims 1-3 or claims 4-5 above.

Images