Vehicle control method and device, terminal equipment and computer readable storage medium

Abstract

The application is suitable for the technical field of vehicle control, and provides a vehicle control method and device, terminal equipment and computer readable storage medium, comprising: when a starting instruction of a game is received and a vehicle is in a parking state, a communication connection between a control equipment of the vehicle and a controlled equipment is cut off; the control equipment of the vehicle is associated with a virtual control device of the game; a scene interface of the game is acquired; and the scene interface is displayed on a windshield and / or window glass of the vehicle through a head-up display system of the vehicle. Through the above method, the intelligent degree of the automobile can be improved, and the user experience is improved.

Description

Technical Field

[0001] This application belongs to the field of vehicle control technology, and in particular relates to a vehicle control method, device, terminal equipment and computer-readable storage medium. Background Technology

[0002] As automobiles become increasingly intelligent, they will no longer be merely a means of transportation; entertainment and multimedia will also become crucial factors in attracting users. A key indicator of a car's intelligence is the human-vehicle interaction within the cockpit. Currently, human-vehicle interaction is limited to the interaction between the user and the central control screen in the cockpit, failing to fully utilize other automotive devices. This results in a low level of intelligence and a poor user experience. Summary of the Invention

[0003] This application provides a vehicle control method, device, terminal equipment, and computer-readable storage medium, which can improve the intelligence level of automobiles and thus enhance the user experience.

[0004] In a first aspect, embodiments of this application provide a vehicle control method, including:

[0005] When a game start command is received and the vehicle is in a parked state, the communication connection between the vehicle's control device and the controlled device is cut off.

[0006] Associate the vehicle's control equipment with the game's virtual control device;

[0007] Obtain the scene interface of the game;

[0008] The scene interface is displayed on the windshield and / or window glass of the vehicle through the vehicle's head-up display system.

[0009] In this embodiment, when the vehicle is parked, the virtual control device for the game can be connected to the vehicle's control equipment to control the virtual scene in the game. Additionally, the game interface can be displayed on the windshield and / or windows via the vehicle's head-up display system, allowing the user to experience the game immersively. Through these methods, the vehicle's control equipment is fully utilized, enhancing human-vehicle interaction in the cockpit, improving the vehicle's intelligence, and ultimately enhancing the user experience.

[0010] In one possible implementation of the first aspect, obtaining the game's scene interface includes:

[0011] Acquire recorded images from the vehicle's dashcam;

[0012] The recorded image is used to generate the scene interface.

[0013] In one possible implementation of the first aspect, removing the vehicle image from the recorded image to obtain the background image includes:

[0014] The recorded image is segmented to obtain a first image that does not contain the vehicle image;

[0015] The first image is subjected to image restoration processing to obtain the background image.

[0016] In one possible implementation of the first aspect, the image inpainting process performed on the first image to obtain the background image includes:

[0017] The first image is coarsely restored using the trained coarse restoration model to obtain the coarsely restored image.

[0018] The background image is obtained by performing fine restoration processing on the coarsely restored image using the trained fine restoration model.

[0019] In one possible implementation of the first aspect, the scene interface includes a first area screen and a second area screen, wherein the first area screen is the screen of the middle area of ​​the scene interface, and the second area screen is the screen of the scene interface excluding the first area screen.

[0020] Displaying the scene interface on the windshield and / or window glass of the vehicle through the vehicle's head-up display system includes:

[0021] The head-up display system displays the first area of ​​the scene interface onto the windshield of the vehicle.

[0022] The head-up display system displays the second area of ​​the scene interface onto the vehicle's window.

[0023] In one possible implementation of the first aspect, displaying the first area of ​​the scene interface onto the windshield of the vehicle via the head-up display system includes:

[0024] When the game is in single-player mode, the first area of ​​the screen is displayed in full screen on the windshield through the head-up display system;

[0025] When the game is in two-player mode, the head-up display system displays half of the first area of ​​the screen from the first perspective on the left side of the windshield, and half of the first area of ​​the screen from the second perspective on the right side of the windshield.

[0026] In one possible implementation of the first aspect, after associating the vehicle's control device with the game's virtual control device, the method further includes:

[0027] If an exit command is received from the game, the connection between the vehicle's control device and the game's virtual control device is severed;

[0028] Restore the communication connection between the vehicle's control equipment and the controlled equipment.

[0029] Secondly, embodiments of this application provide a vehicle control device, including:

[0030] The disconnection unit is used to disconnect the communication connection between the vehicle's control device and the controlled device when a game start command is received and the vehicle is in a parked state.

[0031] The association unit is used to associate the vehicle's control device with the game's virtual control device;

[0032] The acquisition unit is used to acquire the scene interface of the game;

[0033] The projection unit is used to display the scene interface on the windshield and / or window glass of the vehicle through the vehicle's head-up display system.

[0034] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the vehicle control method as described in any one of the first aspects above.

[0035] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the vehicle control method as described in any one of the first aspects above.

[0036] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes the terminal device to execute the vehicle control method described in any one of the first aspects.

[0037] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a schematic flowchart of the vehicle control method provided in the embodiments of this application;

[0040] Figure 2 This is a schematic diagram of the image restoration process provided in the embodiments of this application;

[0041] Figure 3 This is a schematic diagram of the structure of the vehicle control device provided in the embodiments of this application;

[0042] Figure 4 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. Detailed Implementation

[0043] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0044] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0045] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0046] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0047] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0048] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0049] See Figure 1 This is a schematic flowchart of a vehicle control method provided in an embodiment of this application. It is intended as an example and not a limitation. The method may include the following steps:

[0050] S101, when a game start command is received and the vehicle is in a parked state, the communication connection between the vehicle's control device and the controlled device is cut off.

[0051] In one application scenario, a user inputs a game start command via the vehicle's central control screen. Upon receiving the start command via the central control screen, the vehicle's control center checks if the vehicle is currently parked; if so, it disconnects the communication connection between the vehicle's control equipment and the controlled device.

[0052] In this embodiment of the application, one way to determine whether the vehicle is in a parked state is to detect whether the current gear of the vehicle is the parking gear (such as P gear); if so, the vehicle is determined to be in a parked state.

[0053] Furthermore, to enhance safety, the vehicle's current speed and handbrake status can also be detected. For example, it can determine whether the vehicle's current speed is 0 and whether the handbrake is active; if the speed is 0, the handbrake is active, and the current gear is park, then the vehicle is determined to be in a parked state.

[0054] In this embodiment, the vehicle's control devices may include a steering wheel, accelerator pedal, clutch pedal, brake pedal, gear shift lever, etc. The vehicle's controlled devices may include wheels, clutch, brake pads, oil pump, and transmission, etc. The vehicle's control devices and controlled devices have a corresponding relationship. For example, the steering wheel controls the steering of the wheels, the accelerator pedal controls the oil pump, the clutch pedal controls the clutch, the brake pedal controls the brake pads, and the gear shift lever controls the transmission.

[0055] Disconnecting the communication link between the vehicle's control equipment and the controlled equipment prevents the control equipment from controlling the controlled equipment. In one application scenario, the vehicle's control equipment sends control signals to the controlled equipment through the vehicle's control center to achieve control. In this scenario, the vehicle's control center can stop forwarding control signals sent from the control equipment to the controlled equipment, thus disconnecting the communication link. In another application scenario, the vehicle's control equipment sends control signals directly to the controlled equipment. In this scenario, the vehicle's control center can send an intercept signal to block the control signals sent from the control equipment to the controlled equipment, thereby disconnecting the communication link.

[0056] S102, associate the vehicle's control device with the game's virtual control device.

[0057] Taking racing games as an example, the vehicle's steering wheel is associated with the virtual steering wheel in the game, the vehicle's accelerator pedal is associated with the virtual accelerator pedal in the game, the vehicle's clutch pedal is associated with the virtual clutch pedal in the game, the vehicle's brake pedal is associated with the virtual brake pedal in the game, and the vehicle's gear lever is associated with the virtual gear lever in the game.

[0058] For other types of games, the vehicle's steering wheel can be associated with a virtual option device in the game to select different options; the vehicle's accelerator pedal can be associated with a confirmation device in the game to confirm information.

[0059] It should be noted that the above are merely examples of the association between vehicle control devices and virtual controls in the game, and are not intended to impose specific limitations. Different association settings can be implemented for different games. In-vehicle games can pre-configure the association between the vehicle control devices and the virtual controls during development, automatically linking them when the game is launched from the vehicle's control center. Alternatively, users can manually configure the association between the vehicle control devices and the virtual controls when launching the game.

[0060] S103, Obtain the scene interface of the game.

[0061] In this embodiment, the game's scene interface can be pre-defined, for example, a scene interface set during game development. When the game is launched, the vehicle's control center loads the pre-defined scene interface. In another embodiment, the game's scene interface can also be generated later; specific methods are described in the following embodiments and will not be repeated here.

[0062] S104, the scene interface is displayed on the windshield and / or window glass of the vehicle through the vehicle's head-up display system.

[0063] A head-up display (HUD) system can project various vehicle information onto the windshield or two meters in front of the vehicle, so that the information is directly projected into the driver's line of sight.

[0064] In this embodiment, the HUD displays the game screen from the driver's perspective, and can be set to half-screen or full-screen display. Taking a racing game as an example, the displayed content includes lane information, the 3D image of the current vehicle, and information about other players in online racing. HUD technology can also be used on the left and right doors and windows to display the game's road scenery, providing an immersive gaming experience.

[0065] In one embodiment, the scene interface includes a first area screen and a second area screen, wherein the first area screen is the screen of the middle area of ​​the scene interface, and the second area screen is the screen of the scene interface excluding the first area screen.

[0066] For example, if the scene interface is a racing game interface, the first area of ​​the screen is the view of the road in front of the viewer, and the second area of ​​the screen is the view of the scenery on both sides of the road in the viewer.

[0067] Accordingly, S104 may include: displaying the first area of ​​the scene interface on the windshield of the vehicle through the head-up display system; and displaying the second area of ​​the scene interface on the window of the vehicle through the head-up display system.

[0068] Taking racing games as an example, the game screen from the user's perspective includes the view of the road ahead (first area screen) and the view of the scenery on both sides of the road (second area screen). The view of the road ahead is displayed on the windshield, the view of the scenery on the left side of the road is displayed on the driver's side window, and the view of the scenery on the right side of the road is displayed on the passenger side window.

[0069] The above display method can display the game screen in a more three-dimensional way, allowing users to immerse themselves in the game in the cabin and improving the user experience.

[0070] Optionally, the step of displaying the first area of ​​the scene interface onto the windshield of the vehicle via the head-up display system can be divided into the following cases:

[0071] When the game is in single-player mode, the first area of ​​the screen is displayed in full screen on the windshield through the head-up display system;

[0072] When the game is in two-player mode, the head-up display system displays half of the first area of ​​the screen from the first perspective on the left side of the windshield, and half of the first area of ​​the screen from the second perspective on the right side of the windshield.

[0073] The co-pilot can participate in the two-player game module by connecting an external game controller and experience two-player games.

[0074] Optionally, when the game is in two-player mode, the head-up display system can also be used to display half of the first area of ​​the screen from the first perspective on the right side of the windshield, and half of the first area of ​​the screen from the second perspective on the left side of the windshield. Whether it is displayed on the left or right side can be set according to the actual situation, and no specific limitation is made here.

[0075] In this two-player mode, the first-person and second-person perspectives represent the viewpoints of the two users, respectively. Optionally, for two-player mode, the second area of ​​the game screen in the first-person perspective can be displayed on the side of the car window where the first area of ​​the first-person perspective is displayed. For example, if half of the first area of ​​the first-person perspective is displayed on the left side of the windshield, the corresponding half of the second area on the left side of the first area in the first-person perspective will be displayed on the driver's side window. Similarly, if half of the first area of ​​the second-person perspective is displayed on the right side of the windshield, the corresponding half of the second area on the right side of the first area in the second-person perspective will be displayed on the passenger side window.

[0076] In another embodiment, the user can also choose how the game screen is projected. For example, when the game is in single-player mode, the user can choose to display the game screen in full screen or half screen. When choosing half screen display, the user can choose which half of the windshield to display the screen on, depending on their seat. For example, if the user is sitting on the driver's side, they can choose to display the first half of the screen on the left side of the windshield; if the user is sitting on the passenger side, they can choose to display the first half of the screen on the right side of the windshield.

[0077] Using the above display method, during two-player games, the HUD displays in full screen, with the left and right sides showing the perspectives of the driver and passenger, respectively, displaying the game interface for each user. This allows the driver and passenger to play the game together, enhancing the entertainment value of the game.

[0078] In the embodiments S101-S104 described above, when the vehicle is parked, the virtual control device for the game can be connected to the vehicle's control equipment to control the virtual scene in the game. Additionally, the game interface can be displayed on the windshield and / or windows via the vehicle's head-up display system, allowing the user to experience the game immersively. Through these methods, the vehicle's control equipment is fully utilized, enhancing human-vehicle interaction in the cockpit, improving the vehicle's intelligence, and ultimately enhancing the user experience.

[0079] To enhance the realism of the game experience, in one embodiment, S103 includes:

[0080] The recorded images are acquired from the vehicle's dashcam; the recorded images are used to generate the scene interface.

[0081] Using the methods described above, during normal driving, the vehicle's control center collects road information through the dashcam and performs deep algorithm learning to integrate key and distinctive road information into the game. This information can then be shared and published, allowing users to even travel while playing the game. In addition to the default game interface, users can also intelligently generate their own customized game interface with a single click.

[0082] The video recorded by the dashcam contains interfering information such as surrounding vehicles, and since the dashcam records 2D video, it is insufficient for use in games. Therefore, it is necessary to convert the recorded images into a game scene interface. Optionally, generating a scene interface from the recorded images may include the following steps:

[0083] 1. Remove the vehicle image from the recorded image to obtain the background image.

[0084] Alternatively, existing image segmentation techniques can be used to segment out the vehicle image from the recorded image, leaving the remaining image as the background image.

[0085] For example, an end-to-end deep learning approach can be adopted, with the deep learning model based on the UNet encoder-decoder structure. The encoder downsamples the image and extracts rich semantic and texture features, while the decoder upsamples the image, restores it to the input resolution, and fuses the features with the encoder features to obtain a segmented image with relatively complete edges.

[0086] In the above method, because the vehicle image is extracted from the recorded image, the resulting background image has gaps. Therefore, the game scene interface generated directly using this background image is incomplete, affecting the user's gaming experience. To solve this problem, alternatively, step I can be implemented as follows:

[0087] The recorded image is segmented to obtain a first image that does not contain the vehicle image; the first image is then inpainted to obtain the background image.

[0088] Image inpainting is a technique that fills in the empty areas left after a vehicle image has been cut out.

[0089] To improve the precision of image restoration, one possible approach to performing image restoration processing on the first image is as follows:

[0090] The first image is coarsely repaired using a trained coarse repair model to obtain a coarsely repaired image; the background image is then obtained by finely repairing the coarsely repaired image using a trained fine repair model.

[0091] In this embodiment, an end-to-end deep learning approach can be employed, with the deep learning model based on the ContextEncoders structure. This structure is a deep generative model-based method capable of synthesizing novel image structures and explicitly utilizing surrounding image features as a reference for image inpainting during network training, thereby generating better predictions. This structure is primarily used to inpaint images with visually reasonable structure and texture, and is divided into two stages, with the first stage being coarse inpainting processing (e.g., ...). Figure 2 The Coarse Network shown can be used with dilated convolutional networks (such as...). Figure 2 The dilated convolution shown is trained using reconstruction loss to obtain a coarsely repaired image (e.g., ...). Figure 2 The first stage is the Coarse Result shown; the second stage is fine repair treatment (such as...). Figure 2 The Refinement network shown can use a content-aware attention layer to perform fine-grained refinement, resulting in refined refinement results (such as...). Figure 2 The Inpainting Result shown is the final background image.

[0092] like Figure 2 As shown, both the dilated convolutional network in the first stage and the dilated convolutional network in the second node are trained networks. During training, images are recorded ( Figure 2 The image (as shown in the Raw image) is segmented to obtain the first image (e.g., Figure 2 (Input and Mask shown); The first image is input into the first-stage network and the second-stage network in turn to obtain the global result (e.g., Input and Mask); Figure 2 The Global Critic (as shown) and local results (such as...) Figure 2The Local Critic (as shown); calculates the loss value based on the global and local results (e.g., Figure 2 The Spatial Discounted L1 loss is shown, and the network parameters of the first-stage and second-stage networks are updated according to the loss value. Training continues until a preset condition is met, at which point the training terminates, and the trained first-stage and second-stage networks are obtained. The preset condition can be either reaching a preset number of training iterations or reaching a preset training accuracy.

[0093] II. Estimate the depth information of the background image.

[0094] In this embodiment, existing monocular depth estimation techniques can be used to estimate the depth information of the background image. Monocular depth estimation techniques estimate the depth information of each frame of a video image (RGB image) after restoration. This technique employs a self-supervised learning method, which can directly learn the depth information without requiring ground truth values. The deep learning model can be monodepthv2, which uses a combination of depth estimation and pose estimation networks to predict the depth in a single frame image. It achieves this by training an architecture based on a self-supervised loss function on a series of moving image sequences. This architecture includes two networks: one for predicting depth on monocular images and the other for predicting pose between moving images. This method does not require labeled training datasets.

[0095] III. Based on the depth information of the background image, render the background image as the scene interface.

[0096] In this embodiment, image rendering can employ existing rendering engines, such as DirectX or OpenGL. Specific details will not be elaborated upon here.

[0097] Through the above embodiments, road images recorded by the dashcam can be used to generate a game interface, allowing users to experience realistic road scenes during the game and improving the user's gaming experience.

[0098] In one embodiment, after associating the vehicle's control device with the game's virtual control device, the method further includes:

[0099] If an exit command is received from the game, turn off the HUD; disconnect the connection between the vehicle's control device and the game's virtual control device; and restore the communication connection between the vehicle's control device and the controlled device.

[0100] Corresponding to S101, the communication connection between the vehicle's control device and the controlled device is restored, enabling the vehicle's control device to control the controlled device. In application scenarios where the vehicle's control device sends control signals to the controlled device through the vehicle's control center, the vehicle's control center resumes forwarding the control signals sent from the control device to the controlled device, thereby restoring the communication connection between the control device and the controlled device. In application scenarios where the vehicle's control device directly sends control signals to the controlled device, the vehicle's control center can stop sending interception signals, thereby restoring the communication connection between the vehicle's control device and the controlled device.

[0101] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0102] Corresponding to the vehicle control method described in the above embodiments, Figure 3 This is a structural block diagram of the vehicle control device provided in the embodiments of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0103] Reference Figure 3 The device includes:

[0104] The disconnection unit 31 is used to disconnect the communication connection between the vehicle's control device and the controlled device when a game start command is received and the vehicle is in a parked state.

[0105] The association unit 32 is used to associate the vehicle's control device with the game's virtual control device.

[0106] Acquisition unit 33 is used to acquire the scene interface of the game.

[0107] Projection unit 34 is used to display the scene interface on the windshield and / or window glass of the vehicle through the head-up display system of the vehicle.

[0108] Optionally, the acquisition unit 33 is also used for:

[0109] Acquire recorded images from the vehicle's dashcam;

[0110] The recorded image is used to generate the scene interface.

[0111] Optionally, the acquisition unit 33 is also used for:

[0112] Remove the vehicle image from the recorded image to obtain the background image;

[0113] Estimate the depth information of the background image;

[0114] Based on the depth information of the background image, the background image is rendered into the scene interface.

[0115] Optionally, the acquisition unit 33 is also used for:

[0116] The recorded image is segmented to obtain a first image that does not contain the vehicle image;

[0117] The first image is subjected to image restoration processing to obtain the background image.

[0118] Optionally, the acquisition unit 33 is also used for:

[0119] The first image is coarsely restored using the trained coarse restoration model to obtain the coarsely restored image.

[0120] The background image is obtained by performing fine restoration processing on the coarsely restored image using the trained fine restoration model.

[0121] Optionally, the scene interface includes a first area screen and a second area screen, wherein the first area screen is the screen of the middle area of ​​the scene interface, and the second area screen is the screen of the scene interface excluding the first area screen.

[0122] Correspondingly, the projection unit 34 is also used for:

[0123] The head-up display system displays the first area of ​​the scene interface onto the windshield of the vehicle.

[0124] The head-up display system displays the second area of ​​the scene interface onto the vehicle's window.

[0125] Optionally, the projection unit 34 is also used for:

[0126] When the game is in single-player mode, the first area of ​​the screen is displayed in full screen on the windshield through the head-up display system;

[0127] When the game is in two-player mode, the head-up display system displays half of the first area of ​​the screen from the first perspective on the left side of the windshield, and half of the first area of ​​the screen from the second perspective on the right side of the windshield.

[0128] Optionally, device 3 also includes:

[0129] The recovery unit 35 is used to, after associating the vehicle's control device with the game's virtual control device, if it receives an exit command from the game, disconnect the association between the vehicle's control device and the game's virtual control device; and restore the communication connection between the vehicle's control device and the controlled device.

[0130] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0131] in addition, Figure 3 The vehicle control device shown can be a software unit, a hardware unit, or a combination of software and hardware built into an existing terminal device. It can also be integrated into the terminal device as an independent component, or it can exist as a standalone terminal device.

[0132] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0133] Figure 4 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. For example... Figure 4 As shown, the terminal device 4 in this embodiment includes: at least one processor 40 ( Figure 4 (Only one is shown) a processor, a memory 41, and a computer program 42 stored in the memory 41 and executable on the at least one processor 40, which, when executing the computer program 42, implements the steps in any of the above vehicle control method embodiments.

[0134] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. This terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 4This is merely an example of terminal device 4 and does not constitute a limitation on terminal device 4. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0135] The processor 40 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0136] In some embodiments, the memory 41 may be an internal storage unit of the terminal device 4, such as a hard disk or memory of the terminal device 4. In other embodiments, the memory 41 may be an external storage device of the terminal device 4, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal device 4. Furthermore, the memory 41 may include both internal and external storage units of the terminal device 4. The memory 41 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 41 can also be used to temporarily store data that has been output or will be output.

[0137] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0138] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments above.

[0139] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / terminal equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0140] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0141] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0142] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0143] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0144] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A vehicle control method, characterized in that, include: When a game start command is received and the vehicle is in a parked state, the communication connection between the vehicle's control device and the controlled device is cut off. Associate the vehicle's control equipment with the game's virtual control device; Obtain the scene interface of the game; wherein the scene interface includes a first area screen and a second area screen, the first area screen is the screen of the middle area of ​​the scene interface, and the second area screen is the screen of the scene interface excluding the first area screen. The scene interface is displayed on the windshield and / or window glass of the vehicle through the vehicle's head-up display system; Displaying the scene interface on the windshield and / or window glass of the vehicle through the vehicle's head-up display system includes: When the game is in single-player mode, the first area of ​​the screen is displayed in full screen on the windshield through the head-up display system; When the game is in two-player mode, the head-up display system displays half of the first area of ​​the screen from the first perspective on the left side of the windshield, and half of the first area of ​​the screen from the second perspective on the right side of the windshield; wherein, the first perspective and the second perspective are the perspectives of the two users in the two-player mode. The second area of ​​the scene interface is displayed on the vehicle window glass through the head-up display system; The step of obtaining the game's scene interface includes: Acquire recorded images from the vehicle's dashcam; The recorded image is segmented to obtain a first image that does not contain vehicle images; The first image is coarsely restored by the trained coarse restoration model to obtain a coarsely restored image. The trained fine restoration model is used to perform fine restoration processing on the coarse restoration image to obtain the background image; Estimate the depth information of the background image; Based on the depth information of the background image, the background image is rendered into the scene interface.

2. The vehicle control method as described in claim 1, characterized in that, After associating the vehicle's control device with the game's virtual control device, the method further includes: If an exit command is received from the game, the connection between the vehicle's control device and the game's virtual control device is severed; Restore the communication connection between the vehicle's control equipment and the controlled equipment.

3. A vehicle control device, characterized in that, include: The disconnection unit is used to disconnect the communication connection between the vehicle's control device and the controlled device when a game start command is received and the vehicle is in a parked state. The association unit is used to associate the vehicle's control device with the game's virtual control device; An acquisition unit is used to acquire the scene interface of the game; wherein the scene interface includes a first area screen and a second area screen, the first area screen is the screen of the middle area of ​​the scene interface, and the second area screen is the screen of the scene interface excluding the first area screen. A projection unit is used to display the scene interface on the windshield and / or window glass of the vehicle through the head-up display system of the vehicle. The projection unit is also used for: When the game is in single-player mode, the first area of ​​the screen is displayed in full screen on the windshield through the head-up display system; When the game is in two-player mode, the head-up display system displays half of the first area of ​​the screen from the first perspective on the left side of the windshield, and half of the first area of ​​the screen from the second perspective on the right side of the windshield; wherein, the first perspective and the second perspective are the perspectives of the two users in the two-player mode. The second area of ​​the scene interface is displayed on the vehicle window glass through the head-up display system; The acquisition unit is also used for: Acquire recorded images from the vehicle's dashcam; The recorded image is segmented to obtain a first image that does not contain vehicle images; The first image is coarsely restored by the trained coarse restoration model to obtain a coarsely restored image. The trained fine restoration model is used to perform fine restoration processing on the coarse restoration image to obtain the background image; Estimate the depth information of the background image; Based on the depth information of the background image, the background image is rendered into the scene interface.

4. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in claim 1 or 2.

Images