A virtual reality-based vehicle control method and device
By simulating the vehicle control action requests and feedback of real vehicles on virtual reality devices, the problem of lack of immersion in virtual reality vehicle control in existing technologies is solved, and a vivid remote vehicle control experience and information display are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NINGBO GEELY AUTOMOBILE RES & DEV CO LTD
- Filing Date
- 2022-12-20
- Publication Date
- 2026-07-10
AI Technical Summary
Existing virtual reality vehicle control applications lack immersive interaction methods, display incomplete vehicle information, and provide a dull user experience.
By simulating a real-world target vehicle on a virtual reality device, a vehicle control action request is generated. Based on the mapping relationship between vehicle component identification information and vehicle control commands, commands are sent to the cloud to control the real vehicle, and feedback results are received to synchronize the virtual vehicle's actions, thereby enhancing the sense of immersion.
It provides a vivid remote vehicle control experience, comprehensively displays vehicle information, and enhances the user's operating experience.
Smart Images
Figure CN116095125B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent control technology, and specifically to a vehicle control method and device based on virtual reality. Background Technology
[0002] With the development of vehicle networking technology, remote vehicle control via APP has become the choice of many users. However, the interaction methods of existing virtual reality vehicle control applications are generally limited to button-triggered vehicle control commands, pop-up displays of vehicle control feedback effects, and flat displays of alarms and vehicle status data. The overall interaction is not vivid, the vehicle information displayed is incomplete, it is relatively simple and lacks immersion, it cannot demonstrate the vehicle control effect, and the user operation experience is boring. Summary of the Invention
[0003] To address the aforementioned technical problems, this invention provides a vehicle control method and apparatus based on virtual reality, the solution of which is as follows:
[0004] Firstly, a vehicle control method based on virtual reality is provided, and the virtual reality vehicle control application includes:
[0005] Obtain a vehicle control action request for a virtual target vehicle, the vehicle control action request carrying target vehicle component identification information; the virtual target vehicle is modeled based on a real target vehicle according to a preset scale;
[0006] In response to the vehicle control action request, based on the mapping relationship between vehicle component identification information and vehicle control commands, the target vehicle control command corresponding to the target vehicle component identification information is determined; the target vehicle control command is used to control the target vehicle component indicated by the target vehicle component identification information to perform the corresponding vehicle control action;
[0007] Under the condition of meeting the preset communication security conditions, the target vehicle control command is sent to the cloud so that the cloud instructs the target vehicle component in the real target vehicle to perform the corresponding physical vehicle control action based on the target vehicle control command;
[0008] The system receives vehicle control feedback results sent from the cloud for the real target vehicle. When the vehicle control feedback results indicate that the target vehicle component in the real target vehicle successfully executes the corresponding physical vehicle control action based on the target vehicle control command, the system instructs the target vehicle component in the virtual target vehicle to execute a virtual vehicle control action consistent with the physical vehicle control action based on the vehicle control feedback results.
[0009] Optionally, obtaining the vehicle control action request for the virtual target vehicle includes:
[0010] In response to the user's touch command for the virtual target vehicle, a target touch scene animation is generated;
[0011] Based on the correspondence between touch scene animation and vehicle component identification information, the target vehicle component identification information corresponding to the target touch scene animation is determined, and the vehicle control action request for the virtual target vehicle is generated.
[0012] Optionally, instructing the target vehicle components in the virtual target vehicle to perform virtual vehicle control actions consistent with the physical vehicle control actions based on the vehicle control feedback results includes:
[0013] Based on the vehicle control feedback results, the target vehicle components in the virtual target vehicle are determined;
[0014] Based on the correspondence between vehicle component identification information and virtual scene animation, the target virtual scene animation corresponding to the target vehicle component is determined;
[0015] Based on the target virtual scene animation, the target vehicle components in the virtual target vehicle are controlled to perform virtual vehicle control actions that are consistent with the physical vehicle control actions.
[0016] Optionally, the method further includes:
[0017] When the vehicle control feedback result indicates that the target vehicle component in the real target vehicle has failed to successfully execute the corresponding physical vehicle control action based on the target vehicle control command, a failure alarm message is generated based on the vehicle control feedback result.
[0018] The failure alarm information is displayed on the virtual screen of the virtual target vehicle.
[0019] Optionally, the method further includes:
[0020] Obtain the first ciphertext and the pre-stored second ciphertext from the cloud; the first ciphertext is generated by encrypting the target authorization key with the private first key of the cloud; the second ciphertext is generated by encrypting the public first key of the cloud with the second key.
[0021] The public first key is obtained by decrypting the second ciphertext using the second key;
[0022] Based on the first public key, the first ciphertext is decrypted to obtain the authorization key to be compared;
[0023] The authorization key to be compared is compared with the target authorization key, and if the comparison result indicates that the preset communication security conditions are met, it is confirmed that the conditions are met.
[0024] Optionally, the vehicle control method is implemented based on a virtual reality vehicle control application, and before obtaining the vehicle control action request for the virtual target vehicle, it further includes:
[0025] The various functional modules within the virtual reality vehicle control application are initialized and configured, including determining the business priority of each functional module;
[0026] System resources are allocated to each functional module based on a first priority order to enable each functional module to execute corresponding virtual vehicle control actions; the first priority order is determined based on the priority of functional services.
[0027] Optionally, the method further includes:
[0028] When the virtual reality vehicle control application switches to the system background, a connection is established between the keep-alive module and the service module.
[0029] System resources are reallocated for each functional module based on a second priority order; the second priority order is determined based on business priority, functional module performance level, and memory level.
[0030] When the process of the virtual reality vehicle control application is passively stopped, the service module is woken up by the keep-alive module to restart the virtual reality vehicle control application.
[0031] Secondly, a vehicle control device based on virtual reality is provided, comprising:
[0032] The interaction module is used to obtain vehicle control action requests for a virtual target vehicle, wherein the vehicle control action requests carry target vehicle component identification information; the virtual target vehicle is modeled according to a preset scale based on a real target vehicle.
[0033] The vehicle control command determination module is used to respond to the vehicle control action request and determine the target vehicle control command corresponding to the target vehicle component identification information based on the mapping relationship between vehicle component identification information and vehicle control commands; the target vehicle control command is used to control the target vehicle component indicated by the target vehicle component identification information to perform the corresponding vehicle control action;
[0034] The vehicle control module is used to send the target vehicle control command to the cloud when the preset communication security conditions are met, so that the cloud instructs the target vehicle component in the real target vehicle to perform the corresponding physical vehicle control action based on the target vehicle control command;
[0035] The feedback module is used to receive the vehicle control feedback result sent by the cloud for the real target vehicle. When the vehicle control feedback result is that the target vehicle component in the real target vehicle successfully executes the corresponding physical vehicle control action based on the target vehicle control command, the module instructs the target vehicle component in the virtual target vehicle to execute a virtual vehicle control action that is consistent with the physical vehicle control action based on the vehicle control feedback result.
[0036] Optionally, the interaction module includes:
[0037] An interaction unit is used to generate a target touch scene animation in response to a user's touch command for the virtual target vehicle;
[0038] The vehicle control action request determination unit is used to determine the target vehicle component identification information corresponding to the target touch scene animation based on the correspondence between the touch scene animation and the vehicle component identification information, and to generate the vehicle control action request for the virtual target vehicle.
[0039] Optionally, the feedback module includes:
[0040] A virtual target vehicle component determination unit is used to determine the target vehicle component in the virtual target vehicle based on the vehicle control feedback result;
[0041] The target virtual scene animation generation unit is used to determine the target virtual scene animation corresponding to the target vehicle component based on the correspondence between vehicle component identification information and virtual scene animation;
[0042] The virtual vehicle control execution unit is used to control the target vehicle components in the virtual target vehicle to perform virtual vehicle control actions that are consistent with the physical vehicle control actions based on the target virtual scene animation.
[0043] Optionally, the device further includes:
[0044] The failure alarm information generation module is used to generate failure alarm information based on the vehicle control feedback result when the vehicle control feedback result indicates that the target vehicle component in the real target vehicle has failed to successfully execute the corresponding physical vehicle control action based on the target vehicle control command.
[0045] The display module is used to display the failure alarm information on the virtual screen of the virtual target vehicle.
[0046] Optionally, the device further includes:
[0047] The ciphertext acquisition module is used to acquire the first ciphertext and the pre-stored second ciphertext in the cloud; the first ciphertext is generated by encrypting the target authorization key with the private first key in the cloud; the second ciphertext is generated by encrypting the public first key in the cloud with the second key.
[0048] The first decryption module is used to decrypt the second ciphertext based on the second key to obtain the public first key;
[0049] The second decryption module is used to decrypt the first ciphertext based on the public first key to obtain the authorization key to be compared;
[0050] The security verification module is used to compare the authorization key to be compared with the target authorization key, and to confirm that the preset communication security conditions are met if the comparison result indicates that the comparison is correct.
[0051] Optionally, the device is used in a virtual reality vehicle control application, and the device further includes:
[0052] An initialization configuration module is used to initialize and configure various functional modules within the virtual reality vehicle control application. The initialization configuration includes determining the business priority of each functional module.
[0053] The first resource allocation module is used to allocate system resources to the functional modules based on the first priority order of the functional modules so that the functional modules can perform corresponding virtual vehicle control actions; the first priority order is determined based on the functional business priority.
[0054] Optionally, the device further includes:
[0055] The connection module is used to establish a connection between the keep-alive module and the service module when the virtual reality vehicle control application switches to the background of the system.
[0056] The second resource allocation module is used to reallocate system resources to the various functional modules based on a second priority order; the second priority order is determined based on business priority, functional module performance level, and memory level.
[0057] The keep-alive module is used to wake up the service module to restart the virtual reality vehicle control application when the process of the virtual reality vehicle control application is passively stopped.
[0058] Thirdly, an electronic device is provided, including a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the steps of the above method.
[0059] Fourthly, a computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored therein, the at least one instruction or the at least one program being loaded and executed by a processor to implement the steps of the above method.
[0060] Fifthly, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium, a processor of a computer device reading the computer instructions from the computer-readable storage medium, the processor executing the computer instructions, causing the computer device to perform the steps of the above method.
[0061] By adopting the above technical solution, the present invention has the following beneficial effects:
[0062] This invention allows users to perform touch operations on components of a virtual target vehicle modeled to a preset scale, generating corresponding vehicle control commands. These commands are then sent to the cloud, where the real vehicle is instructed to execute the control actions. The cloud then receives the control feedback and instructs the virtual target vehicle to also display the corresponding control actions, i.e., virtual animation. This makes remote vehicle control no longer limited to 2D interaction with an app, but allows users to truly experience the control effects. The entire process is vivid, immersive, and provides a more comprehensive display of vehicle information, enhancing the user experience.
[0063] Other features and advantages of the present invention will be described in detail in the following detailed description section. Attached Figure Description
[0064] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and the same reference numerals usually represent the same parts. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0065] Figure 1 This is a schematic diagram of a remote vehicle control architecture in the prior art;
[0066] Figure 2 This is a schematic diagram of the overall architecture of the vehicle control method based on virtual reality provided in an embodiment of the present invention.
[0067] Figure 3 A flowchart illustrating a vehicle control method based on virtual reality provided in an embodiment of the present invention;
[0068] Figure 4A flowchart illustrating an optional method for implementing a virtual reality-based vehicle control method provided in an embodiment of the present invention;
[0069] Figure 5 A schematic diagram of the overall architecture of an optional method for implementing a virtual reality-based vehicle control method provided in this embodiment of the invention;
[0070] Figure 6a This is a flowchart illustrating an optional method for implementing a virtual reality-based vehicle control method provided in an embodiment of the present invention.
[0071] Figure 6b A flowchart illustrating an optional method for implementing a virtual reality-based vehicle control method provided in an embodiment of the present invention;
[0072] Figure 7 A schematic diagram of a vehicle control device based on virtual reality provided in an embodiment of the present invention;
[0073] Figure 8 A schematic diagram of the terminal hardware structure for implementing the virtual reality-based vehicle control method provided for the present invention. Detailed Implementation
[0074] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0075] The term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the invention. In the description of the invention, it should be understood that the terms "upper," "lower," "top," "bottom," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature. Moreover, the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein.
[0076] Most existing remote vehicle control technologies are implemented through communication between an app (application) and the vehicle network. The architecture for app-based remote vehicle control can be referenced. Figure 1 The app includes an interaction module for users to trigger vehicle control commands and display feedback results. The vehicle-to-everything (V2X) communication module connects to the V2X cloud, sending vehicle control commands to the cloud so that the cloud can control the corresponding components of the vehicle to execute those commands. (Reference) Figure 2 The vehicle control method based on virtual reality of the present invention is implemented through a virtual reality vehicle control application installed on a VR (Virtual Reality) device. Compared with the vehicle control APP in the prior art, the present invention adds a VR engine unit, including an interaction module and a communication module. The interaction module is used for users to interact with the virtual target vehicle model and trigger vehicle control commands. The communication module of the VR engine unit is connected to the communication module in the Android native component unit. The Android native component unit has multiple modules, including a vehicle network communication module, which is used to communicate with the vehicle network cloud. The vehicle network cloud is connected to the vehicle control unit of the real target vehicle to realize remote control of the real target vehicle.
[0077] refer to Figure 3 The diagram illustrates a flowchart of a vehicle control method based on virtual reality provided in an embodiment of the present invention. It should be noted that while this specification provides the operational steps of the method as described in the embodiments or flowchart, more or fewer operational steps may be included based on conventional or non-inventive methods. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual system devices or products, the method can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment) as shown in the embodiments or drawings. The vehicle control method based on virtual reality provided in this embodiment of the present invention includes:
[0078] S301, Obtain a vehicle control action request for a virtual target vehicle, wherein the vehicle control action request carries target vehicle component identification information; the virtual target vehicle is modeled based on a real target vehicle according to a preset ratio;
[0079] In one possible implementation, step S301 may include:
[0080] In response to the user's touch command for the virtual target vehicle, a target touch scene animation is generated;
[0081] Based on the correspondence between touch scene animation and vehicle component identification information, the target vehicle component identification information corresponding to the target touch scene animation is determined, and the vehicle control action request for the virtual target vehicle is generated.
[0082] Specifically, the virtual target vehicle in this embodiment of the invention is obtained by 3D modeling a real target vehicle at a 1:1 scale, which can accurately reproduce the shape and interior data of the real target vehicle, ensuring the user experience. Based on actual vehicle control needs, virtual scene animations and touch scene animations corresponding to various vehicle control commands are designed based on the virtual target vehicle. For example, in one embodiment, the vehicle control command is "control the left window to descend," then the touch scene animation could be pressing the left window descent button, and the virtual scene animation could be the descent of the left window of the virtual target vehicle. The model of the virtual target vehicle is imported into the database of the vehicle network cloud, and the virtual scene animations and touch scene animations are imported into a preset animation library. Various components of the vehicle are encoded in the vehicle network cloud to generate vehicle component identification information, establishing a mapping relationship between animations (virtual scene animations and touch scene animations), vehicle control commands, and vehicle component identification information. This mapping relationship is then synchronized to the virtual reality vehicle control application of this embodiment.
[0083] When a user needs remote vehicle control, they wear a VR device, open the virtual reality vehicle control application, and click on a vehicle component of the virtual target vehicle to generate a touch command. The VR engine unit collects the user's touch commands in real time and generates a target touch scene animation. For example, if the user presses the left window down button, the target touch scene animation shows the button being pressed. In other embodiments, the target touch scene animation can also be replaced by sound prompts or vibration feedback. Based on the correspondence between the touch scene animation and the vehicle component identification information, the target vehicle component identification information corresponding to the target touch scene animation is determined. Continuing with the above example, the target vehicle component identification information is "W02". The vehicle component identification information in this embodiment is for reference only; other identification forms can be set, all of which fall within the protection scope of this invention. Further, a vehicle control action request for the virtual target vehicle is generated, namely, "request to control W02".
[0084] S302, in response to the vehicle control action request, based on the mapping relationship between vehicle component identification information and vehicle control commands, determine the target vehicle control command corresponding to the target vehicle component identification information; the target vehicle control command is used to control the target vehicle component indicated by the target vehicle component identification information to perform the corresponding vehicle control action;
[0085] Specifically, the mapping relationship between the vehicle component identification information of the virtual target vehicle and the real target vehicle and the vehicle control command is consistent. The target vehicle control command can be determined from the target vehicle component identification information carried in the vehicle control action request. For example, the target vehicle control command in the above example can be "control W02 to descend". Then the target vehicle control command is sent to the vehicle network cloud to control the left window of the real target vehicle to descend, and the virtual target vehicle will also perform a virtual scene animation of the left window descending. See the following embodiment for details.
[0086] S303, under the condition of satisfying the preset communication security conditions, the target vehicle control command is sent to the cloud so that the cloud instructs the target vehicle component in the real target vehicle to perform the corresponding physical vehicle control action based on the target vehicle control command;
[0087] Specifically, in common vehicle control applications in existing technologies, the communication process between the APP and the vehicle network cloud is limited by the ease of decompilation of the Android Java language and the lack of certain security strategies, leading to security risks in remote vehicle control. A common practice in existing technologies is to write important information such as the vehicle network key in C and package it into a .so file (a shared library file under Linux, in ELF format) and store it locally on the Android device. However, this solution still does not offer a very high level of security. This invention designs a security mechanism process to improve communication security, referencing... Figure 4 In one possible implementation, the security mechanism process may include the following steps:
[0088] Obtain the first ciphertext and the pre-stored second ciphertext from the cloud; the first ciphertext is generated by encrypting the target authorization key with the private first key of the cloud; the second ciphertext is generated by encrypting the public first key of the cloud with the second key.
[0089] The public first key is obtained by decrypting the second ciphertext using the second key;
[0090] Based on the first public key, the first ciphertext is decrypted to obtain the authorization key to be compared;
[0091] The authorization key to be compared is compared with the target authorization key, and if the comparison result indicates that the preset communication security conditions are met, it is confirmed that the conditions are met.
[0092] Specifically, the vehicle-to-everything (V2X) cloud platform issues the V2X authorization key, also known as the target authorization key or the first ciphertext. This key is encrypted using the ECDH (Elliptic Curve Diffie-Hellman key exchange) asymmetric algorithm to obtain a public key and a private key. The public key is publicly released, while the private key is stored on a proprietary server. The public key is the aforementioned public first key, and the private key is the aforementioned private first key. The target authorization key is then encrypted using the private first key to obtain the first ciphertext. On the VR engine side, a symmetric key, the second key, is generated using the AES symmetric encryption algorithm. This second key is then used to encrypt the public first key from the V2X cloud platform to generate the second ciphertext, which is stored in a local database for use in each communication. Each time the virtual reality vehicle control application of this invention is launched, in order to securely establish a communication connection between the APP and the vehicle network cloud, the first encrypted text and the pre-stored second encrypted text are obtained from the vehicle network cloud. The second encrypted text is decrypted using the second key to obtain the public first key. The first encrypted text is then decrypted using the public first key to obtain the authorization key to be compared. The vehicle network SDK of the Android unit compares the authorization key to be compared with the target authorization key issued by the vehicle network cloud. If they match, it is confirmed that the preset communication security conditions are met, and a communication connection can be established to send the target vehicle control command to the vehicle network cloud.
[0093] The above implementation methods can improve the privacy and security of virtual reality vehicle control applications and vehicle network cloud communication.
[0094] After receiving the target vehicle control command, the cloud parses it to confirm the virtual reality vehicle control application from which it originates. Based on the mapping relationship between the vehicle control command and vehicle component identification information, it identifies the target vehicle component in the real-world target vehicle corresponding to the target vehicle control command and instructs the target vehicle component to execute the corresponding target vehicle control command. Furthermore, it stores other information contained in the vehicle control command for reference during subsequent vehicle control or vehicle condition verification. For example, if the target vehicle control command instructs the left window to lower to full opening, and the next vehicle control command instructs the left window to lower, the stored information indicates that the command cannot be executed, and a failure result will be reported to the virtual reality vehicle control application. Further, in some embodiments, the vehicle network cloud can automatically trigger new vehicle control commands based on the target vehicle control command and vehicle condition. For example, if the target vehicle control command is to report the interior temperature, and the interior temperature exceeds a preset temperature threshold, the cloud will trigger vehicle control commands such as turning on the air conditioning to lower the interior temperature and ensure vehicle safety.
[0095] S304, receive the vehicle control feedback result sent by the cloud for the real target vehicle. When the vehicle control feedback result indicates that the target vehicle component in the real target vehicle successfully executes the corresponding physical vehicle control action based on the target vehicle control command, instruct the target vehicle component in the virtual target vehicle to execute a virtual vehicle control action consistent with the physical vehicle control action based on the vehicle control feedback result.
[0096] In some possible implementations, step S304 may include:
[0097] Based on the vehicle control feedback results, the target vehicle components in the virtual target vehicle are determined;
[0098] Based on the correspondence between vehicle component identification information and virtual scene animation, the target virtual scene animation corresponding to the target vehicle component is determined;
[0099] Based on the target virtual scene animation, the target vehicle components in the virtual target vehicle are controlled to perform virtual vehicle control actions that are consistent with the physical vehicle control actions.
[0100] Specifically, after the cloud instructs the real-world target vehicle to execute the target empty vehicle command, it returns a vehicle control feedback result to the virtual reality vehicle control application. When the vehicle control feedback result indicates that the target vehicle component has successfully executed the target empty vehicle command, the virtual reality vehicle control application determines the target virtual scene animation corresponding to the target vehicle component based on the correspondence between the vehicle component identification information and the virtual scene animation. For example, the virtual scene animation corresponding to the left window descending is indicated. The application instructs the left window in the virtual target vehicle to perform the animation of the window descending, so as to provide feedback on the vehicle control interaction to the user. This allows the user to truly feel the process of the window descending. Compared with the flat interaction of the APP, the vehicle control interaction completed by the embodiment of the present invention through virtual reality is more immersive and realistic.
[0101] In other embodiments, the vehicle control feedback results may also include some new vehicle control commands that are automatically triggered and executed by the vehicle network cloud to inform the user of changes in the vehicle status of the real target vehicle. At the same time, the virtual reality vehicle control application can also interpret the corresponding virtual scene animation based on the information in the vehicle control feedback results to enhance the user's experience.
[0102] Through the above implementation methods, users can truly feel the process of the car window lowering. Compared with the flat interaction of an APP, the vehicle control interaction completed through virtual reality in this embodiment of the invention is more immersive and realistic.
[0103] Furthermore, in some possible implementations, the method further includes:
[0104] When the vehicle control feedback result indicates that the target vehicle component in the real target vehicle has failed to successfully execute the corresponding physical vehicle control action based on the target vehicle control command, a failure alarm message is generated based on the vehicle control feedback result.
[0105] The failure alarm information is displayed on the virtual screen of the virtual target vehicle.
[0106] Specifically, when the real target vehicle fails to execute the corresponding target vehicle control action, such as the left window failing to lower in the example above, a failure alarm message is generated and displayed on the virtual screen of the virtual target vehicle to inform the user. The virtual screen can be the dashboard, the vehicle's infotainment screen, etc.
[0107] In addition, this invention also considers the stability issue of virtual reality vehicle control applications. When the APP is running in the system foreground, refer to Figure 6a In some possible implementations, the method further includes:
[0108] S601a, Initialize the configuration of various functional modules within the virtual reality vehicle control application, the initialization configuration including determining the business priority of each functional module;
[0109] S602a, system resources are allocated to each functional module based on the first priority order of the functional modules so that each functional module can execute the corresponding virtual vehicle control action; the first priority order is determined based on the functional service priority.
[0110] For details, please refer to Figure 5 The system integrates vehicle networking communication modules, keep-alive modules, and other functional modules into the native Android service module. While launching the virtual reality vehicle control application, the service module runs in the background, traverses the various functional modules within the APP, initializes and configures each functional module, determines the business priority of each functional module, and then allocates system resources to each functional module based on the business priority.
[0111] Furthermore, when the app runs in the system background, refer to Figure 6b In some possible implementations, the method further includes:
[0112] S601b, When the virtual reality vehicle control application switches to the system background to run, a connection relationship is established between the keep-alive module and the service module;
[0113] S602b, the system resources of each functional module are reallocated based on a second priority order; the second priority order is determined based on business priority, functional module performance level, and memory level;
[0114] S603b, when the process of the virtual reality vehicle control application is passively stopped, the service module is woken up by the keep-alive module to restart the virtual reality vehicle control application.
[0115] Specifically, when the app runs in the background, some power-consuming and memory-intensive functional modules can be forcibly closed by the system's memory detection mechanism, leading to communication interruptions. Therefore, the service module re-priorsifies and allocates system resources for each functional module. This prioritization considers multiple factors, such as business priority, functional module performance level, and memory usage. Modules with high business priority, low power consumption, and low memory usage are assigned high priority, while those with low business priority, high power consumption, and high memory usage are assigned low priority. This ensures a reasonable allocation of application resources, preventing interference from the system's memory detection mechanism that could affect the stability of vehicle control data communication. In other embodiments, functional modules with high business priority, high power consumption, or large memory usage can be split. For example, an alarm module can be split into an alarm information push module and a processing module. The alarm information push module has low memory usage and can be set to a higher priority to push alarm information to the user in a timely manner. The processing module has high memory usage and high power consumption and can be set to a lower priority. After the user receives the alarm information, the entire alarm module will run in the foreground to handle the alarm event. This can effectively prevent such functional modules with high business priority, high power consumption, or large memory usage from being forcibly stopped by the system, thus improving the application's survival rate. Furthermore, if the APP is killed by the system due to abnormal conditions, the service module can be woken up through the keep-alive module, thereby restoring the normal operation of vehicle control data communication.
[0116] The above implementation methods can improve the stability of virtual reality vehicle control applications, enhance the overall app survival rate through various keep-alive strategies, and ensure normal communication.
[0117] This invention enables user vehicle control interaction through a VR engine, and connects to the vehicle network cloud via native Android components. While remotely controlling the vehicle, users can also experience animated feedback effects from the virtual target vehicle, enhancing the immersion and realism of remote vehicle control and making the entire process more vivid and engaging. Furthermore, it sets up a native Android service module to access the keep-alive SDK, configuring resources for various built-in functional modules in both the system foreground and background, thus improving the overall stability of the virtual reality vehicle control application. A unique security mechanism ensures communication security, enhancing the practicality and user experience of the virtual reality vehicle control application.
[0118] Corresponding to the above-described vehicle control method based on virtual reality, this embodiment of the invention also provides a vehicle control device based on virtual reality. Since the vehicle control device based on virtual reality provided in this embodiment corresponds to the vehicle control method based on virtual reality provided in the above-described embodiments, the implementation methods of the aforementioned vehicle control method based on virtual reality are also applicable to the vehicle control device based on virtual reality provided in this embodiment, and will not be described again in this embodiment of the invention.
[0119] refer to Figure 7 The diagram shows a structural schematic of a vehicle control device based on virtual reality provided in an embodiment of the present invention. This device has the function of implementing the vehicle control method based on virtual reality described in the above-described method embodiments. The function can be implemented by hardware or by hardware executing corresponding software. The device may include:
[0120] The interaction module 710 is used to obtain vehicle control action requests for a virtual target vehicle, wherein the vehicle control action requests carry target vehicle component identification information; the virtual target vehicle is modeled according to a preset scale based on a real target vehicle.
[0121] The vehicle control command determination module 720 is used to respond to the vehicle control action request and determine the target vehicle control command corresponding to the target vehicle component identification information based on the mapping relationship between vehicle component identification information and vehicle control commands; the target vehicle control command is used to control the target vehicle component indicated by the target vehicle component identification information to perform the corresponding vehicle control action;
[0122] The vehicle control module 730 is used to send the target vehicle control command to the cloud when the preset communication security conditions are met, so that the cloud instructs the target vehicle component in the real target vehicle to perform the corresponding physical vehicle control action based on the target vehicle control command;
[0123] The feedback module 740 is used to receive the vehicle control feedback result sent by the cloud for the real target vehicle. When the vehicle control feedback result is that the target vehicle component in the real target vehicle successfully executes the corresponding physical vehicle control action based on the target vehicle control command, the module instructs the target vehicle component in the virtual target vehicle to execute a virtual vehicle control action that is consistent with the physical vehicle control action based on the vehicle control feedback result.
[0124] Optionally, the interaction module 710 includes:
[0125] An interaction unit is used to generate a target touch scene animation in response to a user's touch command for the virtual target vehicle;
[0126] The vehicle control action request determination unit is used to determine the target vehicle component identification information corresponding to the target touch scene animation based on the correspondence between the touch scene animation and the vehicle component identification information, and to generate the vehicle control action request for the virtual target vehicle.
[0127] Optionally, the feedback module 740 includes:
[0128] A virtual target vehicle component determination unit is used to determine the target vehicle component in the virtual target vehicle based on the vehicle control feedback result;
[0129] The target virtual scene animation generation unit is used to determine the target virtual scene animation corresponding to the target vehicle component based on the correspondence between vehicle identification information and virtual scene animation;
[0130] The virtual vehicle control execution unit is used to control the target vehicle components in the virtual target vehicle to perform virtual vehicle control actions that are consistent with the physical vehicle control actions based on the target virtual scene animation.
[0131] Optionally, the device further includes:
[0132] The failure alarm information generation module is used to generate failure alarm information based on the vehicle control feedback result when the vehicle control feedback result indicates that the target vehicle component in the real target vehicle has failed to successfully execute the corresponding physical vehicle control action based on the target vehicle control command.
[0133] The display module is used to display the failure alarm information on the virtual screen of the virtual target vehicle.
[0134] Optionally, the device further includes:
[0135] The ciphertext acquisition module is used to acquire the first ciphertext and the pre-stored second ciphertext in the cloud; the first ciphertext is generated by encrypting the target authorization key with the private first key in the cloud; the second ciphertext is generated by encrypting the public first key in the cloud with the second key.
[0136] The first decryption module is used to decrypt the second ciphertext based on the second key to obtain the public first key;
[0137] The second decryption module is used to decrypt the first ciphertext based on the public first key to obtain the authorization key to be compared;
[0138] The security verification module is used to compare the authorization key to be compared with the target authorization key, and to confirm that the preset communication security conditions are met if the comparison result indicates that the comparison is correct.
[0139] Optionally, the device is used in a virtual reality vehicle control application, and the device further includes:
[0140] An initialization configuration module is used to initialize and configure various functional modules within the virtual reality vehicle control application. The initialization configuration includes determining the business priority of each functional module.
[0141] The first resource allocation module is used to allocate system resources to the functional modules based on the first priority order of the functional modules so that the functional modules can perform corresponding virtual vehicle control actions; the first priority order is determined based on the functional business priority.
[0142] Optionally, the device further includes:
[0143] The connection module is used to establish a connection between the keep-alive module and the service module when the virtual reality vehicle control application switches to the background of the system.
[0144] The second resource allocation module is used to reallocate system resources to the various functional modules based on a second priority order; the second priority order is determined based on business priority, functional module performance level, and memory level.
[0145] The keep-alive module is used to wake up the service module to restart the virtual reality vehicle control application when the process of the virtual reality vehicle control application is passively stopped.
[0146] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.
[0147] This invention also provides an electronic device, including a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the steps of the virtual reality-based vehicle control method described above.
[0148] Memory can be used to store software programs and modules. The processor executes various functional applications by running the software programs and modules stored in the memory. Memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for functions, etc.; the data storage area can store data created based on the use of the device, etc. Furthermore, memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory can also include a memory controller to provide the processor with access to the memory. The processor can be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits (ASICs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.
[0149] The methods and embodiments provided in this invention can be executed on a computer terminal, server, or similar computing device. Taking execution on a terminal as an example, refer to... Figure 8 The diagram shown is a hardware structure diagram of a terminal that runs a virtual reality-based vehicle control method according to an embodiment of the present invention.
[0150] Specifically, the terminal may include an RF (Radio Frequency) circuit 810, a memory 820 including one or more computer-readable storage media, an input unit 830, a display unit 840, a sensor 850, an audio circuit 860, a WiFi (Wireless Fidelity) module 870, a processor 880 including one or more processing cores, and a power supply 890, among other components. Those skilled in the art will understand that... Figure 8 The terminal structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:
[0151] The RF circuit 810 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and hands it over to one or more processors 880 for processing; additionally, it transmits uplink data to the base station. Typically, the RF circuit 810 includes, but is not limited to, an antenna, at least one amplifier, a tuner, one or more oscillators, a Subscriber Identity Module (SIM) card, a transceiver, a coupler, an LNA (Low Noise Amplifier), a duplexer, etc. Furthermore, the RF circuit 810 can also communicate wirelessly with networks and other terminals. The wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System for Mobile communication), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), LTE (Long Term Evolution), email, SMS (Short Messaging Service), etc.
[0152] The memory 820 can be used to store software programs and modules. The processor 880 executes various functional applications and data processing by running the software programs and modules stored in the memory 820. The memory 820 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for the functions, etc.; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 820 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 820 may also include a memory controller to provide access to the memory 820 for the processor 880 and the input unit 830.
[0153] The input unit 830 can be used to receive input digital or character information, and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. Specifically, the input unit 830 may include a touch-sensitive surface 831 and other input devices 832. The touch-sensitive surface 831, also known as a touch display screen or touchpad, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch-sensitive surface 831), and drive the corresponding connection device according to a pre-set program. Optionally, the touch-sensitive surface 831 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to the processor 880, and can receive and execute commands sent by the processor 880. In addition, the touch-sensitive surface 831 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch-sensitive surface 831, the input unit 830 may also include other input devices 832. Specifically, other input devices 832 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.
[0154] The display unit 840 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the terminal. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. The display unit 840 may include a display panel 841, which may optionally be configured as an LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or similar display. Further, a touch-sensitive surface 831 may cover the display panel 841. When the touch-sensitive surface 831 detects a touch operation on or near it, it transmits the information to the processor 880 to determine the type of touch event. Subsequently, the processor 880 provides corresponding visual output on the display panel 841 according to the type of touch event. The touch-sensitive surface 831 and the display panel 841 can be two independent components to implement input and output functions; however, in some embodiments, the touch-sensitive surface 831 and the display panel 841 can be integrated to achieve both input and output functions.
[0155] The terminal may also include at least one sensor 850, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 841 according to the ambient light level, and the proximity sensor can turn off the display panel 841 and / or backlight when the terminal is moved to the ear. As a type of motion sensor, a gravity acceleration sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that identify the terminal's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, tapping), etc. Other sensors that may be configured on the terminal, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.
[0156] Audio circuitry 860, speaker 861, and microphone 862 provide an audio interface between the user and the terminal. Audio circuitry 860 converts received audio data into electrical signals, which are then transmitted to speaker 861, where they are converted into sound signals for output. Conversely, microphone 862 converts collected sound signals into electrical signals, which are received by audio circuitry 860, converted back into audio data, and then processed by processor 880 before being transmitted via RF circuitry 810 to, for example, another terminal, or output to memory 820 for further processing. Audio circuitry 860 may also include an earphone jack to facilitate communication between a peripheral headset and the terminal.
[0157] WiFi is a short-range wireless transmission technology. The terminal, through the WiFi module 870, can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 8 WiFi module 870 is shown, but it is understood that it is not a necessary component of the terminal and can be omitted as needed without changing the nature of the invention.
[0158] The processor 880 is the control center of the terminal, connecting various parts of the terminal via various interfaces and lines. It executes software programs and / or modules stored in the memory 820, and calls data stored in the memory 820 to perform various functions and process data, thereby providing overall monitoring of the terminal. Optionally, the processor 880 may include one or more processing cores; preferably, the processor 880 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 880.
[0159] The terminal also includes a power supply 890 (such as a battery) to power various components. Preferably, the power supply can be logically connected to the processor 880 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 890 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.
[0160] Although not shown, the terminal may also include a camera, Bluetooth module, etc., which will not be described in detail here. Specifically, in this embodiment, the terminal also includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors.
[0161] This invention also provides a computer-readable storage medium storing at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the steps of the virtual reality-based vehicle control method described above. In this invention, the computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.
[0162] This invention also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the virtual reality-based vehicle control method provided in the various optional implementations described above.
[0163] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A vehicle control method based on virtual reality, characterized in that, include: Obtain a vehicle control action request for a virtual target vehicle, the vehicle control action request carrying target vehicle component identification information; the virtual target vehicle is modeled based on a real target vehicle according to a preset ratio; In response to the vehicle control action request, based on the mapping relationship between vehicle component identification information and vehicle control commands, the target vehicle control command corresponding to the target vehicle component identification information is determined; the target vehicle control command is used to control the target vehicle component indicated by the target vehicle component identification information to perform the corresponding vehicle control action; Under the condition of meeting the preset communication security conditions, the target vehicle control command is sent to the cloud so that the cloud instructs the target vehicle component in the real target vehicle to perform the corresponding physical vehicle control action based on the target vehicle control command; The system receives vehicle control feedback results sent from the cloud for the real target vehicle. When the vehicle control feedback results indicate that the target vehicle component in the real target vehicle successfully executes the corresponding physical vehicle control action based on the target vehicle control command, the system instructs the target vehicle component in the virtual target vehicle to execute a virtual vehicle control action consistent with the physical vehicle control action based on the vehicle control feedback results.
2. The vehicle control method based on virtual reality according to claim 1, characterized in that, The process of obtaining vehicle control action requests for the virtual target vehicle includes: In response to the user's touch command for the virtual target vehicle, a target touch scene animation is generated; Based on the correspondence between touch scene animation and vehicle component identification information, the target vehicle component identification information corresponding to the target touch scene animation is determined, and the vehicle control action request for the virtual target vehicle is generated.
3. The vehicle control method based on virtual reality according to claim 1, characterized in that, The step of instructing the target vehicle components in the virtual target vehicle to perform virtual vehicle control actions consistent with the physical vehicle control actions based on the vehicle control feedback results includes: Based on the vehicle control feedback results, the target vehicle components in the virtual target vehicle are determined; Based on the correspondence between vehicle component identification information and virtual scene animation, the target virtual scene animation corresponding to the target vehicle component is determined; Based on the target virtual scene animation, the target vehicle components in the virtual target vehicle are controlled to perform virtual vehicle control actions that are consistent with the physical vehicle control actions.
4. The vehicle control method based on virtual reality according to claim 1, characterized in that, Also includes: When the vehicle control feedback result indicates that the target vehicle component in the real target vehicle has failed to successfully execute the corresponding physical vehicle control action based on the target vehicle control command, a failure alarm message is generated based on the vehicle control feedback result. The failure alarm information is displayed on the virtual screen of the virtual target vehicle.
5. The vehicle control method based on virtual reality according to any one of claims 1 to 4, characterized in that, Also includes: Obtain the first ciphertext and the pre-stored second ciphertext from the cloud; The first ciphertext is generated by encrypting the target authorization key with the private first key in the cloud; The second ciphertext is generated by encrypting the public first key in the cloud with the second key; The public first key is obtained by decrypting the second ciphertext using the second key; Based on the first public key, the first ciphertext is decrypted to obtain the authorization key to be compared; The authorization key to be compared is compared with the target authorization key, and if the comparison result indicates that the preset communication security conditions are met, it is confirmed that the conditions are met.
6. The vehicle control method based on virtual reality according to claim 1, characterized in that, The vehicle control method is implemented based on a virtual reality vehicle control application, and before obtaining the vehicle control action request for the virtual target vehicle, it further includes: The various functional modules within the virtual reality vehicle control application are initialized and configured, including determining the business priority of each functional module; System resources are allocated to each functional module based on a first priority order to enable each functional module to perform its corresponding function; the first priority order is determined based on the functional business priority.
7. The vehicle control method based on virtual reality according to claim 6, characterized in that, Also includes: When the virtual reality vehicle control application switches to the system background, a connection is established between the keep-alive module and the service module. System resources are reallocated for each functional module based on a second priority order; the second priority order is determined based on business priority, functional module performance level, and memory level. When the process of the virtual reality vehicle control application is passively stopped, the service module is woken up by the keep-alive module to restart the virtual reality vehicle control application.
8. A vehicle control device based on virtual reality, characterized in that, include: The interaction module is used to obtain vehicle control action requests for a virtual target vehicle, wherein the vehicle control action requests carry target vehicle component identification information; the virtual target vehicle is modeled according to a preset scale based on a real target vehicle. The vehicle control command determination module is used to respond to the vehicle control action request and determine the target vehicle control command corresponding to the target vehicle component identification information based on the mapping relationship between vehicle component identification information and vehicle control commands; the target vehicle control command is used to control the target vehicle component indicated by the target vehicle component identification information to perform the corresponding vehicle control action; The vehicle control module is used to send the target vehicle control command to the cloud when the preset communication security conditions are met, so that the cloud instructs the target vehicle component in the real target vehicle to perform the corresponding physical vehicle control action based on the target vehicle control command; The feedback module is used to receive the vehicle control feedback result sent by the cloud for the real target vehicle. When the vehicle control feedback result is that the target vehicle component in the real target vehicle successfully executes the corresponding physical vehicle control action based on the target vehicle control command, the module instructs the target vehicle component in the virtual target vehicle to execute a virtual vehicle control action that is consistent with the physical vehicle control action based on the vehicle control feedback result.
9. An electronic device, characterized in that, The method includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the steps of the method as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the steps of the method as claimed in any one of claims 1 to 7.