Vehicle control method and apparatus, and electronic device, product, storage medium and vehicle

Abstract

A vehicle control method and apparatus, and an electronic device, a product, a storage medium and a vehicle. The method comprises: acquiring an electronic image of an intelligent-driving perception interface in real time, wherein the electronic image is configured to display a vehicle indicator and a vehicle control indicator, the vehicle indicator being configured to indicate real-time vehicle information, and the vehicle control indicator being configured to indicate a real-time vehicle control mode of an ego vehicle relative to another vehicle; receiving, in real time, a voice command sent by a user for the electronic image; determining whether a vehicle control mode in the voice command matches the vehicle control mode indicated by the vehicle control indicator, so as to obtain a determination result; and on the basis of the determination result and the real-time vehicle information, controlling whether the ego vehicle travels relative to the other vehicle on the basis of the voice command.

Description

Vehicle control methods, devices, electronic equipment, products, storage media, and vehicles Cross-references to related applications

[0001] This application claims priority to Chinese patent application No. 202510101472.4, filed on January 22, 2025, and Chinese patent application No. 202510105943.9, filed on January 22, 2025, the full text of which is incorporated herein by reference. Technical Field

[0002] This application relates to, but is not limited to, the field of intelligent driving technology, and particularly to a vehicle control method, device, electronic device, program product, storage medium, and vehicle. Background Technology

[0003] In recent years, with the development of intelligent driving technology, more and more vehicles on the market have acquired autonomous driving capabilities. The realization of autonomous driving relies on the vehicle's Advanced Driver Assistance System (ADAS). Once autonomous driving is activated, the ADAS obtains real-time data about the vehicle's surroundings and controls the vehicle to safely travel to its destination along a planned route, following traffic rules. Summary of the Invention

[0004] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0005] This application provides a vehicle control method, device, electronic device, program product, storage medium, and vehicle.

[0006] In a first aspect, embodiments of this application provide a vehicle control method, comprising: acquiring an electronic image of an intelligent driving perception screen in real time; wherein the electronic image is configured to display a vehicle identifier and a vehicle control identifier, the vehicle identifier being configured to indicate real-time vehicle information, and the vehicle control identifier being configured to indicate a real-time vehicle control mode of the vehicle relative to another vehicle; receiving a voice command sent by a user to the electronic image in real time; determining whether the vehicle control mode in the voice command matches the vehicle control mode indicated by the vehicle control identifier, and obtaining a determination result; and controlling whether the vehicle drives relative to the other vehicle according to the voice command based on the determination result and the real-time vehicle information.

[0007] In some embodiments of this application, the vehicle control method in the voice command includes information about a specific vehicle and the specific vehicle control method of the vehicle relative to the specific vehicle; determining whether the vehicle control method in the voice command matches the vehicle control method indicated by the vehicle control identifier includes: determining whether the real-time vehicle information includes vehicle information that matches the information of the specific vehicle based on the information of the specific vehicle; in response to the real-time vehicle information including vehicle information that matches the information of the specific vehicle, identifying the other vehicle corresponding to the vehicle information that matches the information of the specific vehicle as the target vehicle; and determining whether the specific vehicle control method matches the vehicle control method indicated by the vehicle control identifier corresponding to the target vehicle.

[0008] In some embodiments of this application, the other vehicle includes a target vehicle, and the vehicle control identifier includes a target vehicle control identifier configured to indicate the real-time vehicle control mode of the vehicle relative to the target vehicle; determining whether the vehicle control mode in the voice command matches the vehicle control mode indicated by the vehicle control identifier, and obtaining a determination result, includes: performing semantic parsing on the voice command to obtain the vehicle control mode contained in the command content of the voice command; determining whether the vehicle control mode contained in the command content matches the vehicle control mode indicated by the target vehicle control identifier; in response to the vehicle control mode contained in the command content matching the vehicle control mode indicated by the target vehicle control identifier, obtaining a matching result as the determination result; and controlling whether the vehicle drives relative to the other vehicle according to the voice command based on the determination result and the real-time vehicle information includes: controlling the vehicle to drive relative to the target vehicle according to the vehicle control mode in the matching result based on the real-time vehicle information.

[0009] In some embodiments of this application, the other vehicle includes a target vehicle, the vehicle control identifier includes a target vehicle control identifier configured to indicate a real-time vehicle control mode for the vehicle to lock and follow the target vehicle, the vehicle control mode in the voice command includes a lock-follow mode, and the judgment result is a matching result that matches the lock-follow mode; the step of controlling whether the vehicle moves relative to the other vehicle according to the voice command based on the judgment result and the real-time vehicle information includes: controlling the vehicle to accelerate, decelerate, or change lanes according to the lock-follow mode for the target vehicle based on the real-time vehicle information.

[0010] In some embodiments of this application, controlling the vehicle to accelerate, decelerate, or change lanes to follow the target vehicle according to the locked following method based on the real-time vehicle information includes: when the vehicle identifier includes a cut-in vehicle identifier configured to indicate other vehicles entering between the vehicle and the target vehicle, searching for the target vehicle from the surrounding vehicles based on the vehicle attributes of the target vehicle; in response to finding the target vehicle, continuing to accelerate, decelerate, or change lanes to follow the target vehicle; and in response to not finding the target vehicle, notifying the user that locking the following of the target vehicle has failed.

[0011] In some embodiments of this application, the other vehicle includes a target vehicle, the vehicle control identifier includes a target vehicle control identifier configured to instruct the self-vehicle to follow and / or overtake the target vehicle in a real-time vehicle control mode, the vehicle control mode in the voice command includes a following mode, and the judgment result is a matching result that matches the following mode; the step of controlling whether the self-vehicle drives relative to the other vehicle according to the voice command based on the judgment result and the real-time vehicle information includes: controlling the self-vehicle to accelerate or decelerate while following the target vehicle in the following mode according to the real-time vehicle information.

[0012] In some embodiments of this application, controlling the vehicle to accelerate or decelerate in accordance with the following mode based on the real-time vehicle information includes: when the vehicle identifier includes a cut-in vehicle identifier configured to indicate other vehicles entering between the vehicle and the target vehicle, setting the other vehicle as the new target vehicle, and associating the target vehicle control identifier with the new target vehicle; controlling the vehicle to accelerate or decelerate in accordance with the following mode to follow the new target vehicle.

[0013] In some embodiments of this application, the other vehicle includes a target vehicle, the vehicle control identifier includes a target vehicle control identifier configured to instruct the vehicle to follow and / or overtake the target vehicle in a real-time vehicle control mode, the vehicle control mode in the voice command includes an overtaking mode, and the judgment result is a matching result that matches the overtaking mode; the step of controlling whether the vehicle moves relative to the other vehicle according to the voice command based on the judgment result and the real-time vehicle information includes: controlling the vehicle to accelerate according to the overtaking mode based on the real-time vehicle information until it overtakes the target vehicle.

[0014] In some embodiments of this application, before determining whether the vehicle control method in the voice command matches the vehicle control method indicated by the vehicle control identifier, the vehicle control method further includes: determining whether the voice command conforms to a safe intelligent driving strategy; in response to the voice command conforming to a safe intelligent driving strategy, continuing to execute subsequent steps; in response to the voice command not conforming to a safe intelligent driving strategy, refusing to execute the voice command and generating a violation notification; wherein the violation notification is configured to notify the user that the voice command violates a rule; and / or controlling whether the vehicle drives relative to the other vehicle according to the voice command based on the determination result and the real-time vehicle information includes: in response to the real-time vehicle information conforming to a safe intelligent driving strategy, controlling whether the vehicle drives relative to the other vehicle according to the voice command based on the determination result.

[0015] In some embodiments of this application, before receiving voice commands sent by the user to the electronic image in real time, the vehicle control method further includes: generating operation guidance based on the real-time vehicle information and a preset intelligent driving strategy; wherein the operation guidance is configured to guide the user to send currently supported voice commands; displaying the operation guidance on the intelligent driving perception screen, and / or broadcasting the operation guidance by voice.

[0016] In some embodiments of this application, after receiving the voice command sent by the user for the electronic image in real time, the vehicle control method further includes: according to at least one layer of safety judgment mechanism, if it is determined that the voice command is unreasonable, receiving the next command adjacent to the voice command and overwriting the voice command with the next command; the at least one layer of safety judgment mechanism is configured to indicate that the vehicle is driven safely after determining that the voice command is reasonable; and, if it is determined that the voice command is reasonable, allowing the vehicle to drive according to the voice command.

[0017] In some embodiments of this application, the at least one layer of security judgment mechanism includes a user security judgment mechanism. The step of receiving the next instruction adjacent to the voice instruction and overwriting the voice instruction when the voice instruction is determined to be unreasonable, based on the at least one layer of security judgment mechanism, includes: receiving the next instruction re-inputted by the user when the voice instruction is determined to be unreasonable, and overwriting the voice instruction; wherein the user security judgment mechanism is configured to indicate an instruction that has been corrected as unreasonable by the user.

[0018] In some embodiments of this application, the at least one layer of safety judgment mechanism further includes a driver safety judgment mechanism. The step of receiving a next instruction adjacent to the voice instruction and overwriting the voice instruction when the voice instruction is determined to be unreasonable, based on the at least one layer of safety judgment mechanism, includes: receiving the next instruction re-inputted by the driver when the voice instruction is determined to be unreasonable, and overwriting the voice instruction; wherein the driver safety judgment mechanism is configured to indicate that the driver assists the user in correcting the unreasonable instruction.

[0019] In some embodiments of this application, the step of receiving the next instruction re-inputted by the driver to overwrite the voice instruction when it is determined that the voice instruction is unreasonable includes: receiving a disabling instruction re-inputted by the driver to overwrite the voice instruction when it is determined that the abnormal frequency of the voice instruction is greater than a first frequency threshold; wherein the disabling instruction is configured to disable the control soft switch, and the disabling instruction is input through physical button operation.

[0020] In some embodiments of this application, the at least one layer of safety judgment mechanism further includes an intelligent driving system safety judgment mechanism. The step of receiving a next instruction adjacent to the voice instruction and overwriting the voice instruction when it is determined that the voice instruction is unreasonable, according to the at least one layer of safety judgment mechanism, includes: analyzing the rationality and risk of the voice instruction according to the intelligent driving system safety judgment mechanism to determine whether the voice instruction is reasonable; if it is determined that the voice instruction is unreasonable and exceeds the range that the intelligent driving system can handle, generating a request instruction for requesting manual intervention from the driver to overwrite the voice instruction; wherein, the intelligent driving system safety judgment mechanism is configured to indicate the rationality and risk of the intelligent driving system's judgment instruction. And / or, the at least one layer of security judgment mechanism further includes a background security monitoring mechanism. The step of receiving the next instruction adjacent to the voice instruction and overwriting the voice instruction when the voice instruction is determined to be unreasonable according to the at least one layer of security judgment mechanism includes: receiving a deactivation instruction from the background when the background security monitoring mechanism detects that the frequency of disabling exceeds a second frequency threshold, and overwriting the voice instruction with the deactivation instruction, so that the vehicle owner account can apply for relearning and then reactivate; wherein the deactivation instruction is configured to indicate the termination of the intelligent driving function.

[0021] Secondly, embodiments of this application provide a vehicle control device, comprising: an acquisition module configured to acquire electronic images of an intelligent driving perception screen in real time; the electronic images are configured to display a vehicle identifier and a vehicle control identifier, the vehicle identifier being configured to indicate real-time vehicle information, and the vehicle control identifier being configured to indicate the real-time vehicle control mode of the vehicle relative to another vehicle; a receiving module configured to receive voice commands sent by a user to the electronic images in real time; a judgment module configured to judge whether the vehicle control mode in the voice command matches the vehicle control mode indicated by the vehicle control identifier, and obtain a judgment result; and a control module configured to control whether the vehicle drives relative to the other vehicle according to the voice command based on the judgment result and the real-time vehicle information.

[0022] Thirdly, embodiments of this application provide an electronic device including one or more processors configured to implement a vehicle control method as described in any one of the first aspects.

[0023] Fourthly, embodiments of this application provide a computer program product, including a computer program stored on a computer-readable storage medium, wherein when the computer program is executed by at least one processor, it implements the vehicle control method as described in any one of the first aspects.

[0024] Fifthly, embodiments of this application provide a non-transitory computer-readable storage medium storing computer-executable instructions, which, when executed by at least one processor, implement the vehicle control method as described in any one of the first aspects.

[0025] In a sixth aspect, embodiments of this application provide a vehicle, including: a vehicle control device as described in the second aspect; or an electronic device as described in the third aspect; or a computer program product as described in the fourth aspect; or a non-transitory computer-readable storage medium as described in the fifth aspect; or one or more processors configured to implement the vehicle control method as described in any one of the first aspects.

[0026] In a seventh aspect, embodiments of this application also provide a vehicle safety control method, comprising: acquiring electronic images of intelligent driving perception scenes in real time; wherein the electronic images are used to display identification information of the vehicle and identification information of the surrounding environment; receiving current instructions input by a user regarding the identification information of the vehicle and the identification information of the surrounding environment in real time; according to at least one layer of safety judgment mechanism, if it is determined that the current instruction is unreasonable, receiving the next instruction adjacent to the current instruction and overwriting the current instruction with the next instruction; the safety judgment mechanism is configured to indicate that the vehicle is driven safely after determining that the current instruction is reasonable; and, if it is determined that the current instruction is reasonable, allowing the vehicle to drive according to the current instruction.

[0027] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Other aspects will become clear after reading and understanding the accompanying drawings and detailed description. Attached Figure Description

[0028] The accompanying drawings are included to provide a further understanding of the technical solutions of this application, are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application, and are used together with the embodiments to explain the principles of this application, but do not constitute a limitation on the technical solutions of this application.

[0029] Figure 1 is a schematic flowchart of a vehicle control method according to an embodiment of this application.

[0030] Figure 2 is a schematic diagram of an electronic image according to an embodiment of this application.

[0031] Figure 3 shows another flowchart of the vehicle control method according to an embodiment of this application.

[0032] Figure 4 is a schematic diagram of the operation instructions for an embodiment of this application.

[0033] Figure 5 is a flowchart illustrating the vehicle safety control method according to an embodiment of this application.

[0034] Figure 6 is a schematic diagram of the safety judgment mechanism of the vehicle safety control method shown in Figure 5.

[0035] Figure 7 shows a schematic diagram of the soft switch of the vehicle safety control method shown in Figure 5.

[0036] Figure 8 shows a schematic diagram of the physical button used to generate a disable command in the vehicle safety control method shown in Figure 5.

[0037] Figure 9 shows a structural block diagram of the vehicle control device according to an embodiment of this application.

[0038] Figure 10 shows a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0039] The technical solutions in the embodiments of this application will be clearly and completely described herein with reference to the accompanying drawings. In the following description, when referring to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements.

[0040] The embodiments described below are not representative of all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims. It should be noted that the steps of the corresponding methods in other embodiments are not necessarily performed in the order shown and described in this specification. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0041] In an alternative embodiment, in a vehicle with autonomous driving capabilities, the advanced driver assistance system typically controls the vehicle to drive according to a preset driving mode, making it impossible for the user to control the vehicle to drive as they wish.

[0042] To address the technical problem that users cannot control their vehicles to drive according to their wishes during autonomous driving, this application provides a vehicle control method. This method receives voice commands from the user in real time, targeting electronic images of the intelligent driving perception screen. When the vehicle control method in the voice command matches the vehicle control method indicated by the vehicle control icon on the electronic image, the method controls the vehicle to drive relative to other vehicles according to the voice command. This provides a voice interaction method between the user and the intelligent driving system, allowing the user to request the intelligent driving system to control their vehicle to drive relative to other vehicles according to their wishes through voice interaction.

[0043] Furthermore, if the driving modes preset by the advanced driver assistance system (ADAS) do not match the user's driving habits, and the user wants to drive the vehicle according to their own preferences, the user needs to take over control of the vehicle from the ADAS, relinquishing control and allowing the user to drive as the driver. In other words, while the vehicle is in motion, it must either be entirely controlled by the ADAS or entirely driven by the user. This results in inflexible control methods for both manual and autonomous driving, and a poor user experience for autonomous driving. For example, when the ADAS is controlling the vehicle, if the user wants to overtake or follow another vehicle, the ADAS cannot determine whether the user intends to overtake or follow, or which vehicle is being overtaken or followed.

[0044] To address the technical problem of inflexible control methods in both manual and autonomous driving, this application provides a vehicle control method. Before controlling the vehicle to move relative to other vehicles according to voice commands, the method also comprehensively considers real-time vehicle information indicated by vehicle identifiers on electronic images. The user sends voice commands, and the intelligent driving system receives them in real time. This interaction between the user and the intelligent driving system makes the control methods for both manual and autonomous driving more flexible. Furthermore, it enhances the personalization of the intelligent driving system while ensuring driving safety, thus improving the user experience of autonomous driving.

[0045] The vehicle control method provided in this application can be applied to vehicles equipped with displays, and especially to intelligent driving systems. Please refer to Figure 1, which is a schematic flowchart of a vehicle control method according to an embodiment of this application. The vehicle control method provided in this application includes, but is not limited to, the following steps S101 to S104.

[0046] In step S101, an electronic image of the intelligent driving perception screen is acquired in real time; wherein, the electronic image is used to display vehicle identification and vehicle control identification, the vehicle identification is used to indicate real-time vehicle information, and the vehicle control identification is used to indicate the real-time vehicle control mode of the vehicle relative to other vehicles.

[0047] Intelligent driving systems can use various sensors installed on the vehicle (such as millimeter-wave radar, lidar, mono / dual-lens cameras, and satellite navigation) to perceive the vehicle's surroundings in real time while driving, acquiring real-time perception data. Based on this data, they can identify, detect, and track static and dynamic objects. To display the vehicle's surroundings to the user and facilitate real-time awareness of this information, the intelligent driving system can generate and display electronic images of the vehicle's surroundings on the Advanced Driver Assistance System View (ADV).

[0048] The electronic image may include vehicle identification and vehicle control identification. In this embodiment, the vehicle identification is the identification corresponding to vehicles in the surrounding environment of the vehicle, and may include the vehicle identification corresponding to the vehicle itself and the identification corresponding to other vehicles, used to indicate real-time vehicle information in the surrounding environment; the vehicle control identification is the identification corresponding to the vehicle control method of the vehicle relative to other vehicles in the surrounding environment of the vehicle, used to indicate the real-time vehicle control method of the vehicle relative to other vehicles.

[0049] In some embodiments, the electronic image may further include lane markings and lane control markings. In this embodiment, lane markings are markings corresponding to lanes in the vehicle's surrounding environment, and may include markings corresponding to the vehicle's own lane and markings corresponding to other lanes where the vehicle is not located, used to indicate real-time lane information in the surrounding environment; lane control markings are markings corresponding to the lane control method of the vehicle relative to lanes in the surrounding environment, used to indicate the real-time lane control method of the vehicle relative to lanes. This embodiment does not limit the specific types of sensed objects in the vehicle's surrounding environment included in the electronic image.

[0050] Please refer to Figure 2, which is a schematic diagram of an electronic image according to an embodiment of this application. The electronic image includes a vehicle identifier, a other vehicle identifier, a lane identifier, a vehicle control identifier, and a lane control identifier. The lane identifier may include a lane graphic and / or a lane number, and the other vehicle identifier may include a other vehicle graphic and / or a other vehicle number. The electronic image shown in Figure 2 will be described in detail below.

[0051] Lane markings include lane graphics and lane numbers ①, ②, ③, and ④. A solid line and a dashed line form the lane graphic for the rightmost lane (lane ①) traveling in the same direction as the vehicle. Two dashed lines form the lane graphic for the middle lane (lane ②) traveling in the same direction as the vehicle. A dashed line and a solid line form the lane graphic for the leftmost lane (lane ③) traveling in the same direction as the vehicle. A dashed line and a solid line form the lane graphic for the leftmost lane (lane ④) traveling in the opposite direction to the vehicle.

[0052] The other vehicle identification includes a graphic of the other vehicle and its number, A, B, C, or D. Vehicles A, B, and C are traveling in the same direction as your vehicle, respectively, in lanes ①, ②, and ③. Vehicle D is traveling in the opposite direction to your vehicle, in lane ④.

[0053] Lane control signs include the first letters of the Chinese pinyin for the real-time lane control mode: B, Y, and S. B indicates lane changing, Y indicates priority lane driving, and S indicates lane locking. If the vehicle can change lanes to lane 1 or 2 and maintain priority or lock in lane 1 or 2, the lane control signs corresponding to lane 1 and 2 will be B, Y, and S, representing lane changing, priority lane driving, and lane locking, respectively. If the vehicle is currently in lane 3 and cannot change lanes to lane 3 but can maintain priority or lock in lane 3, the lane control signs corresponding to lane 3 will be Y and S, representing priority lane driving and lane locking, respectively. If the vehicle cannot change lanes to lane 4 or maintain priority or lock in lane 4, the lane control sign corresponding to lane 4 will be empty.

[0054] The vehicle control indicators include the first letters of the Chinese pinyin for the real-time vehicle control mode: C, G, and S. C indicates overtaking, G indicates priority following, and S indicates locked following. If the vehicle cannot overtake, follow, or lock-follow vehicle A or D, the vehicle control indicator corresponding to the other vehicle indicator for A and D will be empty. If the vehicle can follow or lock-follow vehicle B, the vehicle control indicator corresponding to the other vehicle indicator for B will be G and S, representing following and locked following, respectively. If the vehicle can overtake, follow, or lock-follow vehicle C, the vehicle control indicator corresponding to the other vehicle indicator for C will be C, G, and S, representing overtaking, following, and locked following, respectively.

[0055] It should be noted that the display methods of lane signs and lane control signs, and the display methods of other vehicle signs and vehicle control signs, are not limited to those shown in Figure 2. Each lane control sign is not necessarily displayed above or around its corresponding lane number, and each vehicle control sign is not necessarily displayed below or around its corresponding other vehicle sign. It is sufficient to establish and display a connection between each lane control sign and its corresponding lane sign, and between each vehicle control sign and its corresponding other vehicle sign. For example, different lane signs and other vehicle signs can be set to different colors, and each lane control sign and vehicle control sign can be set to the same color as its corresponding lane sign and other vehicle sign. This application embodiment does not limit the specific display methods of lane signs and lane control signs, and the specific display methods of other vehicle signs and vehicle control signs.

[0056] It should also be noted that in the electronic image shown in Figure 2, the vehicle's graphic is black, the graphics of other vehicles (B and C) are white, the graphics of other vehicles (A and D) are gray, the lane graphics of lanes ①, ②, and ③ are white, and the lane graphic of lane ④ is gray (represented by diagonally filled lines in the figure). A white other vehicle graphic indicates that the current vehicle can perform at least one vehicle control mode (such as overtaking or following) relative to the other vehicle represented by that graphic, while a gray other vehicle graphic indicates that the current vehicle cannot perform any vehicle control mode relative to the other vehicle represented by that graphic. Similarly, a white lane graphic indicates that the current vehicle can perform at least one lane control mode (such as changing lanes) relative to the lane represented by that lane graphic, while a gray lane graphic indicates that the current vehicle cannot perform any lane control mode relative to the lane represented by that lane graphic. In specific scenarios, the colors of the other vehicle graphics corresponding to different other vehicles and the lane graphics corresponding to different lanes can be specifically set, and the colors of the other vehicle numbers corresponding to different other vehicles and the lane numbers corresponding to different lanes can also be different accordingly. For example, the vehicle identification mark can be set to the vehicle's actual color, the other vehicle identification mark (which can be controlled by at least one vehicle control method) can be set to green, and the other vehicle identification mark (which cannot be controlled by any vehicle control method) can be set to red. This application embodiment does not limit the specific colors of different lane markings, different other vehicle identification marks, and the vehicle identification mark.

[0057] In step S102, voice commands sent by the user for the electronic image are received in real time.

[0058] Because users may want to control the vehicle differently depending on the road conditions, and they may want to cancel the voice command they sent one second later, it is necessary to receive the voice commands sent by the user to the electronic image in real time to meet the user's real-time needs.

[0059] Receiving voice commands from users can be achieved through the vehicle's audio recording equipment (such as a microphone). For specific implementation details, please refer to relevant documentation in the field; these will not be elaborated upon here.

[0060] In step S103, it is determined whether the vehicle control mode in the voice command matches the vehicle control mode indicated by the vehicle control icon, and the determination result is obtained.

[0061] Because the vehicle control method in the voice command sent by the user may not be consistent with the vehicle control method indicated by the vehicle control sign, it is necessary to determine whether the vehicle control method in the voice command matches the vehicle control method indicated by the vehicle control sign before executing the voice command sent by the user, and obtain the determination result.

[0062] For example, in the electronic image shown in Figure 2, the vehicle control icons corresponding to the vehicle B's other vehicle icon are G and S, representing following and locked following, indicating that the user's vehicle can currently follow or lock onto following vehicle B. If the user sends a voice command to follow vehicle B, the vehicle control method in the voice command matches the vehicle control method indicated by the vehicle control icon, and the result is a match result; if the user sends a voice command to overtake vehicle B, the vehicle control method in the voice command does not match the vehicle control method indicated by the vehicle control icon, and the result is a mismatch result.

[0063] In step S104, based on the judgment result and the real-time vehicle information, the system controls whether the vehicle moves relative to other vehicles according to the voice command.

[0064] Even if the judgment result obtained in step S103 is a matching result, the intelligent driving system will not control the vehicle to drive according to the voice command if the real-time vehicle information does not allow it. In other words, regardless of whether the judgment result obtained in step S103 is a matching result or a non-matching result, the intelligent driving system must also determine whether the real-time road conditions are safe before deciding whether to control the vehicle to drive according to the voice command. For example, in the electronic image shown in Figure 2, if the user sends a voice command to overtake vehicle C, although the control method in the voice command matches the control method indicated by the control icon, and the judgment result is a matching result, if vehicle B slows down at this time, and another vehicle is approaching from behind vehicle B in lane ②, then the real-time road conditions are not safe, and the vehicle cannot be immediately controlled to overtake vehicle C. Only when the judgment result obtained in step S103 is a matching result and the real-time road conditions are safe will the intelligent driving system control the vehicle to drive according to the voice command. Specifically, the intelligent driving system can determine whether the real-time road conditions are safe based on real-time vehicle information. For details on how to determine whether the real-time road conditions are safe based on real-time vehicle information, please refer to the relevant descriptions in this field, which will not be elaborated here.

[0065] In some embodiments, the intelligent driving system may wait until the real-time road conditions are safe before controlling the vehicle to drive according to the voice command. In other embodiments, the intelligent driving system may also discard the currently received voice command and wait for the user to resend the voice command before executing the above steps S103 and S104. This application does not limit the specific processing method of voice commands when real-time road conditions are unsafe.

[0066] In the vehicle control method provided in this application embodiment, after receiving the voice command sent by the user to the electronic image in real time and before controlling the vehicle to drive according to the voice command, the safety of the real-time road conditions indicated by the real-time vehicle information is also comprehensively considered. This improves the personalization of the intelligent driving system while ensuring driving safety, thereby further improving the user experience of autonomous driving.

[0067] Before controlling the vehicle to move relative to other vehicles according to voice commands, the intelligent driving system needs to identify the target vehicle from among the other vehicles in order to control the vehicle to move relative to the target vehicle according to voice commands. Based on this, the vehicle control method of this application embodiment can be implemented in at least the following two ways.

[0068] In some embodiments, real-time vehicle information includes only one other vehicle. In this case, if only one other vehicle is traveling in front of the user, then only that one other vehicle can be identified as the target vehicle. When the user sends a voice command, there is no need to specify a specific vehicle; the vehicle control method in the voice command only includes the specific vehicle control method. The intelligent driving system will then identify the only other vehicle in the real-time vehicle information as the target vehicle and directly determine whether the vehicle control method in the user's voice command matches the vehicle control method indicated by the vehicle control identifier corresponding to that single other vehicle.

[0069] In other embodiments, real-time vehicle information includes multiple other vehicles. In this case, if there is more than one other vehicle driving in front of the user, it is necessary to identify the target vehicle from among the multiple other vehicles in front. When sending a voice command, the user needs to specify a specific vehicle, so that the vehicle control method in the voice command includes information about the specific vehicle and the specific vehicle control method relative to that specific vehicle. When the intelligent driving system determines whether the vehicle control method in the voice command matches the vehicle control method indicated by the vehicle control identifier, it specifically includes the following steps: based on the information of the specific vehicle, determining whether the real-time vehicle information includes vehicle information that matches the information of the specific vehicle; in response to the real-time vehicle information including vehicle information that matches the information of the specific vehicle, identifying the other vehicle corresponding to the vehicle information that matches the information of the specific vehicle as the target vehicle; and determining whether the specific vehicle control method matches the vehicle control method indicated by the vehicle control identifier corresponding to the target vehicle.

[0070] Please refer to Figures 1 to 3 together. Figure 3 shows another schematic flowchart of the vehicle control method according to an embodiment of this application.

[0071] In some embodiments, before receiving the voice command sent by the user for the electronic image in real time in step S102 above, the vehicle control method of this application embodiment further includes the following steps S201 and S202.

[0072] In step S201, an operation guide is generated based on the real-time vehicle information and the preset intelligent driving strategy; wherein, the operation guide is used to guide the user to send currently supported voice commands.

[0073] Based on real-time vehicle information and preset intelligent driving strategies, the intelligent driving system determines the vehicle control methods that can be performed relative to other vehicles in the surrounding environment, and generates operation instructions. These instructions guide the user to send currently supported voice commands, ensuring the user knows which voice commands are supported and can be sent beforehand. This reduces the likelihood of users being informed that a voice command cannot be executed, reduces user steps, improves the user experience, and conserves vehicle network and power resources.

[0074] In step S202, the operation instructions are displayed on the intelligent driving perception screen and / or broadcast via voice.

[0075] Please refer to Figure 4, which is a schematic diagram of the operation instructions according to an embodiment of this application. The intelligent driving system can display the operation instructions in text form on the intelligent driving perception screen, or it can broadcast the operation instructions in voice form inside the vehicle without displaying them on the intelligent driving perception screen. Alternatively, it can display the operation instructions in text form on the intelligent driving perception screen while simultaneously broadcasting them in voice form inside the vehicle, or it can present the operation instructions to the user in other forms. This application embodiment does not limit the specific presentation method of the operation instructions.

[0076] In other embodiments, the intelligent driving system may not display operation instructions on the intelligent driving perception screen. That is, the electronic image of the intelligent driving perception screen does not contain operation instructions, the intelligent driving system does not generate operation instructions, directly receives voice commands sent by the user to the vehicle control icon in the electronic image, and executes subsequent steps.

[0077] In some embodiments, before determining whether the vehicle control mode in the voice command matches the vehicle control mode indicated by the vehicle control identifier in step S103 above, and obtaining the determination result, the vehicle control method of this application embodiment further includes the following steps S203 to S205.

[0078] In step S203, it is determined whether the voice command conforms to a safe intelligent driving strategy.

[0079] Because users sending voice commands to intelligent driving systems may not be fully familiar with traffic rules or may not be in a good mental state when sending them, the voice commands received by the system may not conform to safe intelligent driving strategies. Therefore, before determining whether the vehicle control method in the voice command matches the vehicle control method indicated by the vehicle control sign, it is also necessary to determine whether the voice command conforms to the safe intelligent driving strategy. For example, when a vehicle approaches an intersection, according to the intelligent driving strategy, the system should control the vehicle to slow down. However, if the user sends a voice command at this time requesting the vehicle to accelerate, then the user's voice command does not conform to the safe intelligent driving strategy.

[0080] In step S204, if the voice command does not conform to the intelligent driving strategy, the voice command is refused to be executed and a violation notification is generated; wherein, the violation notification is used to notify the user that the voice command is in violation.

[0081] When the voice command received by the intelligent driving system does not conform to the safe intelligent driving strategy, the intelligent driving system refuses to execute the voice command sent by the user and generates a violation notification to inform the user that the voice command has violated the rules. Through the settings of steps S203 and S204, safe driving is ensured on the one hand, avoiding the safety hazards caused by human intervention to autonomous driving; on the other hand, it allows users to better understand the intelligent driving strategy and the interaction method between the user and the intelligent driving system, improving the user's subsequent user experience.

[0082] In step S205, if the voice command conforms to the intelligent driving strategy, the subsequent steps are executed.

[0083] In addition to the above-mentioned methods for determining the safety of commands, this application also provides a vehicle safety control method to determine the safety of commands issued by the user. The following will describe the method in detail with reference to Figures 5 to 8, but will not be elaborated here.

[0084] In some embodiments, when controlling whether the vehicle drives relative to another vehicle according to the voice command based on the judgment result and the real-time vehicle information in step S104 above, the specific steps include: if the real-time vehicle information conforms to a safe intelligent driving strategy, then controlling whether the vehicle drives relative to another vehicle according to the voice command based on the judgment result.

[0085] If the real-time vehicle information conforms to the safe intelligent driving strategy, then only the judgment result in step S103 needs to be considered. Based on whether the vehicle control method in the voice command matches the vehicle control method indicated by the vehicle control icon, it is decided whether to control the vehicle to drive relative to other vehicles according to the voice command.

[0086] In some embodiments, the other vehicle in step S101 includes the target vehicle, and the vehicle control identifier includes a target vehicle control identifier for indicating the real-time vehicle control mode of the vehicle relative to the target vehicle; when determining whether the vehicle control mode in the voice command matches the vehicle control mode indicated by the vehicle control identifier in step S103, and obtaining the determination result, the steps S111 to S113 are specifically included; when controlling whether the vehicle drives relative to the other vehicle according to the voice command based on the determination result and the real-time vehicle information in step S104, the step S114 is specifically included.

[0087] In step S111, the voice command is semantically parsed to obtain the vehicle control method contained in the command content.

[0088] Although intelligent driving systems generate operation guidelines to guide users in sending currently supported voice commands, different users have different language habits, and the same vehicle control method can be expressed in multiple ways. For example, regarding the vehicle control method of overtaking relative to car A, some users are used to saying "overtake car A," while others are used to saying "drive in front of car A." Therefore, when determining whether the vehicle control method in the voice command matches the vehicle control method indicated by the vehicle control icon, it is necessary to first perform semantic analysis on the voice command. For example, using artificial intelligence tools, the words and sentences in the voice command can be semantically generalized to understand the command content of voice commands expressed in different ways, thereby obtaining the vehicle control method contained in the command content.

[0089] By semantically parsing the voice commands sent by users, the intelligent driving system can adapt to users with different language habits. At the same time, it can also make the intelligent driving system understand the voice commands sent by users more accurately, thereby improving the efficiency of voice interaction between users and the intelligent driving system and enhancing the user experience of autonomous driving.

[0090] In step S112, it is determined whether the vehicle control method in the instruction content matches the vehicle control method indicated by the target vehicle control identifier.

[0091] S113, in response to the vehicle control mode in the instruction content matching the vehicle control mode indicated by the target vehicle control identifier, the matching result is used as the judgment result.

[0092] If the vehicle control method in the voice command matches the vehicle control method indicated by the target vehicle control icon, the matching result is used as the judgment result; otherwise, the non-matching result is used as the judgment result.

[0093] In step S114, based on the real-time vehicle information, the vehicle is controlled to drive relative to the target vehicle according to the vehicle control method in the matching result.

[0094] If the judgment result in step S113 is a matching result, then based on the real-time vehicle information, the vehicle is controlled to drive relative to the target vehicle according to the vehicle control method in the matching result; if the judgment result in step S113 is a non-matching result, then based on the real-time vehicle information, the vehicle is controlled to maintain the current state of driving.

[0095] In some embodiments, the other vehicle in step S101 includes the target vehicle, and the vehicle control identifier includes a target vehicle control identifier for indicating the vehicle control mode of locking and following the target vehicle; the vehicle control mode in step S102 is the lock-follow mode; the judgment result obtained in step S103 is a matching result that matches the lock-follow mode; when controlling whether the vehicle drives relative to the other vehicle according to the voice command based on the judgment result and the real-time vehicle information in step S104, the following step S124 is specifically included.

[0096] In step S124, based on the real-time vehicle information, the vehicle is controlled to follow the target vehicle in the locked following mode, accelerating, decelerating, or changing lanes.

[0097] If the vehicle control mode in the voice command sent by the user is locked following mode, and the judgment result matches the locked following mode, the intelligent driving system, based on real-time vehicle information, controls the vehicle to follow the target vehicle in locked following mode. Specifically: the system controls the vehicle to travel to a preset following distance behind the target vehicle and maintains that distance; if the target vehicle is in its own lane, the system controls the vehicle to accelerate to the preset following distance behind the target vehicle; if the target vehicle is not in its own lane, the system controls the vehicle to change lanes to the target vehicle's lane and accelerate to the preset following distance behind the target vehicle; when the system detects the target vehicle accelerating or decelerating, the system controls the vehicle to accelerate or decelerate accordingly; when the system detects the target vehicle changing lanes, the system controls the vehicle to change lanes accordingly.

[0098] In this embodiment, although the following distance is preset, it is not a fixed value. To ensure that the vehicle will not rear-end the target vehicle after braking in an emergency, the following distance is a dynamically changing value related to the vehicle's real-time speed. Furthermore, different road surfaces have different coefficients of friction with the vehicle's tires, resulting in varying distances the vehicle slides after braking. Therefore, the following distance is related not only to the real-time speed but also to the road conditions.

[0099] In some embodiments, when controlling the vehicle to accelerate, decelerate, or change lanes according to the real-time vehicle information in step S124 above, the following steps are specifically included: when the vehicle identifier includes an identifier for other vehicles cutting in between the vehicle and the target vehicle, the target vehicle is searched for from the surrounding vehicles based on the vehicle attributes of the target vehicle; if the target vehicle is found, the vehicle continues to accelerate, decelerate, or change lanes according to the target vehicle; if the target vehicle is not found, the user is notified that the lock-following of the target vehicle has failed.

[0100] Because the "cutting in" vehicle indicator is used to indicate other vehicles entering between the vehicle and the target vehicle, when the vehicle indicator includes the "cutting in" vehicle indicator, it means that another vehicle has cut in between the vehicle and the target vehicle during the tracking process. To avoid losing track of the target vehicle, it is necessary to search for the target vehicle among the surrounding vehicles based on its vehicle attributes. If the target vehicle is found within a preset time period, the tracking continues; otherwise, the user is notified that the tracking failed. In this embodiment, vehicle attributes may include standard attributes of the vehicle at the time of manufacture (such as brand, logo, model, color, partial outline, and overall vehicle outline) and / or non-standard attributes after the vehicle leaves the factory (such as license plate number, personalized decorations affixed to the vehicle body, etc.).

[0101] In some embodiments, a database can be pre-established to store the correspondence between all standard attributes of different vehicles, such as brand, logo, model, color, partial outline, and overall vehicle outline, and the database can be stored in the cloud. After determining and locking onto the target vehicle, the intelligent driving system uploads some standard attributes of the target vehicle (such as color and rear outline) collected on-site to the database. After matching all the standard attributes of the target vehicle, it receives other standard attributes of the target vehicle (such as brand, logo, and overall vehicle outline) from the cloud to facilitate more accurate locking onto and following of the target vehicle, as well as searching for the target vehicle from surrounding vehicles in more dimensions after other vehicles cut in.

[0102] In some embodiments, the other vehicle in step S101 includes the target vehicle, and the vehicle control identifier includes a target vehicle control identifier for indicating the vehicle control mode of following and / or overtaking the target vehicle; the vehicle control mode in step S102 is the following mode; the judgment result obtained in step S103 is the matching result that matches the following mode; when controlling whether the vehicle moves relative to the other vehicle according to the voice command based on the judgment result and the real-time vehicle information in step S104, the following step S134 is specifically included.

[0103] In step S134, based on the real-time vehicle information, the vehicle is controlled to accelerate or decelerate in accordance with the following method to follow the target vehicle.

[0104] If the vehicle control mode in the voice command sent by the user is "follow mode," and the judgment result matches the following mode, the intelligent driving system, based on real-time vehicle information, controls the vehicle to follow the target vehicle in the following mode. Specifically: the system controls the vehicle to travel to a preset following distance behind the target vehicle and maintains that distance; if the target vehicle is in its own lane, the system controls the vehicle to accelerate to the preset following distance behind the target vehicle; if the target vehicle is not in its own lane, the system controls the vehicle to change lanes to the target vehicle's lane and accelerate to the preset following distance behind the target vehicle; when the system detects the target vehicle accelerating or decelerating, the system controls the vehicle to accelerate or decelerate accordingly; when the system detects the target vehicle changing lanes, the system controls the vehicle to remain in its current lane.

[0105] In some embodiments, when the above step S134 controls the vehicle to accelerate or decelerate in accordance with the following mode based on the real-time vehicle information, it specifically includes the following steps: when the vehicle identifier includes a cut-in vehicle identifier for indicating other vehicles entering between the vehicle and the target vehicle, the other vehicle is set as the new target vehicle, and the target vehicle control identifier is associated with the updated target vehicle; the vehicle is controlled to accelerate or decelerate in accordance with the following mode to follow the updated target vehicle.

[0106] Because the lane-cutting vehicle indicator is used to indicate other vehicles entering between the vehicle and the target vehicle, when the vehicle indicator includes the lane-cutting vehicle indicator, it means that other vehicles have cut in between the vehicle and the target vehicle during the following process. In this case, abandon following the original target vehicle, disconnect the target vehicle control indicator from the original target vehicle, set the other vehicle that cut in between the vehicle and the original target vehicle as the new target vehicle, associate the target vehicle control indicator with the new target vehicle, and control the vehicle to follow the new target vehicle by accelerating or decelerating.

[0107] In some embodiments, the other vehicle in step S101 includes the target vehicle, and the vehicle control identifier includes a target vehicle control identifier for indicating the vehicle control mode of following and / or overtaking the target vehicle; the vehicle control mode in step S102 is the overtaking mode; the judgment result obtained in step S103 is a matching result that matches the overtaking mode; when controlling whether the vehicle moves relative to the other vehicle according to the voice command based on the judgment result and the real-time vehicle information in step S104, it specifically includes the following step S144.

[0108] In step S144, based on the real-time vehicle information, the vehicle is controlled to accelerate in accordance with the overtaking method until it overtakes the target vehicle.

[0109] If the user sends a voice command specifying overtaking as the vehicle control mode, and the system determines that the overtaking mode matches, the intelligent driving system, based on real-time vehicle information, controls the vehicle to follow the target vehicle using the overtaking mode. Specifically: it determines whether the target vehicle is in its own lane; if the target vehicle is in its own lane, it controls the vehicle to change lanes and accelerate until it overtakes the target vehicle; if the target vehicle is not in its own lane, it determines whether acceleration is permitted in the current lane; if acceleration is permitted in the current lane, it controls the vehicle to maintain its position in the lane and accelerate until it overtakes the target vehicle; if acceleration is not permitted in the current lane, it controls the vehicle to change lanes and accelerate until it overtakes the target vehicle.

[0110] It's important to note that when controlling the vehicle to accelerate to overtake a target vehicle traveling in another lane, acceleration is generally permitted in the vehicle's own lane. Therefore, the vehicle can maintain its acceleration within the lane until it overtakes the target vehicle. However, there are special cases where acceleration is not permitted in the vehicle's own lane. In such cases, the vehicle needs to change lanes and then accelerate until it overtakes the target vehicle. For example, if there is a slow-moving vehicle ahead in the vehicle's lane, or if the lane has a speed limit and the vehicle has already reached that limit, the intelligent driving system cannot control the vehicle to accelerate directly within its own lane.

[0111] In some embodiments, when controlling the vehicle to change lanes and accelerate to overtake a target vehicle in its lane, the intelligent driving system also needs to determine whether the voice command contains the overtaking direction and the requirement not to merge. If the voice command contains the overtaking direction and the requirement not to merge, the system controls the vehicle to change lanes in the overtaking direction and accelerate until it overtakes the target vehicle, then controls the vehicle to stay in the lane after the lane change. If the voice command contains the overtaking direction but does not contain the requirement not to merge, the system controls the vehicle to change lanes in the overtaking direction and accelerate until it overtakes the target vehicle, then controls the vehicle to change lanes again and return to its original lane. If the voice command does not contain the overtaking direction but does contain the requirement not to merge, the system controls the vehicle to change lanes according to the recorded user driving habits and accelerate until it overtakes the target vehicle, then controls the vehicle to stay in the lane after the lane change. If the voice command does not contain the overtaking direction and the requirement not to merge, the system controls the vehicle to change lanes according to the recorded user driving habits and accelerate until it overtakes the target vehicle, then controls the vehicle to change lanes again and return to its original lane.

[0112] This application also provides a vehicle safety control method to provide a mechanism for making safety judgments based on user-issued commands. Please refer to Figures 5 to 8. Figure 5 is a flowchart illustrating the vehicle safety control method of this application embodiment; Figure 6 is a schematic diagram of the safety judgment mechanism of the vehicle safety control method shown in Figure 5; Figure 7 is a schematic diagram of the soft switch in the vehicle safety control method shown in Figure 5; and Figure 8 is a schematic diagram of the physical button used to generate a disable command in the vehicle safety control method shown in Figure 5.

[0113] As shown in Figure 5, the vehicle safety control method may include, but is not limited to, steps 210 to 240.

[0114] In step 210, electronic images of the intelligent driving perception screen are acquired in real time.

[0115] In this embodiment, the electronic image is used to display vehicle identification information and surrounding environment identification information. The electronic image may include one or more of the vehicle identification and surrounding environment identification information. For example, the electronic image may display information about the actual distance of the vehicle relative to the surrounding environment; correspondingly, the display distance of the vehicle identification relative to the surrounding environment identification is displayed on the electronic image. Surrounding environment identification may include, but is not limited to, other vehicle identification and lane markings in the surrounding environment. Vehicle identification, other vehicle identification, and lane markings represent information indicating road conditions for the vehicle and the surrounding environment, respectively representing the actual distribution of the vehicle, other vehicles, and lanes in the actual environment. Thus, this identification information is used to describe the distribution of the actual environment, thereby reflecting the real-world situation.

[0116] The electronic images in this article can be displayed on the intelligent driving perception screen (ADV) of the HMI (Human-Machine Interface) to achieve human-machine interaction.

[0117] As described above, the electronic image may include, but is not limited to, vehicle identification, vehicle control identification, lane identification, and lane control identification. However, the embodiments of this application are not limited to this. For example, the electronic image may also include other control identification for controlling the vehicle, such as, but not limited to, a speed adjustment identification for indicating speed adjustment.

[0118] The following description uses a control identifier as an example to illustrate the method for generating the corresponding identifier. Specifically, the control identifier can be generated using steps A through C.

[0119] In step A, a vehicle icon is generated.

[0120] Because the vehicle's surroundings are constantly changing while it is in motion, and the sensing devices are mounted on the vehicle body, the location of objects in the surrounding environment can only be determined by using the vehicle itself as a reference. Therefore, it is necessary to first generate a vehicle identifier corresponding to the vehicle that is "stationary" relative to the sensing devices.

[0121] In step B, the relative position of the sensing object and the vehicle is determined based on real-time sensing data.

[0122] By collecting real-time sensing data from sensing devices, the position of objects in the surrounding environment relative to the vehicle can be determined; that is, the relative position of the objects to the vehicle. Within the sensing range of the sensing devices, there may be multiple objects in the surrounding environment, and each object has a relative position to the vehicle. For example, when the first object is in the vehicle's lane, the first object is directly below the vehicle; when the second object is a vehicle in front of the vehicle in the left or right adjacent lane, the second object is located to the left or right front of the vehicle.

[0123] In step C, with the vehicle identifier as a reference, an identifier for the perceived object is generated at the relative position of the vehicle identifier, and a control identifier is generated.

[0124] The vehicle sign is considered as the origin of the reference coordinate system. Using the vehicle sign as a reference, the sign of the perceived object is generated at the corresponding position relative to the vehicle sign, based on the relative position of the perceived object and the vehicle. For example, when the first perceived object (e.g., the vehicle lane) is directly below the vehicle, the sign of the first perceived object (the vehicle lane) is generated directly below the vehicle sign; when the third perceived object (e.g., a vehicle ahead in the vehicle lane) is directly in front of the vehicle, the sign of the second perceived object (the vehicle ahead in the vehicle lane) is generated directly in front of the vehicle sign.

[0125] In step 220, the current instructions input by the user for the electronic image (e.g., the vehicle's identification information and the identification information of the surrounding environment in the electronic image) are received in real time.

[0126] In step 230, based on at least one layer of security judgment mechanism, if it is determined that the current instruction is unreasonable, the next instruction adjacent to the current instruction is received and the next instruction overwrites the current instruction.

[0127] In the embodiments of this application, the next instruction in "overwrite the current instruction with the next instruction" can be a reasonable instruction, or it can be the instruction that is finally determined to be reasonable after a series of unreasonable instructions.

[0128] The at least one layer of security judgment mechanism in the embodiments of this application is used to prioritize security and ensure that all instructions are ultimately reasonable, thereby improving security.

[0129] The safety judgment mechanism in this paper is used to indicate whether the current instruction is reasonable, thus enabling safe intelligent driving of the vehicle.

[0130] "Unreasonable current instruction" indicates that the current instruction will affect driving safety. This unreasonable current instruction can include, but is not limited to, instructions for dangerous or malicious driving. An example of an unreasonable current instruction is one that is frequently issued within a short period. Another example is when the surrounding environment is surrounded by seawater on both sides without roads, but the current instruction is to turn left or right.

[0131] Next, the input method for the aforementioned current command can be via touch input or via voice input, and this application does not impose any restrictions on this.

[0132] In step 240, if the current instruction is determined to be reasonable, the vehicle is allowed to drive according to the current instruction. This enables intelligent driving by controlling the vehicle to follow the current instruction.

[0133] This article not only supports the driver (referred to as the primary driver) in judging whether the instruction is reasonable, but also supports the co-driver in judging whether the current instruction is reasonable. Please see below for details.

[0134] Continuing with Figures 5 and 6, the safety judgment mechanism of the vehicle safety control method may include, but is not limited to, one or more of the following: user safety judgment mechanism, driver safety judgment mechanism, intelligent driving system safety judgment mechanism, and background safety monitoring mechanism. In practical use, one or more of the above safety judgment mechanisms can be used. When using all safety judgment mechanisms, the user safety judgment mechanism, driver safety judgment mechanism, intelligent driving system safety judgment mechanism, and background safety monitoring mechanism can be performed sequentially, which will not be elaborated further here.

[0135] In this regard, for step 230 above, at least one optional embodiment can be used to implement the next instruction overwriting the current instruction.

[0136] In a first optional embodiment, based on the user security judgment mechanism within the security judgment mechanism, if it is determined that the current instruction is unreasonable, the next instruction re-entered by the user is received and overwritten. In this embodiment, the user security judgment mechanism is used to indicate an instruction that has been corrected as unreasonable by the user.

[0137] The aforementioned next instruction may include, but is not limited to, deceleration instructions and / or braking instructions. This can create a relatively safe operating environment for the vehicle. However, the embodiments of this application are not limited to this. For example, the aforementioned next instruction may also include, but is not limited to, instructions involving vehicle control methods such as priority following, locking following, or overtaking.

[0138] The user security assessment mechanism is as follows.

[0139] 1. After a user issues a command, if the intelligent driving system agrees to execute it, the user can cancel the execution of the command at any time during the process using the cancel button on the screen or a voice cancellation command. The user here can include the driver and other people besides the driver. For example, a front passenger issues a command. Another example is a rear-seat user issuing a command.

[0140] 2. After issuing a command, if the user finds the command unreasonable, they can override the previous command with another command, allowing the intelligent driving system to perform a more reasonable operation.

[0141] 3. If a risk is detected after the user issues a command, the user can use a deceleration command to immediately slow down the vehicle to avoid the risk.

[0142] 4. If a risk is detected after the user issues a command, the risk can be avoided by using the braking command.

[0143] In this embodiment, if a user finds that they have entered an incorrect current command, they can re-enter the next command to overwrite the unreasonable or incorrect command, thereby correcting the error command in a timely manner and improving the security and accuracy of the command.

[0144] In this embodiment, the driver and the intelligent driving system need to have the power, authority, and means to effectively disable this function. See below for details.

[0145] In an optional embodiment, in the human-machine co-driving mode, when a user controls the vehicle via screen touch or voice, there may be situations where the operation is not performed by the driver. Such situations, without the driver's consent, may pose safety risks. The vehicle safety control method provided in this application eliminates the risks associated with non-driver human-machine co-driving by allowing the driver to override non-driver operations through the screen and voice.

[0146] Referring again to Figures 5 and 6, in the second optional embodiment, based on the driver safety judgment mechanism in the safety judgment mechanism, if it is determined that the current instruction is unreasonable, the driver will receive the next instruction re-entered by the driver to overwrite the current instruction; wherein, the driver safety judgment mechanism is used to indicate that the driver is assisting the user in correcting unreasonable instructions.

[0147] It should be noted that the driver has the highest authority and can take over or disable touchscreen and voice control at any time. Therefore, even if the intelligent driving system errs in its perception, misses or misjudges the risk of a command, the driver can cancel, terminate, or take over.

[0148] The above-mentioned driver safety judgment mechanism is as follows.

[0149] 1. After the user issues a command, the driver can cancel or terminate the execution of the command by using the cancel command on the screen or by using the voice cancel command.

[0150] 2. After the user issues the command, the driver can directly take over the vehicle to terminate the co-driving command.

[0151] 3. The driver can disable touchscreen and voice control with a single button press using the steering wheel combination. Once disabled, the system can only be unlocked via the steering wheel buttons; the soft switch is ineffective to prevent unauthorized access after unlocking via the soft switch. The driver-vehicle co-driving button has lower priority than the two buttons that disable touchscreen and voice control.

[0152] In an optional embodiment, when using the touch screen or voice-activated human-machine interaction to control intelligent driving functions during autonomous driving, the passenger in the front seat or other people in the vehicle cabin may maliciously interfere (e.g., children, pets). In this embodiment, the risk of malicious interference is eliminated through various methods such as gesture intent inference, central control shortcut key locking, cloud-based identification of malicious interference in the cabin, and driver manual takeover.

[0153] In this embodiment of the application, the driver can assist the user in correcting unreasonable commands, thereby improving the safety and rationality of command input.

[0154] Continuing as shown in Figures 5 and 7, if the current instruction is determined to be unreasonable, the next instruction re-entered by the driver is received and overridden. This includes: if the abnormal frequency of the current instruction is determined to be greater than a first frequency threshold, a disabling instruction re-entered by the driver is received and overridden. The disabling instruction is used to disable the soft switch, and it is entered via a physical button. The disabling instruction also indicates that all software input control permissions are disabled during this power-on cycle.

[0155] It should be noted that the intelligent driving system will analyze the rationality of touch screen and voice control. If multiple or high-frequency unreasonable operations are detected, touch screen and voice control will be temporarily disabled until the owner confirms that they can be released and passes the retest.

[0156] The aforementioned first frequency threshold indicates the permissible range of erroneous commands allowed by the vehicle. This first frequency threshold is set based on user requirements. The higher the first frequency threshold, the more frequently erroneous commands are tolerated.

[0157] The abnormal frequency of the current command exceeding a first frequency threshold indicates that a malicious input command has been received, such as touch or voice input without a clear instruction. In this embodiment, such malicious input commands must be rejected. To avoid further losses caused by malicious input commands, a disable command is input to disable the soft switch function, rendering it ineffective. Thus, even if malicious input commands continue to exist, they are considered invalid, reducing the losses caused by malicious input commands.

[0158] As shown in Figure 7, the soft switch function will be activated using any of the following methods: 1) The human-vehicle co-driving touch interaction method is not disabled; 2) NOA (Navigation On Autopilot) is activated, ADV (Advanced Driver Assistance System) map is opened, and this function will automatically take effect.

[0159] The soft switch function will be deactivated by any of the following methods: (1) The touch interaction mode of human-vehicle co-driving is disabled; (2) ADV map is deactivated; (3) NOA is deactivated; (4) Manual takeover is initiated.

[0160] In this embodiment, the system can fully withstand user misoperation and erratic operation, while also preventing such operations, thereby improving vehicle control safety. Furthermore, the system can reject intentions that may pose risks to the outside world, intentions that may pose risks to the user's own vehicle, and intentions that violate laws or regulations. The intelligent driving system has the final decision-making power; if the user misoperates or operates erratically, the intelligent driving system will reject the operation if it deems it unreasonable, unsafe, or impossible in the current environment.

[0161] As shown in Figure 8, the physical buttons include physical buttons located on both sides of the vehicle's steering wheel, and when the physical buttons on both sides are pressed simultaneously, a disable command is generated.

[0162] As shown in Figure 8, the above vehicle safety control method may also include, but is not limited to, the following two steps: 1. Receiving a disable command from the vehicle's physical buttons (which may be referred to as buttons), the physical buttons being used to disable the soft switch function; 2. The disable command overwrites the current command and responds to the disable command, controlling the disable of the soft switch function on the touch screen, and / or controlling the disable of the touch screen's touch function.

[0163] In addition, after disabling the command that overrides the current command, the above methods can also include, but are not limited to, exiting the function of the aforementioned soft switch, disabling the touch function of the aforementioned touch screen, and switching the touch screen to other screens. Other screens are commonly used screens convenient for the driver. Other screens may include navigation screens, or entertainment / music screens.

[0164] In this embodiment of the application, physical buttons are manually operated to forcibly disable them, thereby reducing interference from other users to the driver and improving the safety of the driver.

[0165] Because vehicles have numerous physical buttons in various locations—for example, buttons on the steering wheel or on the center console—the number of such buttons can be one or more, and there is no limitation on this. A detailed explanation follows.

[0166] Continuing with the physical buttons shown in the box in Figure 8, the physical buttons in this article include the physical buttons located on both sides of the vehicle's steering wheel, and when the physical buttons on both sides are pressed simultaneously, a disable command is generated.

[0167] The physical buttons arranged on opposite sides of the steering wheel can be two or more (excluding two). For example, symmetrical arrangement of two physical buttons is not only aesthetically pleasing, but also makes it faster and more convenient for the driver to operate the steering wheel with both hands on it.

[0168] In this embodiment, physical buttons are manually operated to forcibly disable them, reducing interference from the passenger to the driver. Furthermore, simultaneous operation of multiple physical buttons achieves forced disabling, not only making it convenient for the driver to disable other users' operations but also preventing accidental operation by the driver.

[0169] In some practical application scenarios, after the current driving session ends and the vehicle is powered on again, the process returns to steps 210 to 240 to continue executing the judgment on whether the current instruction is reasonable, etc.

[0170] For the driver, the next instruction mentioned above includes one or more of the following: cancel the current instruction, terminate co-pilot instruction for physical takeover of the driver, and disable instruction for controlling soft switches to be ineffective; the terminate co-pilot instruction has the highest authority.

[0171] In this application embodiment, physical takeover may include, but is not limited to, forcibly disengaging from co-driving by having the user operate one or more of the following: the steering wheel, the accelerator pedal, and the brake pedal.

[0172] Referring again to Figures 5 and 6, in the third optional embodiment, the safety judgment mechanism further includes an intelligent driving system safety judgment mechanism. Accordingly, in step 1, the rationality and risk of the current instruction are analyzed according to the intelligent driving system safety judgment mechanism to determine whether the current instruction is reasonable. In step 2, if the current instruction is determined to be unreasonable and exceeds the scope that the intelligent driving system can handle, a request instruction for manual takeover by the driver is generated to override the current instruction. In step 3, if the current instruction is determined to be unreasonable but within the scope that the intelligent driving system can handle, control of the intelligent driving system is implemented. In step 4, if the current instruction is determined to be reasonable, step 240 above is executed.

[0173] In this embodiment of the application, the intelligent driving system safety judgment mechanism is used to indicate the rationality and risk of the intelligent driving system's judgment instruction.

[0174] It's important to note that while the intelligent driving system is granted high-level privileges, touchscreen or voice commands are merely weak requests. Whether and how these commands are executed is determined by the intelligent driving system. The system incorporates commands into its safety assessment, rejecting those that are risky or beyond its capabilities (similar to lane changing via a lever; the system will execute promptly, delay execution, or refuse execution depending on the situation). Therefore, even if someone maliciously manipulates the system and issues dangerous commands, the vehicle will not execute them.

[0175] The safety judgment mechanism of the above-mentioned intelligent driving system is as follows.

[0176] 1. Upon receiving a command from the user, the intelligent driving system analyzes the rationality and risk of the command based on its own safety rules. Commands that pose a risk will be refused or delayed.

[0177] 2. During the execution of user commands, if the intelligent driving system finds that the dynamic environment is no longer suitable for continued execution, it will pause or terminate the execution of the commands and automatically adopt the most appropriate strategy to deal with the situation (i.e., machine management).

[0178] 3. During the execution of user instructions, if the intelligent driving system finds that the dynamic environment is no longer suitable for continued execution, and the emergency situation exceeds the scope of the intelligent driving system's capabilities, the system will promptly request the user to take over.

[0179] 4. After receiving commands from the user, the system will analyze each command. If malicious operations are detected, a warning will be issued. If the situation does not improve, all touchscreen and voice control permissions will be disabled for the remainder of the current power cycle. For example, receiving multiple obviously unreasonable commands in a short period of time, or frequent, irregular multi-finger gestures appearing on the screen in a short period of time.

[0180] Referring again to Figures 5 and 6, in the fourth optional embodiment, the method further includes receiving a deactivation command from the background if the background security monitoring mechanism in the security judgment mechanism detects that the frequency of disabling is greater than a second frequency threshold, and overwriting the current command with the deactivation command, so that the vehicle owner account can apply for relearning and then enable it; wherein, the deactivation command is used to indicate the termination of the intelligent driving function. In this way, the deactivation command can be used to overwrite the current command. In this embodiment, the frequency of disabling refers to the number of times a disabling command is generated within a certain period. As can be seen from the above description, the disabling command can be a disabling command generated in the driver safety judgment mechanism (e.g., a disabling command input by the driver through a physical button based on subjective judgment), or a disabling command generated in other safety judgment mechanisms (e.g., a disabling command generated by the intelligent driving system when malicious operation is detected in the intelligent driving system safety judgment mechanism), and the disabling command is used to disable all touch screen operations (e.g., soft switches on the touch screen, etc.) and voice operations during the vehicle power-on cycle.

[0181] For example, if the intelligent driving system's safety judgment mechanism temporarily disables functions multiple times within a certain period, the backend will deactivate the vehicle's touchscreen and voice controls. This involves receiving a deactivation command from the backend so that the vehicle owner's account can apply for relearning and reactivation. Only after passing the learning and testing process can the functions be restored to normal use.

[0182] As one embodiment, the method further includes: Step 1, after allowing the vehicle to drive according to the current instruction, displaying an interruption icon for interrupting the execution of the current instruction and a continuation icon for continuing the execution of the current instruction in the electronic image, wherein the interruption icon is used to interrupt the execution of the current instruction and the continuation icon is used to continue the execution of the current instruction; Step 2, if an operation is received for the interruption icon, then the intelligent driving of the vehicle according to the current instruction is interrupted; Step 3, if an operation is received for the continuation icon, then the vehicle is controlled to continue intelligent driving according to the current instruction.

[0183] In one example, step A involves displaying an interruption touch icon and / or a continue execution touch icon within the electronic image; the interruption touch icon is used to interrupt the execution of the first instruction; the continue execution touch icon is used to continue the execution of the first instruction. Step B involves interrupting the control of the vehicle to drive intelligently according to the first instruction if the interruption touch icon is touched. Step C involves controlling the vehicle to continue driving intelligently according to the first instruction if the continue execution touch icon is touched.

[0184] In another example, step a involves displaying an interruption indicator and / or a continue execution indicator within an electronic image; the interruption indicator is used to interrupt the execution of the first instruction; the continue execution indicator is used to continue the execution of the first instruction. Step b involves, if a voice command is received for the interruption indicator, the control of the vehicle to intelligently drive according to the first instruction is interrupted. Step c involves, if a voice command is received for the continue execution indicator, the control of the vehicle to continue intelligently driving according to the first instruction is resumed.

[0185] In the embodiments of this application, the execution of instructions can be interrupted in a timely manner, facilitating timely human-computer interaction with the user.

[0186] In this article, the authority of the human-vehicle co-driving command is lower than the authority of the intelligent driving system's judgment, and the authority of the intelligent driving system's judgment is lower than the authority of physical takeover. The driving command sent by the user is merely a wish (a desire for the vehicle to fulfill their wish), not an order or control, and certainly not driving itself; it is simply communication with the intelligent driving system. Human-vehicle co-driving establishes a channel for transmitting the user's intentions to the system, and each intention is ensured to undergo feasibility and risk assessment by the system. Both the system and the driver can reject or cancel the intention midway. How to realize the intention is entirely decided by the system, and the decision-making process is unaffected by the sender.

[0187] Next, it supports reasonable operations by the front passenger (through touch swipes) or other occupants. The front passenger simply sends their intentions without directly operating the vehicle. Previously, the front passenger relayed their intentions to the driver; in the shared driving framework, this is achieved through the intelligent driving system. Both the driver and the intelligent driving system are licensed drivers and fully comply with the existing architecture of "intelligent driving + driver supervision."

[0188] Finally, current intelligent driving systems are a black hole for users. While the systems focus on functionality, they lack interaction with users during use, resulting in issues such as excessive control, lack of personalization, lack of confidence, and lack of security. Co-driving introduces an abstract intermediary layer between driving control and the driver, encapsulating driving behavior and decoupling driving control from the driver, making the vehicle oriented towards the driver's intentions rather than their control. Conversely, co-driving also encapsulates the system's perception and decisions into a reasonable form and presents them to the user through an HMI, thereby building a two-way interaction channel between people and the vehicle, giving users a fully controlled intelligent driving experience.

[0189] This application also provides a vehicle control device. Please refer to FIG9, which is a structural block diagram of the vehicle control device according to an embodiment of this application. The vehicle control device 21 provided in this application embodiment may include: an acquisition module 211, a receiving module 212, a judgment module 213, and a control module 214.

[0190] The acquisition module 211 is configured to acquire electronic images of the intelligent driving perception screen in real time; wherein, the electronic image is used to display vehicle identification and vehicle control identification, the vehicle identification is used to indicate real-time vehicle information, and the vehicle control identification is used to indicate the real-time vehicle control mode of the vehicle relative to other vehicles.

[0191] The receiving module 212 is configured to receive voice commands sent by the user for the electronic image in real time.

[0192] The judgment module 213 is configured to determine whether the vehicle control mode in the voice command matches the vehicle control mode indicated by the vehicle control identifier, and obtain the judgment result.

[0193] The control module 214 is configured to control whether the vehicle moves relative to other vehicles according to the voice command based on the judgment result and the real-time vehicle information.

[0194] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, which can achieve the same technical effect, and will not be repeated here.

[0195] In some embodiments, the vehicle control device 21 provided in this application may further include a guidance module.

[0196] The guidance module is configured to generate operation guidance based on the real-time vehicle information and preset intelligent driving strategy before the receiving module 212 receives the voice command sent by the user for the electronic image in real time, and to display the operation guidance on the intelligent driving perception screen and / or broadcast the operation guidance by voice; wherein, the operation guidance is used to guide the user to send the currently supported voice commands.

[0197] In some embodiments, the vehicle control device 21 provided in this application may further include a management and control module.

[0198] The control module is configured to, before the judgment module 213 determines whether the vehicle control method in the voice command matches the vehicle control method indicated by the vehicle control identifier and obtains the judgment result, determine whether the voice command conforms to the safe intelligent driving strategy, and refuse to execute the voice command and generate a violation notification when the voice command does not conform to the intelligent driving strategy, and continue to execute subsequent steps when the voice command conforms to the intelligent driving strategy; wherein, the violation notification is used to notify the user that the voice command violates the rule.

[0199] In some embodiments, the vehicle control method in the voice command includes information about a specific vehicle and the specific vehicle control method of the vehicle relative to that specific vehicle. When the judgment module 213 performs the judgment on whether the vehicle control method in the voice command matches the vehicle control method indicated by the vehicle control identifier, it is specifically configured to: determine whether the real-time vehicle information includes vehicle information that matches the information of the specific vehicle based on the information of the specific vehicle; in response to the real-time vehicle information including vehicle information that matches the information of the specific vehicle, identify the other vehicle corresponding to the vehicle information that matches the information of the specific vehicle as the target vehicle; and determine whether the specific vehicle control method matches the vehicle control method indicated by the vehicle control identifier corresponding to the target vehicle.

[0200] In some embodiments, the other vehicle includes a target vehicle, and the vehicle control identifier includes a target vehicle control identifier for indicating the real-time vehicle control mode of the vehicle relative to the target vehicle; when the judgment module 213 performs a judgment on whether the vehicle control mode in the voice command matches the vehicle control mode indicated by the vehicle control identifier and obtains a judgment result, the specific configuration is as follows: performing semantic parsing on the voice command to obtain the vehicle control mode contained in the command content of the voice command; judging whether the vehicle control mode in the command content matches the vehicle control mode indicated by the target vehicle control identifier; in response to the vehicle control mode in the command content matching the vehicle control mode indicated by the target vehicle control identifier, obtaining a matching result as the judgment result; in response to the vehicle control mode in the command content not matching the vehicle control mode indicated by the target vehicle control identifier, obtaining a non-matching result as the judgment result.

[0201] When the control module 214 executes the command to control whether the vehicle moves relative to another vehicle according to the voice command based on the judgment result and the real-time vehicle information, the specific configuration is as follows: based on the matching result and the real-time vehicle information, control the vehicle to move relative to the target vehicle according to the vehicle control method in the matching result; or based on the non-matching result and the real-time vehicle information, control the vehicle to maintain the current state of driving.

[0202] In some embodiments, the other vehicle includes a target vehicle, and the vehicle control identifier includes a target vehicle control identifier for indicating the vehicle control mode for the self-vehicle to follow and / or overtake the target vehicle. The vehicle control mode in the voice command received by the receiving module 212 is a lock-follow mode, and the judgment result obtained by the judgment module 213 is a matching result that matches the lock-follow mode. When the control module 214 executes the command to control whether the self-vehicle drives relative to the other vehicle according to the voice command based on the judgment result and the real-time vehicle information, it is specifically configured to control the self-vehicle to follow the target vehicle in the lock-follow mode by accelerating, decelerating, or changing lanes according to the real-time vehicle information.

[0203] In some embodiments, when the control module 214 executes the command to control the vehicle to accelerate, decelerate, or change lanes according to the real-time vehicle information and follow the target vehicle in the locked following mode, the specific configuration is as follows: when the vehicle identifier includes a cut-in vehicle identifier for indicating other vehicles entering between the vehicle and the target vehicle, the control module 214 searches for the target vehicle from the surrounding vehicles according to the vehicle attributes of the target vehicle; in response to finding the target vehicle, the control module continues to follow the target vehicle by accelerating, decelerating, or changing lanes; in response to not finding the target vehicle, the control module notifies the user that the locked following of the target vehicle has failed.

[0204] In some embodiments, the other vehicle includes a target vehicle, and the vehicle control identifier includes a target vehicle control identifier for indicating the vehicle control mode for the self-vehicle to follow and / or overtake the target vehicle. The vehicle control mode in the voice command received by the receiving module 212 is the following mode, and the judgment result obtained by the judgment module 213 is a matching result that matches the following mode. When the control module 214 executes the command to control whether the self-vehicle drives relative to the other vehicle according to the voice command based on the judgment result and the real-time vehicle information, it is specifically configured to control the self-vehicle to accelerate or decelerate in accordance with the following mode to follow the target vehicle according to the real-time vehicle information.

[0205] In some embodiments, when the control module 214 executes the command to control the vehicle to accelerate or decelerate in accordance with the following mode based on the real-time vehicle information, the specific configuration is as follows: when the vehicle identifier includes a cut-in vehicle identifier for indicating other vehicles entering between the vehicle and the target vehicle, the other vehicle is set as the new target vehicle, and the target vehicle control identifier is associated with the updated target vehicle; the vehicle is then controlled to accelerate or decelerate in accordance with the following mode to follow the updated target vehicle.

[0206] In some embodiments, the other vehicle includes a target vehicle, and the vehicle control identifier includes a target vehicle control identifier for indicating the vehicle control mode for following and / or overtaking the target vehicle. The vehicle control mode in the voice command received by the receiving module 212 is an overtaking mode, and the judgment result obtained by the judgment module 213 is a matching result that matches the overtaking mode. When the control module 214 executes the command to control whether the vehicle moves relative to the other vehicle according to the voice command based on the judgment result and the real-time vehicle information, it is specifically configured to control the vehicle to accelerate according to the overtaking mode based on the real-time vehicle information until it overtakes the target vehicle.

[0207] Furthermore, based on the same inventive concept as the vehicle safety control method described above with reference to Figures 5 to 8, the vehicle control device of the present application embodiment can also be further used to implement the above vehicle safety control method.

[0208] In some embodiments, as described above, in the vehicle control device, the acquisition module 211 can be configured to acquire electronic images of the intelligent driving perception screen in real time, and the receiving module 212 can be configured to receive current instructions input by the user in response to the electronic images in real time.

[0209] In addition, the vehicle control device may also include an instruction update module, which is configured to receive the next instruction adjacent to the current instruction and overwrite the current instruction if the current instruction is determined to be unreasonable according to at least one layer of safety judgment mechanism; the safety judgment mechanism is used to indicate that the vehicle is driven safely after determining that the current instruction is reasonable.

[0210] Furthermore, the control module 214 of the vehicle's control device can also be configured to allow the vehicle to drive according to the current instruction if it is determined that the current instruction is reasonable.

[0211] As one embodiment, the control device for the vehicle described above may further include: an icon display module, configured to display, within an electronic image, an interruption icon (for interrupting the execution of the current instruction) and a continuation icon (for continuing the execution of the current instruction) after allowing the vehicle to drive according to the current instruction; the interruption icon (for interrupting the execution of the current instruction) and the continuation icon (for continuing the execution of the current instruction); and an interruption control module, configured to interrupt intelligent driving of the vehicle according to the current instruction if an operation is received for the interruption icon (for the current instruction). Furthermore, the control module 214 is also configured to control the vehicle to continue intelligent driving according to the current instruction if an operation is received for the continuation icon (for the current instruction).

[0212] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, which can achieve the same technical effect, and will not be repeated here.

[0213] This application also provides an electronic device that may include the vehicle control device 21 described above. Please refer to FIG10, which is a structural block diagram of the electronic device according to an embodiment of this application. The electronic device 20 may include one or more processors 22, which are configured to implement the vehicle control method described above.

[0214] In some embodiments, the electronic device 20 may further include one or more computer-readable storage media 23, which may store a program that can be invoked by the processor 22, and may include non-volatile storage media. In other embodiments, the electronic device 20 may further include memory 24 and an interface 25. In still other embodiments, the electronic device 20 may also include other hardware depending on the specific application.

[0215] The computer-readable storage medium 23 provided in this application embodiment stores a program thereon, which, when executed by the processor 22, is used to implement the above-described vehicle control method.

[0216] This application may take the form of a computer program product implemented on one or more computer-readable storage media 23 (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. The computer-readable storage media 23 includes permanent or non-permanent, removable or non-removable media, and information storage can be implemented using any method or technology. This information may be computer-readable instructions, data structures, program modules, or other data. The computer-readable storage media 23 includes, but is not limited to: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information accessible by a computing device.

[0217] Of course, in some embodiments of this application, the electronic device may be one or more of a server device and a PC (Personal Computer) device. The server device and the PC device may include, but are not limited to, a server, a desktop computer, a tablet computer, or a laptop computer.

[0218] In some embodiments, this electronic device may include, but is not limited to, an in-vehicle terminal connected to the vehicle and a mobile terminal independent of the vehicle. The in-vehicle terminal connected to the vehicle may be, but is not limited to, a body processor, controller, center console, or vehicle HUD (Head-Up Display). The mobile terminal independent of the vehicle may include, but is not limited to, a smartphone, smartwatch, tablet, or laptop.

[0219] This application also provides a computer program product, which includes a computer program that can be stored on a computer-readable storage medium. When the computer program is executed by at least one processor, it can perform the vehicle control methods described in the above embodiments.

[0220] This application also provides a vehicle, including a vehicle control device as described in the above embodiments, or an electronic device as described in the above embodiments, or a computer-readable storage medium as described in the above embodiments, or a computer program product as described in the above embodiments.

[0221] For example, the vehicle may be one or more of heavy trucks, light commercial vehicles, and passenger cars. Thus, the method of this application embodiment can be applied to various application scenarios such as heavy trucks, light commercial vehicles, and passenger cars, thereby improving the versatility of the method.

[0222] The vehicle control method, device, electronic device, program product, storage medium, and vehicle provided in this application embodiment receive voice commands sent by the user to the electronic image of the intelligent driving perception screen in real time. When the vehicle control mode in the voice command matches the vehicle control mode indicated by the vehicle control icon on the electronic image, the system controls the vehicle to drive relative to other vehicles according to the voice command. This provides a voice interaction method between the user and the intelligent driving system, allowing the user to request the intelligent driving system to control the vehicle to drive relative to other vehicles according to their wishes through voice interaction. In addition, before controlling the vehicle to drive relative to other vehicles according to the voice command, the system also comprehensively considers the real-time vehicle information indicated by the vehicle icon on the electronic image, which improves the personalization of the intelligent driving system while ensuring driving safety, thereby improving the user experience of autonomous driving.

[0223] Those skilled in the art will understand that all or part of the steps in the above methods can be implemented by a program instructing related hardware (e.g., a processor), and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. Optionally, all or part of the steps in the above embodiments can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiments can be implemented in hardware, such as by using an integrated circuit to implement its corresponding function, or it can be implemented in the form of a software functional module, such as by a processor executing a program / instruction stored in memory to implement its corresponding function. This application is not limited to any particular combination of hardware and software.

[0224] The above description is merely an exemplary embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification shall be included within the scope of protection of this specification.

[0225] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, an element qualified by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A method for controlling a vehicle, comprising: The system acquires electronic images of the intelligent driving perception scene in real time; wherein the electronic images are configured to display vehicle identification and vehicle control identification, the vehicle identification is configured to indicate real-time vehicle information, and the vehicle control identification is configured to indicate the real-time vehicle control mode of the vehicle relative to other vehicles. It can receive voice commands from users regarding the electronic images in real time. Determine whether the vehicle control mode in the voice command matches the vehicle control mode indicated by the vehicle control icon, and obtain the determination result; Based on the judgment result and the real-time vehicle information, control whether the vehicle moves relative to the other vehicle according to the voice command.

2. The vehicle control method according to claim 1, wherein, The vehicle control method in the voice command includes information about the specific vehicle and the specific vehicle control method of the vehicle relative to the specific vehicle. The step of determining whether the vehicle control mode in the voice command matches the vehicle control mode indicated by the vehicle control identifier includes: Based on the information of the specific vehicle, determine whether the real-time vehicle information includes vehicle information that matches the information of the specific vehicle; In response to the real-time vehicle information including vehicle information that matches the information of the specific vehicle, the other vehicle corresponding to the vehicle information that matches the information of the specific vehicle is identified as the target vehicle. Determine whether the specific vehicle control method matches the vehicle control method indicated by the vehicle control identifier corresponding to the target vehicle.

3. The vehicle control method according to claim 1, wherein, The other vehicle includes the target vehicle, and the vehicle control identifier includes a target vehicle control identifier configured to indicate the real-time vehicle control mode of the vehicle relative to the target vehicle. The step of determining whether the vehicle control mode in the voice command matches the vehicle control mode indicated by the vehicle control identifier, and obtaining the determination result, includes: The voice command is semantically parsed to obtain the vehicle control method contained in the command content; Determine whether the vehicle control method contained in the instruction content matches the vehicle control method indicated by the target vehicle control identifier; In response to the vehicle control method contained in the instruction content matching the vehicle control method indicated by the target vehicle control identifier, a matching result is obtained as the judgment result; The step of controlling whether the vehicle moves relative to the other vehicle according to the voice command based on the judgment result and the real-time vehicle information includes: Based on the real-time vehicle information, the vehicle is controlled to drive relative to the target vehicle according to the vehicle control method in the matching result.

4. The vehicle control method according to claim 1, wherein, The other vehicle includes the target vehicle, the vehicle control identifier includes a target vehicle control identifier configured to indicate that the vehicle locks and follows the target vehicle in a real-time vehicle control mode, the vehicle control mode in the voice command includes a lock-follow mode, and the judgment result is a matching result that matches the lock-follow mode; The step of controlling whether the vehicle moves relative to the other vehicle according to the voice command based on the judgment result and the real-time vehicle information includes: Based on the real-time vehicle information, the vehicle is controlled to follow the target vehicle in the locked following mode, accelerating, decelerating, or changing lanes.

5. The vehicle control method according to claim 4, wherein, The step of controlling the vehicle to accelerate, decelerate, or change lanes according to the real-time vehicle information and following the target vehicle in the locked following mode includes: If the vehicle identifier includes a cut-in vehicle identifier configured to indicate other vehicles entering between the vehicle and the target vehicle, the target vehicle is searched for from the vehicles surrounding the vehicle based on the vehicle attributes of the target vehicle. In response to the detection of the target vehicle, continue to follow the target vehicle by accelerating, decelerating, or changing lanes; In response to the inability to find the target vehicle, the user is notified that locking onto and following the target vehicle has failed.

6. The vehicle control method according to claim 1, wherein, The other vehicle includes the target vehicle, the vehicle control identifier includes a target vehicle control identifier configured to indicate the vehicle to follow and / or overtake the target vehicle in real time, the vehicle control mode in the voice command includes a following mode, and the judgment result is a matching result that matches the following mode; The step of controlling whether the vehicle moves relative to the other vehicle according to the voice command based on the judgment result and the real-time vehicle information includes: Based on the real-time vehicle information, the vehicle is controlled to accelerate or decelerate in accordance with the following method to follow the target vehicle.

7. The vehicle control method according to claim 6, wherein, The step of controlling the vehicle to accelerate or decelerate in accordance with the following method based on the real-time vehicle information includes: In the case where the vehicle identifier includes a cut-in vehicle identifier configured to indicate other vehicles entering between the vehicle and the target vehicle, the other vehicle is set as the new target vehicle, and the target vehicle control identifier is associated with the new target vehicle; Control the vehicle to accelerate or decelerate in accordance with the following method to follow the new target vehicle.

8. The vehicle control method according to claim 1, wherein, The other vehicle includes the target vehicle, the vehicle control identifier includes a target vehicle control identifier configured to indicate the vehicle to follow and / or overtake the target vehicle in real time, the vehicle control mode in the voice command includes an overtaking mode, and the judgment result is a matching result that matches the overtaking mode; The step of controlling whether the vehicle moves relative to the other vehicle according to the voice command based on the judgment result and the real-time vehicle information includes: Based on the real-time vehicle information, the vehicle is controlled to accelerate in accordance with the overtaking method until it overtakes the target vehicle.

9. The vehicle control method according to any one of claims 1 to 8, wherein, Before determining whether the vehicle control mode in the voice command matches the vehicle control mode indicated by the vehicle control identifier, and obtaining the determination result, the vehicle control method further includes: Determine whether the voice command conforms to a safe intelligent driving strategy; In response to the voice command conforming to a safe intelligent driving strategy, continue to execute subsequent steps; In response to the voice command not conforming to the safe intelligent driving strategy, the system refuses to execute the voice command and generates a violation notification; wherein the violation notification is configured to notify the user that the voice command is in violation. and / or The step of controlling whether the vehicle moves relative to the other vehicle according to the voice command based on the judgment result and the real-time vehicle information includes: In response to the real-time vehicle information conforming to a safe intelligent driving strategy, the system controls whether the vehicle moves relative to the other vehicle according to the voice command, based on the judgment result.

10. The vehicle control method according to any one of claims 1 to 9, wherein, Before receiving voice commands from the user regarding the electronic image in real time, the vehicle control method further includes: Based on the real-time vehicle information and the preset intelligent driving strategy, an operation guide is generated; wherein, the operation guide is configured to guide the user to send currently supported voice commands; The operation instructions are displayed on the intelligent driving perception screen and / or read aloud via voice.

11. The vehicle control method according to claim 1, wherein, After receiving voice commands from the user regarding the electronic image in real time, the vehicle control method further includes: According to at least one layer of safety judgment mechanism, if it is determined that the voice command is unreasonable, the next command adjacent to the voice command is received, and the next command overwrites the voice command; the at least one layer of safety judgment mechanism is configured to indicate that the vehicle drives safely after determining that the voice command is reasonable; and, If the voice command is deemed reasonable, the vehicle may be controlled to drive in accordance with the voice command.

12. The vehicle control method according to claim 11, wherein, The at least one layer of security judgment mechanism includes a user security judgment mechanism. The step of receiving the next instruction adjacent to the voice instruction and overwriting the voice instruction with the next instruction, based on the at least one layer of security judgment mechanism, includes: Based on the user security judgment mechanism, if it is determined that the voice command is unreasonable, the system receives the next command re-inputted by the user and overwrites the voice command. The user security judgment mechanism is configured to indicate an instruction that the user has corrected an unreasonable situation.

13. The vehicle control method according to claim 11 or 12, wherein, The at least one layer of safety judgment mechanism further includes a driver safety judgment mechanism. The step of receiving the next instruction adjacent to the voice instruction and overwriting the voice instruction with the next instruction, based on the at least one layer of safety judgment mechanism, includes: Based on the driver safety judgment mechanism, if it is determined that the voice command is unreasonable, the driver will re-enter the next command and overwrite the original voice command. The driver safety judgment mechanism is configured to indicate that the driver assists the user in correcting unreasonable instructions.

14. The vehicle control method according to claim 13, wherein, The step of receiving the next command re-entered by the driver when it is determined that the voice command is unreasonable, and overwriting the voice command, includes: If it is determined that the abnormal frequency of the voice command is greater than a first frequency threshold, the driver re-enters a disable command to override the voice command. The disable command is configured to disable the soft switch, and the disable command is input via physical button operation.

15. The vehicle control method according to any one of claims 11 to 13, wherein, The at least one layer of safety judgment mechanism also includes an intelligent driving system safety judgment mechanism. According to the at least one layer of safety judgment mechanism, if it is determined that the voice command is unreasonable, receiving the next command adjacent to the voice command and overwriting the voice command with the next command includes: Based on the safety judgment mechanism of the intelligent driving system, the rationality and risks of the voice command are analyzed to determine whether the voice command is reasonable. If it is determined that the voice command is unreasonable and exceeds the processing range of the intelligent driving system, a request command for the driver to take over is generated to override the voice command. The intelligent driving system's safety judgment mechanism is configured to indicate the rationality and risk of the intelligent driving system's judgment commands, and / or The at least one layer of security judgment mechanism also includes a background security monitoring mechanism. The step of receiving the next instruction adjacent to the voice instruction and overwriting the voice instruction with the next instruction, based on the at least one layer of security judgment mechanism, includes: If the frequency of disabling is detected to be greater than the second frequency threshold according to the background security monitoring mechanism, a deactivation command is received from the background and the deactivation command is overwritten to enable the vehicle owner account to apply for relearning and then be reactivated. The deactivation command is configured to indicate the termination of the intelligent driving function.

16. A vehicle control device, comprising: The acquisition module is configured to acquire electronic images of the intelligent driving perception screen in real time; the electronic images are configured to display vehicle identification and vehicle control identification, the vehicle identification is configured to indicate real-time vehicle information, and the vehicle control identification is configured to indicate the real-time vehicle control mode of the vehicle relative to other vehicles. The receiving module is configured to receive voice commands sent by the user for the electronic image in real time; The judgment module is configured to determine whether the vehicle control mode in the voice command matches the vehicle control mode indicated by the vehicle control identifier, and obtain a judgment result; The control module is configured to control whether the vehicle moves relative to the other vehicle according to the voice command based on the judgment result and the real-time vehicle information.

17. An electronic device comprising one or more processors configured to implement a vehicle control method as claimed in any one of claims 1 to 15.

18. A non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by at least one processor, implement the vehicle control method as described in any one of claims 1 to 15.

19. A computer program product comprising a computer program stored on a computer-readable storage medium, wherein the computer program, when executed by at least one processor, implements the vehicle control method as described in any one of claims 1 to 15.

20. A vehicle comprising: The vehicle control device as described in claim 16, or The electronic device as claimed in claim 17, or The non-transitory computer-readable storage medium as described in claim 18, or The computer program product as described in claim 19, or One or more processors configured to implement the vehicle control method as described in any one of claims 1 to 15.

Images