Autonomous driving remote control method and control system that meets safety level requirements

Abstract

The present invention provides an automatic driving remote control method and control system that meets the safety level requirements, wherein the smart device side obtains the original control data required for the control behavior; the smart device side is connected to the vehicle side, and the smart device side continuously transmits the acquired original control data to the vehicle side, and the vehicle side learns the control behavior and the original control data until the control behavior is consistent with the original control data, completing the learning process of the vehicle side and the pairing with the smart device side; the smart device side continuously transmits the real-time control data of the exercise request to the vehicle side, and the vehicle side compares the real-time control data with the original control data to verify the validity of the real-time control data. The hardware of the smart device side only needs to be developed according to QM; the software of the smart device side and the related software and hardware of the vehicle side are developed according to the equivalent ASIL, effectively realizing the collaborative work of the smart device side and the vehicle side, and achieving the required ASIL level on the complete safety path; the goal of saving development costs.

Description

Technical Field

[0001] The present invention relates to the field of vehicle autonomous driving, and in particular to an autonomous driving remote control method that meets safety level requirements and a control system using the method. Background Art

[0002] Autonomous vehicles (Self-driving automobiles), also known as driverless cars, computer-driven cars or wheeled mobile robots, are intelligent cars that are unmanned by computer systems. They have been around for decades in the 20th century and are approaching practical use in the early 21st century.

[0003] The ideal state of self-driving cars is to rely on artificial intelligence, visual computing, radar, monitoring devices and global positioning systems to work together, so that smart devices can automatically and safely operate motor vehicles without any active human operation. However, in the current actual operation and testing process, this state is far from being achieved. One of the important issues is driving safety, especially how smart devices and vehicles work together to achieve the required safety level (ASIL level) on the complete safety path.

[0004] At present, there are clear safety standards for automotive parts, such as the ISO-26262 ASIL level (Automotive Safety Integrity Level), which describes the probability that the system can achieve a specified safety goal. Each safety function requirement includes two parts: safety goal and ASIL safety level. However, there are no similar standards for smart devices that realize autonomous driving, such as background computers and mobile phones. International car manufacturers are currently considering extending the applicability of ISO-26262 ASIL levels to smart devices. If smart devices must be developed in accordance with ISO-26262 ASIL level requirements, the following problems will arise:

[0005] (1) Development costs have increased significantly compared to original consumer products, increasing by at least 300%, especially hardware costs;

[0006] (2) Due to scale issues, developers investing in this field will see their investment significantly reduced.

[0007] Therefore, based on the above technologies, technicians in this field are committed to providing autonomous driving remote control methods and control systems that meet safety level requirements, shifting the cost issues faced in consumer product development and the compliance issues faced when integrating automotive components. Summary of the invention

[0008] The technical problem to be solved by the present invention is to provide an automatic driving remote control method and control system that meets the safety level requirements, so as to solve the problems in the background technology.

[0009] In order to solve the above problems, the present invention provides an automatic driving remote control method that meets the safety level requirements, which includes at least two control end operations, namely, the smart device end and the vehicle end:

[0010] Step 1: The smart device obtains the original control data required for the control behavior;

[0011] Step 2: Connect the smart device and the vehicle. The smart device continuously transmits the acquired original control data to the vehicle. The vehicle learns the control behavior and the original control data until the control behavior is consistent with the original control data. The smart device is notified to complete the learning process of the vehicle and pairing with the smart device.

[0012] Step 3: When entering the driving state, the smart device will continuously transmit the real-time control data of the driving request to the vehicle. The smart device needs to transmit at least N (N>=2) frames of control data to the vehicle. The vehicle will compare the real-time control data with the original control data for N consecutive frames to verify the validity of the real-time control data, and then determine whether the vehicle responds to the real-time control data.

[0013] It should be noted that the implementation order of the above Step 1 and Step 2 is adjustable.

[0014] Furthermore, in step Step 1, the original control data is the physical quantity collected by the smart device hardware and is data that has not been processed by the internal software of the smart device, including but not limited to visual images, audio signals, fingerprint signals, physical dynamic trajectory signals, etc.

[0015] Furthermore, in step Step 2, when the vehicle side is learning the control behavior and the original control data, the smart device side needs to continuously output the original control data to the vehicle side until the learning behavior of the vehicle side is completed.

[0016] Furthermore, in step Step 2, the pairing of the smart device end and the vehicle end may be one smart device end and one vehicle end, one smart device end and multiple vehicle ends, or multiple smart device ends and one vehicle end.

[0017] Furthermore, in step 2, the specific steps for implementing vehicle-side learning include at least:

[0018] Step 20: Connect the smart device to the vehicle.

[0019] Step 21: The smart device and the vehicle exchange information, and the smart device displays information according to the information exchange steps of the vehicle.

[0020] Step 22: The user operates according to the information displayed on the smart device;

[0021] Step 23: The smart device transmits the original control data of the operation process to the vehicle;

[0022] Step 24: The vehicle side continuously learns through a convolutional neural network or an expert system based on the received original control data until the control behavior is consistent with the original control data, forming a control memory and completing the learning process;

[0023] Step 25: The smart device receives the signal from the vehicle indicating that learning is complete and instructs the next step.

[0024] Furthermore, in step Step 3, when the smart device continuously transmits the real-time original control data obtained in the driving state to the vehicle, the real-time control data needs to be attached with an end-to-end protection mechanism or an information encryption and decryption key mechanism.

[0025] Based on the above-mentioned automatic driving remote control method, the present invention also provides a control system, which at least includes an intelligent device end and a vehicle end, and the intelligent device end and the vehicle end are connected through a communication module; the intelligent device end at least includes a signal acquisition module, a data receiving module and an operation control module, wherein the signal acquisition module acquires the original control data and real-time control data for the user to implement the control behavior on the vehicle end; the original data acquired by the signal acquisition module is transmitted to the vehicle end through the above-mentioned communication module; the data receiving module receives feedback information from the vehicle end; the operation control module is used for the user to implement specific operations; the vehicle end at least includes a control module, a data processing module and a verification module, the data processing module receives the original control data from the intelligent device end, completes the learning and training, feeds back to the intelligent device end, and stores the original control data; the verification module has an end-to-end protection verification mechanism or an information encryption and decryption key mechanism, the verification module realizes the matching relationship between the real-time control data of the control behavior and the stored original data, and feeds back; the control module receives the real-time control data verified by the verification module, and controls whether the vehicle end responds to the real-time control data.

[0026] Furthermore, each frame of raw data collected by the signal acquisition module is not subjected to any algorithm processing and is directly transmitted to the vehicle end through the communication module.

[0027] Furthermore, the communication module and the signal acquisition module use the same frequency to transmit the original control data to the vehicle end in real time.

[0028] By implementing the autonomous driving remote control method and control system that meet the safety level requirements provided by the present invention, the technical effect is as follows: the hardware development of the smart device in this technical solution no longer needs to be carried out in accordance with the ASIL standard, because the failure mode caused by the hardware can be completely covered by the ASIL level continuous verification mechanism on the vehicle side, saving huge development costs for hardware sharing; at the same time, through the learning before officially entering the driving state and the verification mechanism on the vehicle side, the collaborative work of the smart device and the vehicle is effectively realized, solving the problem of the collaborative operation of automotive-grade safety components and non-automotive-grade safety components to achieve the ASIL level required on the complete safety path. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention.

[0030] Figure 1 This is a schematic diagram of the control system structure in an embodiment of the present invention;

[0031] Figure 2 This is a schematic diagram of learning and training in an embodiment of the present invention;

[0032] Figure 3 Schematic diagram of collaborative work in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] The technical solution of the present invention is described in detail below using specific implementation methods.

[0035] like Figure 1 The figure shows an automatic driving remote control system that meets the safety level requirements shown in this embodiment. The control system includes at least a smart device end 1 and a vehicle end 2. The smart device end 1 and the vehicle end 2 are connected through a communication module 3; wherein the smart device end 1 can at least be a mobile phone, a PC end, and a background, generally referring to a smart device that can realize process and remote operation. The communication module 3 can at least be a communication method that can realize data transmission, such as 4G, 5G, WiFi, Bluetooth, etc., and the specific selection can be determined according to the bandwidth required for transmitting data.

[0036] The smart device end 1 at least includes a signal acquisition module 10, a data receiving module 11 and an operation control module 12, wherein the signal acquisition module 10 collects the original control data and real-time control data of the user to implement the control behavior on the vehicle end 2, and the original control data and real-time control data include but are not limited to visual images, audio signals, fingerprint signals, and physical dynamic trajectory signals. In actual operation, the signal acquisition module 10 can generally use sensors such as cameras, microphones, touch screens, etc. to collect original control data and real-time control data. The acquisition device can be selected according to the type of data to be collected; each frame of original data collected by the signal acquisition module 10 is not processed by any algorithm, and is directly transmitted to the vehicle end 2 through the communication module 3; this requires that the communication module 3 and the signal acquisition module 10 use the same frequency to transmit the original control data to the vehicle end 2 in real time; and each frame of data collected by the signal acquisition module 10 (the first frame-the Nth frame), N>=2, needs to be continuously transmitted to the vehicle end 2 through the communication module 3. The data receiving module 11 receives feedback information from the vehicle end 2; the operation control module 12 is used for the user to implement specific operations.

[0037] The vehicle end 2 is installed on the autonomous driving vehicle, and at least includes a control module 20, a data processing module 21 and a verification module 22. The data processing module 21 has a CNN module (convolutional neural network). The CNN module will record the original control data transmitted by the user or the smart device end 1 during the learning and training phase, such as facial features, audio features, fingerprint features, etc.; after the smart control end 1 is connected to the vehicle end 2, the data processing module 21 continuously receives the original control data from the smart device end 1, which can be specifically manifested as: the user can continuously provide the original control data from multiple angles, such as using the mobile phone camera to surround the face, and the data processing module 21 receives the information and issues a successful reminder after processing, or continuously presses the touch screen until the data processing module 21 processes and issues a successful reminder, etc. When the data processing module 21 completes the learning and training through the CNN module (or expert system module), it stores the original control data and feeds back the signal of learning completion to the data receiving module 11 of the smart device end 1 through the communication module 3, and the smart device end 1 indicates the next operation according to the received feedback.

[0038] In the above description and illustrations, the smart device end 1, the signal acquisition module 10, the data receiving module 11, the operation control module 12 and the communication module 3 are all QM (Quality Management) industrial-grade hardware; the vehicle end 2, the control module 20, the data processing module 21 and the verification module 22 are ASIL automotive-grade hardware.

[0039] Under the above learning and training, the learning tasks that need to be completed and qualified include at least the following tasks:

[0040] (1) The pairing between the smart device 1 (including the relevant control APP) and the vehicle 2 (including the function control software) is completed. After the learning is completed, the smart device 1 is locked to a specific vehicle. In actual operation, one smart device 1 can be paired with multiple vehicle 2s, or multiple smart devices 1 can be paired with one vehicle 2; or one smart device 1 can be paired with one vehicle 2, which can be set by the vehicle manufacturer.

[0041] (2) After pairing the smart device 1 with the vehicle 2, you can continue to use the verification procedure under the learning training;

[0042] (3) Under learning and training, the original control data corresponding to different control behaviors can be set by the user, or it can be pre-set by the car manufacturer, and then the user completes the pairing through the learning program.

[0043] It should be noted that, in the learning process, this embodiment uses a CNN module for training, and in actual operation, a traditional expert system can also be used for training, for example, by entering a password and verifying that the password is correct to complete the matching problem of the operation behavior and the control signal.

[0044] In the data processing module 21, the learning and training process is as follows Figure 2 As shown, including at least:

[0045] (1) The smart device 1 triggers the learning mode and connects to the data processing module 21 of the vehicle 2 through the communication module 3; at this time, the data processing module of the vehicle 2 accepts the connection and processes the learning mode;

[0046] (2) The smart device 1 and the data processing module 21 of the vehicle 2 perform information handshaking, and the smart device 1 displays information according to the information handshaking steps of the vehicle 2;

[0047] (3) The user performs operations according to the information displayed on the smart device 1;

[0048] (4) The smart device 1 transmits the original control data of the operation process to the data processing module 21 of the vehicle 2;

[0049] (5) The data processing module 21 of the vehicle end 2 continuously learns through the convolutional neural network according to the received original control data until the control behavior is consistent with the original control data, forming a control memory and completing the learning process;

[0050] (6) The smart device 1 receives a signal indicating that learning is complete from the vehicle 2 and instructs the next step.

[0051] The verification module 22 of the vehicle end 2 has an end-to-end protection verification mechanism (such as a CRC verification mechanism) or an information encryption and decryption key mechanism to verify that the original data is not damaged or tampered with by hackers during transmission through the communication module. The verification module realizes the matching relationship between the real-time control data of the control behavior and the stored original data, and feeds back to the control module 20; specifically, the intelligent control terminal 1 transmits the real-time control data generated by the user to the verification module 22 of the vehicle end 2 in the working state (exercise state). The verification mechanism in the verification module 22 compares the real-time control data with the original control data stored under learning and training in the data processing module 21 and compares the end-to-end verification or information encryption and decryption key mechanism verification results to verify the validity and integrity of the real-time control data. The transmission of the real-time control data and the original control data is continuous (from the 1st frame to the Nth frame of the data), N>=2, such as Figure 3 As shown, the working state (driving state) ends; the original control data required by the verification module to verify the validity of the data will vary according to the real-time control data generated by the user. In actual operation, different verification categories can be set as needed, such as function start request, function release request, vehicle trajectory control, vehicle dynamic stop request, etc. The CNN module (or expert system) confirms the validity of the real-time control data through continuous comparison over a defined time.

[0052] The control module 20 receives the real-time control data verified by the verification module 22. When the verification mechanism is verified, the control module 20 responds to the real-time control data and controls the vehicle end 2 to implement the control behavior; if the verification mechanism is not verified, the control module 20 does not respond to the control behavior.

[0053] It should be noted that, unless otherwise defined, the technical terms or scientific terms used herein should be understood by people with ordinary skills in the field to which the present invention belongs. After considering the specification and practicing the invention here, those skilled in the art will easily think of other embodiments of the present invention. This application is intended to cover any use or adaptation of the present invention, which follows the general principles of the present invention and includes common knowledge or customary technical means in the art that are not disclosed by the present invention. The specification and examples are only regarded as exemplary, and the true scope and spirit of the present invention are indicated by the claims.

[0054] It should be understood that the present invention is not limited to the structures described above and shown in the drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A remote control method for autonomous driving that meets safety level requirements, characterized in that: At least including the operation of two control terminals, the smart device terminal and the vehicle terminal, wherein the smart device terminal and the vehicle terminal are connected via a communication module: Step 1: The smart device obtains the original control data required for the control behavior; Step 2: Connect the smart device and the vehicle. The smart device continuously transmits the acquired original control data to the vehicle. The vehicle learns the control behavior and the original control data until the control behavior is consistent with the original control data. The smart device is notified to complete the learning process of the vehicle and pairing with the smart device. Step 3: When entering the driving state, the smart device will continuously transmit the real-time control data of the driving request to the vehicle. The smart device needs to transmit at least N (N>=2) frames of control data to the vehicle. The vehicle will compare the real-time control data with the original control data for N consecutive frames to verify the validity of the real-time control data, and then determine whether the vehicle responds to the real-time control data. Among them, the order of Step1 and Step2 is adjustable.

2. The remote control method for autonomous driving that meets the safety level requirements according to claim 1, characterized in that: In step 1, the original control data is the physical quantity collected by the smart device hardware and is not processed by the internal software of the smart device, including but not limited to visual images, audio signals, fingerprint signals, and physical dynamic trajectory signals.

3. The remote control method for autonomous driving that meets the safety level requirements as claimed in claim 2, characterized in that: In step 2, when the vehicle side is learning the control behavior and the original control data, the smart device side needs to continuously output the original control data to the vehicle side until the learning behavior of the vehicle side is completed.

4. The remote control method for autonomous driving that meets the safety level requirements as claimed in claim 3, characterized in that: In step Step 2, the pairing of the smart device end and the vehicle end is pairing of one smart device end and one vehicle end, or pairing of one smart device end and multiple vehicle ends, or pairing of multiple smart device ends and one vehicle end.

5. The remote control method for autonomous driving that meets the safety level requirements as claimed in claim 4, characterized in that: In step 2, the specific steps for implementing vehicle-side learning include at least: Step 20: Connect the smart device to the vehicle. Step 21: The smart device and the vehicle exchange information, and the smart device displays information according to the information exchange steps of the vehicle. Step 22: The user operates according to the information displayed on the smart device; Step 23: The smart device transmits the original control data of the operation process to the vehicle; Step 24: The vehicle side continuously learns through a convolutional neural network or an expert system based on the received original control data until the control behavior is consistent with the original control data, forming a control memory and completing the learning process; Step 25: The smart device receives the signal from the vehicle indicating that learning is complete and instructs the next step.

6. The remote control method for autonomous driving that meets the safety level requirements as claimed in claim 5, characterized in that: In step 3, when the smart device continuously transmits the real-time original control data obtained in the driving state to the vehicle, the real-time control data needs to be attached with an end-to-end protection mechanism or an information encryption and decryption key mechanism.

7. A remote control system for autonomous driving that meets safety level requirements, the system implements safe control of an autonomous driving vehicle through the control method described in any one of claims 1 to 6, characterized in that: The control system at least includes an intelligent device end and a vehicle end, wherein the intelligent device end and the vehicle end are connected via a communication module; the intelligent device end at least includes a signal acquisition module, a data receiving module and an operation control module, wherein the signal acquisition module acquires original control data and real-time control data for the user to implement control actions on the vehicle end; the original data acquired by the signal acquisition module is transmitted to the vehicle end via the communication module; the data receiving module receives feedback information from the vehicle end; the operation control module is used for the user to implement specific operations; the vehicle end at least includes a control module, a data processing module and a verification module, wherein the data processing module receives the original control data from the intelligent device end, completes learning and training, feeds back to the intelligent device end, and stores the original control data; The verification module has an end-to-end protection verification mechanism or an information encryption and decryption key mechanism. The verification module realizes the matching relationship between the real-time control data of the control behavior and the stored original data, and feeds back; the control module receives the real-time control data verified by the verification module, and controls whether the vehicle end responds to the real-time control data; The smart device side is QM industrial-grade hardware, and the vehicle side is ASIL automotive-grade hardware.

8. The remote control system for autonomous driving that meets the safety level requirements as claimed in claim 7, characterized in that: Each frame of raw data collected by the signal acquisition module is not processed by any algorithm and is directly transmitted to the vehicle end through the communication module; And / or communication modules are QM industrial grade hardware.

9. The remote control system for autonomous driving that meets the safety level requirements as claimed in claim 8, characterized in that: The communication module and the signal acquisition module use the same frequency to transmit the original control data to the vehicle end in real time.

Images