A remote control vehicle method and system

Abstract

The present invention discloses a remote control vehicle method and system, which belongs to the field of vehicle technology, stores detected vehicle surrounding environment information inside the vehicle, stores vehicle cabin environment in advance inside the vehicle, sends vehicle surrounding environment information and vehicle cabin environment information to a remote signal receiving device; uses remote signal receiving device projection technology to project vehicle surrounding environment information and vehicle cabin environment information around a remote driver, so that the remote driver can perform remote control according to a virtual driving environment and cabin environment; after system authentication and remote control permission, performs remote driving operations according to vehicle surrounding environment information and vehicle cabin environment information, monitors the driving intention of the remote driver, and sends control instructions to the vehicle based on the detected driving intention. The present invention enables the remote control personnel to perform complex operations such as shifting, braking, throttle, turn signal, etc. in a relatively real environment through a virtual driving environment.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicles, and more specifically, relates to a method and system for remotely controlling a vehicle. Background Art

[0002] Vehicle remote control mainly relies on mobile devices for simple short-distance operations, such as moving forward and backward. Most vehicles cannot simulate normal driving vision and operation methods, resulting in remote control personnel being unable to fully and accurately control vehicle operation and causing accidents.

[0003] Currently, the vehicle is mainly controlled by Bluetooth or cellular signals of mobile phones to perform simple forward, reverse and turn signal operations, which cannot simulate a more realistic driving state. Summary of the invention

[0004] In view of the above defects or improvement needs of the prior art, the present invention proposes a method and system for remotely controlling a vehicle, which enables the remote control personnel to perform complex operations such as shifting, braking, accelerator, turn signal, etc. in a relatively real environment through a virtual driving environment.

[0005] To achieve the above object, according to one aspect of the present invention, a method for remotely controlling a vehicle is provided, comprising:

[0006] storing the detected vehicle surrounding environment information inside the vehicle, storing the vehicle cabin environment in advance inside the vehicle, and sending the vehicle surrounding environment information and the vehicle cabin environment information to a remote signal receiving device;

[0007] After the position of the remote signal receiving device is fixed, the vehicle surrounding environment information and the vehicle cockpit environment information are projected around the remote driver using the remote signal receiving device projection technology, so that the remote driver can perform remote control according to the virtual driving environment and cockpit environment;

[0008] After system authentication and remote control permission, remote driving operations are performed based on the vehicle's surrounding environment information and vehicle cabin environment information, and the remote driver's driving intention is monitored. The detected driving intention is sent to the vehicle to send control instructions to control the vehicle.

[0009] In some optional implementation schemes, the remote driver's driving intention is monitored, including:

[0010] Monitor the changes in the remote driver's hand position on the steering wheel to determine the remote driver's left turn intention and right turn intention.

[0011] In some optional implementation schemes, monitoring the change in the position of the remote driver's hand holding the steering wheel to determine the remote driver's intention to turn left includes:

[0012] At time t0, the right hand feature is detected on the right side, and the center coordinates of the prediction box position are (x1, y1). At time t0+△t, the right hand feature is detected above the steering wheel, and the center coordinates of the prediction box position are (x2, y2). The hand movement speed is calculated as Vx = ((x2-x1) / △t), Vy = ((y2-y1) / △t), where the prediction box is used to track the hand movement;

[0013] If Vx≥the first preset value, or Vy≥the second preset value, it is determined to be an erroneous operation. If the erroneous operation condition is not met, it is determined that the driver is performing a left turn operation.

[0014] In some optional implementation schemes, monitoring the change in the position of the remote driver's hand holding the steering wheel to determine the remote driver's intention to turn right includes:

[0015] At t0', the left hand feature is detected on the left side, and the center coordinates of the prediction box position are (x1', y1'). At t0'+△t, the left hand feature is detected above the steering wheel, and the center coordinates of the prediction box position are (x2', y2'). The hand movement speed is calculated as Vx'=((x2'-x1') / △t), Vy'=((y2'-y1') / △t);

[0016] If Vx'=≥ the first preset value, or Vy'≥ the second preset value, it is determined to be an erroneous operation. If the erroneous operation condition is not met, it is determined that the driver is performing a right turn operation.

[0017] In some optional implementation schemes, after sending a control instruction to the vehicle based on the detected driving intention to control the vehicle, the method further includes:

[0018] Receive the operation images sent by the vehicle and re-update the projected virtual driving environment.

[0019] According to another aspect of the present invention, there is provided a remote control vehicle system, comprising:

[0020] A virtual driving environment generation module is used to store the detected vehicle surrounding environment information inside the vehicle and store the vehicle cabin environment inside the vehicle in advance;

[0021] A wireless communication module, used to send vehicle surrounding environment information and vehicle cabin environment information to a remote signal receiving device;

[0022] A remote signal receiving device, which is used to project the vehicle's surrounding environment information and the vehicle's cockpit environment information around the remote driver using projection technology after its position is fixed, so that the remote driver can perform remote control according to the virtual driving environment and cockpit environment;

[0023] The remote signal receiving device is also used to perform remote driving operations based on the vehicle's surrounding environment information and vehicle cabin environment information after system authentication and remote control permission, monitor the remote driver's driving intention, and send control instructions to the vehicle based on the detected driving intention to control the vehicle.

[0024] In some optional implementation schemes, the remote signal receiving device is used to monitor the change in the position of the remote driver's hands holding the steering wheel to determine the remote driver's left turn intention and right turn intention.

[0025] In some optional implementation schemes, the remote signal receiving device is used to detect that the right-hand feature is on the right side at time t0, and the center coordinates of the prediction box position are (x1, y1). At time t0+△t, the right-hand feature is detected above the steering wheel, and the center coordinates of the prediction box position are (x2, y2). The hand movement speed is calculated as Vx=((x2-x1) / △t), Vy=((y2-y1) / △t), wherein the prediction box is used to track hand movement; if Vx≥the first preset value, or Vy≥the second preset value, it is judged as an error operation. If the error operation condition is not met, it is judged that the driver is performing a left turn operation.

[0026] In some optional implementation schemes, the remote signal receiving device is used to detect that the left hand feature is on the left side at time t0', and the center coordinates of the prediction box position are (x1', y1'); at time t0'+△t, the left hand feature is detected above the steering wheel, and the center coordinates of the prediction box position are (x2', y2'); the hand movement speed is calculated as Vx'=((x2'-x1') / △t), Vy'=((y2'-y1') / △t); if Vx'=≥first preset value, or Vy'≥second preset value, it is judged as an error operation; if the error operation condition is not met, it is judged that the driver is performing a right turn operation.

[0027] In some optional implementation schemes, the remote signal receiving device is also used to receive the operation image sent by the vehicle and re-update the projected virtual driving environment after sending a control instruction to the vehicle based on the detected driving intention to control the vehicle.

[0028] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0029] The present invention performs remote control of the vehicle by simulating driving operations, can simulate a real driving environment, can provide timely feedback on the vehicle's operating status, improves operational convenience and safety, and allows the driver to control the vehicle's movement in a more accurate and safe manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1is a flow chart of a method for remotely controlling a vehicle provided by an embodiment of the present invention;

[0031] Figure 2 is a schematic diagram of a remote control vehicle system provided by an embodiment of the present invention;

[0032] Figure 3 The present invention provides a remote control personnel operation schematic diagram. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0034] In the examples of the present invention, "first", "second", etc. are used to distinguish different objects rather than to describe a specific order or sequence.

[0035] In the example of the present invention, the system for realizing remote control of a vehicle comprises two parts: a virtual driving environment and a remote control function.

[0036] The virtual driving environment mainly uses the on-board camera to identify the surrounding environment in real time, and at the same time superimposes the cockpit environment pre-stored in the vehicle with the surrounding environment to jointly construct a virtual driving environment. Figure 3 As shown, the cockpit environment includes but is not limited to the brake pedal, accelerator pedal, gear shift mechanism, steering wheel, handbrake, door, etc. Figure 2 As shown, the virtual driving environment is transmitted to the remote control personnel through wireless communication technology, and the remote control personnel should have an electronic device for receiving, which can receive communication and can project.

[0037] The remote control function is that the remote controller uses the image projected in the environment by the receiving electronic device to virtually drive the vehicle, such as operating gear shifting, stepping on the accelerator, stepping on the brake pedal, etc. At the same time, the receiving electronic device has an image recognition system that can identify the operation of the remote controller, and then send the control instructions to the vehicle through communication for control.

[0038] like Figure 1 As shown, in the example of the present invention, the remote control of the vehicle can be achieved by the following method:

[0039] Step 1: Vehicle surrounding environment monitoring: The vehicle's surrounding cameras and other sensor devices are activated to detect the environment and convert the video into digital signals and store them inside the vehicle;

[0040] Step 2: Vehicle cockpit environment simulation: The vehicle cockpit environment is stored in the vehicle in advance, including devices that can control vehicle movement, such as brakes, accelerators, gear shifts, steering, and other auxiliary devices, such as doors, lights and other image information. The cockpit environment simulation should be able to be scaled and changed according to the driver's habits;

[0041] Step 3: Vehicle signal transmission: The image or video signal of step 1 and step 2 is sent to the signal receiving device of the remote driver via a wireless radio frequency signal, such as Figure 2 A receiving device in;

[0042] Step 4: Driver posture adjustment: The remote driver fixes the signal receiving device in an appropriate position, and then uses the receiving device projection technology to project the video signal received in step 3 around the driver. The driver adjusts the cockpit environment according to the virtual driving environment and simulates the driving operation posture, such as Figure 2 Driving posture in the middle, ready to control the vehicle;

[0043] Step 5: Remote driving operation: After system authentication and remote control permission, the driver simulates the operation of the steering wheel, accelerator, gear shifting, etc. according to the cockpit image transmitted by the vehicle;

[0044] Step 6: Driving intention detection: The receiving device should be able to perform driver monitoring, such as Figure 2 Monitoring devices in Figure 3 When the driver operates the steering wheel to turn and moves from point A to point B, the driver monitoring system will detect the change in the position of the driver's hand on the steering wheel and detect the hand features. When it is detected that the right hand feature is on the right at time t0, the center coordinates of the predicted bounding box position are (x1, y1). At time t0+△t, the device detects that the right hand feature is above the steering wheel, and the center coordinates of the predicted bounding box position are (x2, y2). The hand movement speed is calculated as: Vx=((x2-x1) / △t), Vy=((y2-y1) / △t). Considering the actual possible misoperation, the excessive speed should be ignored. When Vx=((x2-x1) / △t)≥the first preset value (such as 0.5m / s) or Vy=((y2-y1) / △t)≥the second preset value (such as 0.5m / s), it is judged as a misoperation. When the system does not meet the above conditions, it is in accordance with Figure 3 It should be determined that the driver is turning left, and the monitoring device continues to convert the judgment logic into a driver's operation intention control signal. The monitoring device may be a camera or other device that can identify changes in the driver's posture;

[0045] At t0', the left hand feature is detected on the left side, and the center coordinates of the prediction box position are (x1', y1'). At t0'+△t, the left hand feature is detected above the steering wheel, and the center coordinates of the prediction box position are (x2', y2'). The hand movement speed is calculated as Vx'=((x2'-x1') / △t), Vy'=((y2'-y1') / △t);

[0046] If Vx'=≥ the first preset value, or Vy'≥ the second preset value, it is determined to be an erroneous operation. If the erroneous operation condition is not met, it is determined that the driver is performing a right turn operation.

[0047] Step 7: Vehicle control: The receiving device sends the driver's operation intention in step 6 to the vehicle. On the one hand, the vehicle works according to the operation intention, such as steering and braking. On the other hand, the vehicle transmits the operation image, such as the cockpit image that changes with the steering wheel turning, back to the receiving device. The receiving device re-updates the projected virtual driving environment to facilitate the driver to observe the vehicle status in real time.

[0048] It should be pointed out that, according to the needs of implementation, the various steps / components described in this application can be split into more steps / components, and two or more steps / components or partial operations of steps / components can be combined into new steps / components to achieve the purpose of the present invention.

[0049] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for remotely controlling a vehicle, characterized in that: include: storing the detected vehicle surrounding environment information inside the vehicle, storing the vehicle cabin environment in advance inside the vehicle, and sending the vehicle surrounding environment information and the vehicle cabin environment information to a remote signal receiving device; After the position of the remote signal receiving device is fixed, the vehicle surrounding environment information and the vehicle cockpit environment information are projected around the remote driver using the remote signal receiving device projection technology, so that the remote driver can perform remote control according to the virtual driving environment and cockpit environment; After system authentication and remote control permission, remote driving operations are performed based on the vehicle's surrounding environment information and vehicle cabin environment information, and the remote driver's driving intention is monitored, and the detected driving intention is sent to the vehicle to control the vehicle; Monitor the driving intention of the remote driver, including: Monitor the changes in the remote driver's hand position on the steering wheel to determine the remote driver's left turn intention and right turn intention; Monitor the changes in the remote driver's hand position on the steering wheel to determine the remote driver's intention to turn left, including: At time t0, the right hand feature is detected on the right side, and the center coordinates of the prediction box position are (x1, y1). At time t0+△t, the right hand feature is detected above the steering wheel, and the center coordinates of the prediction box position are (x2, y2). The hand movement speed is calculated as Vx = ((x2-x1) / △t), Vy = ((y2-y1) / △t), where the prediction box is used to track the hand movement; If Vx≥the first preset value, or Vy≥the second preset value, it is determined to be an erroneous operation. If the erroneous operation condition is not met, it is determined that the driver is performing a left turn operation.

2. The method according to claim 1, characterized in that Monitor the changes in the remote driver's hand position on the steering wheel to determine the remote driver's intention to turn right, including: At t0', the left hand feature is detected on the left side, and the center coordinates of the prediction box position are (x1', y1'). At t0'+△t, the left hand feature is detected above the steering wheel, and the center coordinates of the prediction box position are (x2', y2'). The hand movement speed is calculated as Vx'=((x2'-x1') / △t), Vy'=((y2'-y1') / △t); If Vx'=≥ the first preset value, or Vy'≥ the second preset value, it is determined to be an erroneous operation. If the erroneous operation condition is not met, it is determined that the driver is performing a right turn operation.

3. The method according to claim 2, characterized in that After sending a control instruction to the vehicle based on the detected driving intention to control the vehicle, the method further includes: Receive the operation images sent by the vehicle and re-update the projected virtual driving environment.

4. A remote control vehicle system, characterized in that: include: A virtual driving environment generation module is used to store the detected vehicle surrounding environment information inside the vehicle and store the vehicle cabin environment inside the vehicle in advance; A wireless communication module, used to send vehicle surrounding environment information and vehicle cabin environment information to a remote signal receiving device; A remote signal receiving device, which is used to project the vehicle's surrounding environment information and the vehicle's cockpit environment information around the remote driver using projection technology after its position is fixed, so that the remote driver can perform remote control according to the virtual driving environment and cockpit environment; The remote signal receiving device is also used to perform remote driving operations according to the vehicle's surrounding environment information and vehicle cabin environment information after system authentication and remote control permission, and monitor the driving intention of the remote driver, and send control instructions to the vehicle based on the detected driving intention to control the vehicle; A remote signal receiving device, used to monitor the change in the position of the remote driver's hand holding the steering wheel, so as to determine the remote driver's intention to turn left or right; A remote signal receiving device is used to detect that the right hand feature is on the right side at time t0, and the center coordinates of the prediction box position are (x1, y1). At time t0+△t, the right hand feature is detected above the steering wheel, and the center coordinates of the prediction box position are (x2, y2). The hand movement speed is calculated as Vx=((x2-x1) / △t), Vy=((y2-y1) / △t), wherein the prediction box is used to track hand movement; if Vx≥a first preset value, or Vy≥a second preset value, it is judged as an erroneous operation. If the erroneous operation condition is not met, it is judged that the driver is performing a left turn operation.

5. The system according to claim 4, characterized in that A remote signal receiving device is used to detect that the left hand feature is on the left side at time t0', and the center coordinates of the prediction box position are (x1', y1'). At time t0'+△t, the left hand feature is detected above the steering wheel, and the center coordinates of the prediction box position are (x2', y2'). The hand movement speed is calculated as Vx'=((x2'-x1') / △t), Vy'=((y2'-y1') / △t); if Vx'=≥first preset value, or Vy'≥second preset value, it is judged as an erroneous operation. If the erroneous operation condition is not met, it is judged that the driver is performing a right turn operation.

6. The system according to claim 5, characterized in that The remote signal receiving device is also used to receive the operation image sent by the vehicle and re-update the projected virtual driving environment after sending the control instruction of the detected driving intention to the vehicle to control the vehicle.

Images