A method, system, device and computer-readable storage medium for controlling a vehicle based on a mobile phone

Abstract

The present invention relates to a method, system, device and computer-readable storage medium for controlling a vehicle based on a mobile phone. The method includes step S1, keeping the vehicle gear in the forward gear, and the driver opens a dedicated mobile phone APP; S2, judging whether the driver is concentrating and operating the mobile phone's instructions through the mobile phone to control the vehicle; S3, judging whether the vehicle is in automatic driving mode, if so, entering step S4; if not, turning to step S2; S4, the vehicle performs automatic driving, entering step S6; S5, the vehicle exits the automatic driving mode and stops, entering step S2; S6, judging whether the vehicle has arrived at the destination, if so, entering step S7, if not, turning to step S2; S7, end. The present invention proposes a method, system, device and computer-readable storage medium for controlling a vehicle based on a mobile phone, which facilitates the driver to realize remote control of the vehicle and improves the user experience.

Description

Technical Field

[0001] The present invention relates to the field of vehicle intelligent driving technology, and in particular to a method, system, device and computer-readable storage medium for controlling a vehicle based on a mobile phone. Background Art

[0002] As vehicle autonomous driving technology continues to develop, its applications are becoming more widespread. However, due to the restrictions of laws and regulations, as well as the challenges of complex road traffic environments, drivers still need to continuously monitor the vehicle during autonomous driving, and take over and control the vehicle when necessary.

[0003] In some application scenarios of autonomous driving technology, the driver is not in the cockpit but is operating outside the cockpit, such as automatic valet parking technology and remote vehicle summoning technology. For automatic valet parking technology, the driver and passengers can get off the car directly when they arrive at the destination, leaving the subsequent parking work entirely to the vehicle itself. For remote vehicle summoning technology, the vehicle can be summoned to the driver's designated location. The maturity of these two technologies not only helps drivers save the energy of finding parking spaces and parking, but also reduces the time it takes for drivers to travel to and from the parking lot. Similarly, for this type of autonomous driving technology where the driver is outside the cabin, the driver is still required to be responsible for the vehicle's driving safety through remote monitoring.

[0004] The current methods used to require drivers to stay focused while remotely controlling an autonomous vehicle have significant drawbacks:

[0005] In the cloud-based smart parking system created by WM Motor, users need to continuously draw circles on the phone screen during the autonomous parking process. Once the finger leaves the screen, the parking will stop. In Tesla's automatic remote parking, the driver needs to press and hold the button to control the movement of the vehicle. In Roewe's Marvel X model, the finger needs to be swiped continuously on the mobile phone app to allow the vehicle to continue the automatic driving behavior.

[0006] The above existing common methods for drivers to monitor and control autonomous vehicles outside the cabin all rely on the driver to continuously interact with the mobile phone screen to ensure the driver's concentration. There are the following disadvantages:

[0007] (1) It requires the driver to continue manual operation during the parking process, which may last for several minutes or even more than ten minutes, which puts a great burden on the driver;

[0008] (2) If the driver accidentally releases the button, the system will determine that the driver is not paying attention or intends to stop the vehicle, and will stop the vehicle, which will cause trouble to the driver and a very poor user experience;

[0009] (3) The method of judging whether the driver is paying attention based on whether the hands perform repetitive touch operations is not appropriate. The driver can pay attention to things other than the autonomous driving vehicle while performing the required operations with his hands, which is inconsistent with the system's requirement for the driver to concentrate. Summary of the invention

[0010] In view of the above problems in the prior art, the present invention proposes a method, system, device and computer-readable storage medium for controlling a vehicle based on a mobile phone, which facilitates the driver to remotely control the vehicle and improves the user experience.

[0011] Specifically, the present invention proposes a method for controlling a vehicle based on a mobile phone, comprising the steps of:

[0012] S1, keep the vehicle gear in the forward gear, and the driver opens the dedicated mobile phone APP;

[0013] S2, determining whether the driver is concentrating through the mobile phone and operating the mobile phone to control the vehicle;

[0014] If the driver is paying attention and the command operation is to enter the automatic driving mode, the process proceeds to step S4; if the driver is paying attention and there is no command operation, the process proceeds to step S3; if the driver is paying attention and the command operation is to exit the automatic driving mode, the process proceeds to step S5; if the driver is not paying attention, the process proceeds to step S5;

[0015] S3, determining whether the vehicle is in the automatic driving mode, if so, proceeding to step S4; if not, proceeding to step S2;

[0016] S4, the vehicle performs automatic driving and proceeds to step S6;

[0017] S5, the vehicle exits the automatic driving mode and stops, notifies the driver via a mobile phone, and proceeds to step S2;

[0018] S6, determining whether the vehicle has arrived at the destination, if so, proceeding to step S7, if not, proceeding to step S2;

[0019] S7, end.

[0020] According to one embodiment of the present invention, in step S2, judging whether the driver is paying attention comprises the following steps:

[0021] S201, determining whether the driver is wearing sunglasses; if so, proceeding to step S202, if not, proceeding to step S203;

[0022] S202, monitoring the driver's head posture, and proceeding to step S204;

[0023] S203, monitoring the driver's sight direction, and proceeding to step S205;

[0024] S204, if the driver's head posture is not facing the mobile phone and the duration is greater than or equal to t1, then proceed to step S210; if not, then proceed to step S206;

[0025] S205, if the driver's line of sight is not towards the mobile phone and the duration is greater than or equal to t1, then proceed to step S210; if not, then proceed to step S207;

[0026] S206, if the driver's head posture is not facing the mobile phone and the duration is greater than t2, then proceed to step S208, if not, then proceed to step S209;

[0027] S207, if the driver's line of sight is not towards the mobile phone and the duration is greater than t2, then proceed to step S208, if not, then proceed to step S209;

[0028] S208, prompting the driver to concentrate through the mobile phone;

[0029] S209, determining that the driver is paying attention, and proceeding to step S211;

[0030] S210, determining that the driver is not paying attention;

[0031] S211, end.

[0032] According to one embodiment of the present invention, t1=1s, t2=0.5s.

[0033] According to one embodiment of the present invention, in step S1, the dedicated APP is used to display real-time video of the vehicle and its surroundings.

[0034] According to an embodiment of the present invention, in step S2, the instruction operation includes instruction key input or instruction voice input.

[0035] According to one embodiment of the present invention, in step 208, the mobile phone gives a voice prompt to the driver.

[0036] The present invention also provides a system for controlling a vehicle based on a mobile phone, using the above-mentioned method for controlling a vehicle based on a mobile phone, and the system comprises:

[0037] A mobile phone driver monitoring module, comprising a mobile phone front camera and a monitoring processor, wherein the mobile phone front camera is used to capture the driver's head posture and sight direction, and the monitoring processor determines whether the driver is concentrating according to the captured information of the mobile phone front camera;

[0038] A mobile phone human-computer interaction module, comprising a vehicle information display device, a command input device and a voice prompt device, wherein the vehicle information display device is used to display a real-time video of the vehicle and its surroundings, the command input device is used to obtain command operations on the mobile phone, and the voice prompt device is used to prompt the driver;

[0039] A vehicle-side driving decision module generates a driving decision instruction according to a judgment result of whether the driver is concentrating by the mobile phone-side driver monitoring module and an instruction operation obtained by the instruction input device;

[0040] A vehicle-side execution module, used to receive and execute the driving decision instruction to realize the control of the vehicle;

[0041] The vehicle-side environment perception module communicates with the mobile phone-side driver monitoring module, the mobile phone-side human-computer interaction module and the vehicle-side driving decision-making module. The vehicle-side environment perception module is used to obtain dynamic information of the vehicle and its surrounding environment.

[0042] According to one embodiment of the present invention, the vehicle-side environmental perception module includes an environmental perception camera, a millimeter-wave radar and a bus data set, the environmental perception camera obtains real-time video of the vehicle and its surrounding environment, the millimeter-wave radar is used to detect the vehicle's surrounding environment, and the bus data set is used to record the vehicle condition.

[0043] The present invention also provides a device for controlling a vehicle based on a mobile phone, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the aforementioned method when executing the computer program.

[0044] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the aforementioned method when executed by a processor.

[0045] The present invention provides a method, system, device and computer-readable storage medium for controlling a vehicle based on a mobile phone, which ensures the safety of the automatic driving function by judging the driver's attention, facilitates the driver to realize remote control of the vehicle, and improves the user experience.

[0046] It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings are included to provide further explanation of the present invention, and they are incorporated into and constitute a part of this application. The accompanying drawings illustrate embodiments of the present invention and together with this specification serve to explain the principles of the present invention. In the accompanying drawings:

[0048] Figure 1 A flowchart of a method for controlling a vehicle based on a mobile phone according to an embodiment of the present invention is shown.

[0049] Figure 2 A schematic structural diagram of a system for controlling a vehicle based on a mobile phone according to an embodiment of the present invention is shown.

[0050] Figure 3 A schematic diagram of a mobile phone interface according to an embodiment of the present invention is shown.

[0051] The above drawings include the following reference numerals:

[0052] Mobile 201

[0053] Car end 202

[0054] Mobile driver monitoring module 203

[0055] Mobile terminal human-computer interaction module 204

[0056] Vehicle-side driving decision module 205

[0057] Vehicle-side execution module 206

[0058] Vehicle-side environment perception module 207

[0059] Mobile phone front camera 208

[0060] Monitoring processor 209

[0061] Vehicle information display device 210

[0062] Command input device 211

[0063] Voice prompt device 212

[0064] Environmental perception camera 213

[0065] Millimeter wave radar 214

[0066] Bus Dataset 215

[0067] Lens 301

[0068] Vehicle real-time video 302

[0069] Vehicle status information 303

[0070] Voice prompt 304

[0071] Command input keyboard 305 DETAILED DESCRIPTION

[0072] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application may be combined with each other.

[0073] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present application and its application or use. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0074] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0075] Unless otherwise specifically stated, the relative arrangement, numerical expressions and numerical values ​​of the parts and steps set forth in these embodiments do not limit the scope of the application. Meanwhile, it should be understood that, for ease of description, the sizes of the various parts shown in the accompanying drawings are not drawn according to actual proportional relationships. The technology, methods and equipment known to those of ordinary skill in the relevant art may not be discussed in detail, but in appropriate cases, the technology, methods and equipment should be considered as a part of the specification. In all examples shown and discussed here, any specific value should be interpreted as being merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters represent similar items in the following drawings, and therefore, once a certain item is defined in an accompanying drawing, it does not need to be further discussed in subsequent drawings.

[0076] In the description of the present application, it should be understood that the directions or positional relationships indicated by directional words such as "front, back, up, down, left, right", "lateral, vertical, perpendicular, horizontal" and "top, bottom" are usually based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present application and simplifying the description. Unless otherwise specified, these directional words do not indicate or imply that the device or element referred to must have a specific direction or be constructed and operated in a specific direction, and therefore cannot be understood as limiting the scope of protection of the present application; the directional words "inside and outside" refer to the inside and outside relative to the contours of each component itself.

[0077] In addition, it should be noted that the use of words such as "first" and "second" to define components is only for the convenience of distinguishing the corresponding components. If not otherwise stated, the above words have no special meaning and cannot be understood as limiting the scope of protection of this application. In addition, although the terms used in this application are selected from well-known and commonly used terms, some terms mentioned in the specification of this application may be selected by the applicant at his or her discretion, and their detailed meanings are explained in the relevant parts of the description of this article. In addition, it is required to understand this application not only by the actual terms used, but also by the meaning implied by each term.

[0078] Figure 1 The flowchart of a method for controlling a vehicle based on a mobile phone according to an embodiment of the present invention is shown. As shown in the figure, the present invention provides a method for controlling a vehicle based on a mobile phone, comprising the steps of:

[0079] S1, keep the vehicle gear in the forward gear, and the driver opens the dedicated mobile phone APP. The dedicated mobile phone APP is used to maintain communication with the vehicle, and the mobile phone interface displays the interface containing the dedicated APP. As an example but not limitation, the driver can be the current driver of the vehicle, or other operators, as long as they have the right to control the vehicle, and perform operations such as parking on the vehicle. Conventionally, the driver can leave the vehicle cabin to operate the mobile phone, or operate at any position in the vehicle cabin, and the operation method is contactless.

[0080] S2, the vehicle is controlled by judging whether the driver is paying attention and the instructions of the mobile phone through the mobile phone. In short, there are two factors for vehicle control, one is the driver's attention, and the other is the instructions received by the mobile phone. The following four conditions are generated by these two factors:

[0081] If the driver is paying attention and instructs the operation to enter the automatic driving mode, then the process goes to step S4; if this condition is met, it means that the vehicle is under driver monitoring and the driver determines that the vehicle and the surrounding environment are normal, and automatic driving can be performed.

[0082] If the driver is paying attention and has not given any instructions, the process proceeds to step S3; if this condition is met, it means that the vehicle is under driver monitoring and the driver approves the current vehicle status.

[0083] If the driver is paying attention and instructs the operation to exit the automatic driving mode, the process proceeds to step S5; if this condition is met, it means that the vehicle is under driver monitoring, but the driver determines that the vehicle is in an emergency (the vehicle itself fails, or encounters environmental factors that the vehicle in automatic driving cannot handle) and needs to take over, so the driver takes over the vehicle and controls the vehicle to stop.

[0084] If the driver is not paying attention, the process goes to step S6; if this condition is met, it means that the vehicle is out of the driver's monitoring. At this time, the vehicle will switch out of the automatic driving mode and stop to reduce the risk. Only when the driver refocuses on the vehicle monitoring can he regain the initiative in driving the vehicle.

[0085] S3, determining whether the vehicle is in the automatic driving mode, if so, proceeding to step S4; if not, proceeding to step S2;

[0086] S4, the vehicle performs automatic driving and enters step S6;

[0087] S5, the vehicle exits the automatic driving mode and stops, notifies the driver via the mobile phone, and proceeds to step S2;

[0088] S6, determine whether the vehicle has arrived at the destination, if yes, proceed to step S7, if no, proceed to step S2;

[0089] S7, end.

[0090] Preferably, in step S2, judging whether the driver is concentrating includes the following steps:

[0091] S201, determine whether the driver is wearing sunglasses; if so, proceed to step S202, if not, proceed to step S203;

[0092] S202, monitor the driver's head posture, and proceed to step S204;

[0093] S203, monitoring the driver's sight direction, and proceeding to step S205;

[0094] S204, if the driver's head posture is not facing the mobile phone and the duration is greater than or equal to t1, then proceed to step S210; if not, then proceed to step S206;

[0095] S205, if the driver's line of sight is not towards the mobile phone and the duration is greater than or equal to t1, then proceed to step S210; if not, then proceed to step S207;

[0096] S206, if the driver's head posture is not facing the mobile phone and the duration is greater than t2, then proceed to step S208, if not, then proceed to step S209;

[0097] S207, if the driver's line of sight is not towards the mobile phone and the duration is greater than t2, then proceed to step S208, if not, then proceed to step S209;

[0098] S208, reminding the driver to concentrate via mobile phone;

[0099] S209, determining that the driver is paying attention, and proceeding to step S211;

[0100] S210, determining that the driver is not paying attention;

[0101] S211, end.

[0102] It is easy to understand that to determine whether the driver is paying attention, first determine whether the driver is wearing sunglasses. If the driver is not wearing sunglasses, the driver's attention direction is determined by the direction of his line of sight facing the mobile phone; if the driver is wearing sunglasses, the information of the line of sight is blocked and cannot be read, and the driver's attention direction is determined based on the driver's head posture facing the mobile phone. Once the head posture direction or the line of sight leaves the mobile phone for more than the threshold t1, it is determined that the driver is not paying attention and enters the next link of the process; if the time when the driver's head posture direction or the line of sight leaves the mobile phone is less than or equal to t2, it is determined that the driver is paying attention and enters the next link of the process. When the time when the driver's head posture direction or the line of sight leaves the mobile phone is greater than t2 and less than t1, the driver is prompted to pay attention through the mobile phone voice, but the result of the concentration judgment is still output to the next link to provide a buffer for the driver's operation. The corresponding measures refer to Table 1.

[0103] Table 1

[0104] Time of attention away from mobile phone t (s) System output judgment signal Other Tips 0≤t<t2 Focus / t2<t<t1 Focus Voice prompts for drivers t1≤t Poor concentration /

[0105] Preferably, t1 = 1s, t2 = 0.5s. In practical applications, the values ​​of t1 and t2 can be personalized according to the actual situation of the driver, or can be recommended based on further scientific research results in the future. If necessary, different thresholds can be set for head posture or line of sight direction.

[0106] Preferably, in step S1, a dedicated APP is used to display real-time video of the vehicle and its surroundings, so that the driver can observe the operation of the vehicle in real time.

[0107] Preferably, in step S2, the command operation includes command key input or command voice input. The dedicated mobile phone APP can provide two command operation modes, one is to input the operation command through the keys on the mobile phone, and the other is to input the operation command through the voice recognition technology on the mobile phone.

[0108] Preferably, in step 208, the mobile phone gives a voice prompt to the driver. As an example and not a limitation, the mobile phone may also display the prompt information on the screen.

[0109] The present invention also provides a system for controlling a vehicle based on a mobile phone, and adopts a method for controlling a vehicle based on a mobile phone. Figure 2 The schematic diagram of the structure of a system for controlling a vehicle based on a mobile phone according to an embodiment of the present invention is shown. As shown in the figure, the system mainly includes a mobile phone terminal 201 and a vehicle terminal 202. The mobile phone terminal 201 includes a mobile phone driver monitoring module 203 and a mobile phone human-computer interaction module 204, and the vehicle terminal 202 includes a vehicle-side driving decision module 205, a vehicle-side execution module 206 and a vehicle-side environment perception module 207. The specific description is as follows:

[0110] The mobile phone driver monitoring module 203 includes a mobile phone front camera 208 and a monitoring processor 209. The mobile phone front camera 208 is used to capture the driver's head posture and line of sight, and the monitoring processor 209 determines whether the driver is concentrating based on the captured information of the mobile phone front camera 208. With the development of technology, the functions of the mobile phone front camera 208 are becoming more and more perfect, especially the use of infrared binocular camera technology, which makes it technically feasible to complete the above functions through the mobile phone front camera 208.

[0111] The mobile phone human-computer interaction module 204 includes a vehicle information display device 210, a command input device 211 and a voice prompt device 212. The vehicle information display device 210 is used to display a real-time video of the vehicle and its surroundings. The command input device is used to obtain command operations on the mobile phone. The voice prompt device 212 is used to prompt the driver. Figure 3A schematic diagram of a mobile phone interface of an embodiment of the present invention is shown. As shown in the figure, the mobile phone interface 300 includes a dedicated APP interface, and the mobile phone interface 300 includes a lens 301 of the front camera 208 of the mobile phone, a real-time video of the vehicle 302, vehicle status information 303 (such as vehicle speed), a voice prompt 304 and a command input keyboard 305 from top to bottom. The real-time video of the vehicle 302 includes a real-time video of the vehicle and the surrounding environment. In this embodiment, the vehicle environment and the virtual body are superimposed at a top-down angle to display the on-site environment; different viewing angles can also be switched to facilitate the driver to observe the distant view in a certain direction. The vehicle status information 303 is superimposed on the real-time video 302 to facilitate the driver to observe the vehicle status. For easy understanding, the voice prompt 304 is expressed in the form of a text box. The voice prompt 304 is issued when the driver is not paying attention, the vehicle is stopped or in other states. The command input keyboard 305 can be a touch keyboard, and the driver inputs the operation command by touching the key to control the start and stop of the vehicle. The command input keyboard 305 can also use traditional keys to input commands. In addition to using the command input keyboard, the command input device can also use voice input.

[0112] The vehicle-side driving decision module 205 generates a driving decision instruction based on the judgment result of whether the driver is concentrating by the mobile phone-side driver monitoring module 203 and the instruction operation obtained by the instruction input device. The decision instruction includes stopping, continuing driving, turning, etc.

[0113] The vehicle-side execution module 206 is used to receive and execute driving decision instructions to control the vehicle. The vehicle-side execution module 206 includes a brake pedal, an accelerator pedal, a steering mechanism, etc.

[0114] The vehicle-side environment perception module 207 communicates with the mobile phone-side driver monitoring module 203, the mobile phone-side human-computer interaction module 204 and the vehicle-side driving decision module 205. The vehicle-side environment perception module 207 is used to obtain dynamic information of the vehicle and its surrounding environment, which also includes the vehicle speed, location information, and the types and locations of surrounding traffic participants. The vehicle information display device 210 displays real-time video of the vehicle and its surrounding environment through the dynamic information captured by the vehicle-side environment perception module 207.

[0115] Preferably, the vehicle-side environment perception module 207 includes an environment perception camera 213, a millimeter wave radar 214 and a bus data set 215. The environment perception camera 213 obtains real-time video of the vehicle and its surroundings, the millimeter wave radar 214 is used to detect the vehicle's surroundings, and the bus data set 215 is used to record the vehicle's condition, including information such as vehicle speed and location.

[0116] The present invention also provides a device for controlling a vehicle based on a mobile phone, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of any one of the aforementioned methods for controlling a vehicle based on a mobile phone are implemented.

[0117] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the aforementioned methods for controlling a vehicle based on a mobile phone are implemented.

[0118] Among them, the specific implementation methods and technical effects of the device for controlling a vehicle based on a mobile phone and the computer-readable storage medium can be found in the embodiments of the method for controlling a vehicle based on a mobile phone provided by the present invention, and will not be repeated here.

[0119] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of the two. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps are generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. The technician may implement the described functionality in different ways for each specific application, but such implementation decisions should not be interpreted as resulting in a departure from the scope of the present invention.

[0120] The various illustrative logic modules and circuits described in conjunction with the embodiments disclosed herein may be implemented or executed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in an alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in cooperation with a DSP core, or any other such configuration.

[0121] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read and write information from / to the storage medium. In an alternative, a storage medium may be integrated into a processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and the storage medium may reside in a user terminal as discrete components.

[0122] In one or more exemplary embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented as a computer program product in software, each function may be stored on or transmitted by a computer-readable medium as one or more instructions or codes. Computer-readable media include both computer storage media and communication media, including any media that facilitates the transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a computer. As an example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer. Any connection is also properly referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of the medium. Disk and disc as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0123] The present invention provides a method, system, device and computer-readable storage medium for controlling a vehicle based on a mobile phone, which ensures the safety of the automatic driving function by judging the driver's attention, facilitates the driver to realize remote control of the vehicle, and improves the user experience, and has the following advantages:

[0124] (1) More accurately judge the driver’s attention state.

[0125] (2) Eliminate the large amount of extra work that the driver has to do in order to determine the driver's status in the prior art.

[0126] (3) It eliminates the possibility that the driver may accidentally operate a button, causing the vehicle to perform an action against the driver’s will.

[0127] (4) Drivers can remotely control vehicles more accurately and flexibly.

[0128] It will be apparent to those skilled in the art that various modifications and variations may be made to the above exemplary embodiments of the present invention without departing from the spirit and scope of the present invention. Therefore, it is intended that the present invention covers modifications and variations of the present invention that fall within the scope of the appended claims and their equivalent technical solutions.

Claims

1. A method for controlling a vehicle based on a mobile phone, comprising the steps of: S1, keep the vehicle gear in the forward gear, and the driver opens the dedicated mobile phone APP; S2, determining whether the driver is concentrating through the mobile phone and operating the mobile phone to control the vehicle; If the driver is paying attention and the command operation is to enter the automatic driving mode, the process proceeds to step S4; if the driver is paying attention and there is no command operation, the process proceeds to step S3; if the driver is paying attention and the command operation is to exit the automatic driving mode, the process proceeds to step S5; if the driver is not paying attention, the process proceeds to step S5; S3, determining whether the vehicle is in the automatic driving mode, if so, proceeding to step S4; if not, proceeding to step S2; S4, the vehicle performs automatic driving and proceeds to step S6; S5, the vehicle exits the automatic driving mode and stops, notifies the driver via a mobile phone, and proceeds to step S2; S6, determining whether the vehicle has arrived at the destination, if so, proceeding to step S7, if not, proceeding to step S2; S7, end; Wherein, in step S2, judging whether the driver is paying attention includes the following steps: S201, determining whether the driver is wearing sunglasses; if so, proceeding to step S202, if not, proceeding to step S203; S202, monitoring the driver's head posture, and proceeding to step S204; S203, monitoring the driver's sight direction, and proceeding to step S205; S204, if the driver's head posture is not facing the mobile phone and the duration is greater than or equal to t1, then proceed to step S210; if not, then proceed to step S206; S205, if the driver's line of sight is not towards the mobile phone and the duration is greater than or equal to t1, then proceed to step S210; if not, then proceed to step S207; S206, if the driver's head posture is not facing the mobile phone and the duration is greater than t2, then proceed to step S208, if not, then proceed to step S209; S207, if the driver's line of sight is not towards the mobile phone and the duration is greater than t2, then proceed to step S208, if not, then proceed to step S209; S208, prompting the driver to concentrate through the mobile phone; S209, determining that the driver is paying attention, and proceeding to step S211; S210, determining that the driver is not paying attention; S211, end; Among them, t2<t1.

2. The method for controlling a vehicle based on a mobile phone as claimed in claim 1, characterized in that: t1=1s, t2=0.5s.

3. The method for controlling a vehicle based on a mobile phone as claimed in claim 1, characterized in that: In step S1, the dedicated APP is used to display real-time video of the vehicle and its surroundings.

4. The method for controlling a vehicle based on a mobile phone as claimed in claim 1, characterized in that: In step S2, the instruction operation includes instruction key input or instruction voice input.

5. The method for controlling a vehicle based on a mobile phone as claimed in claim 1, characterized in that: In step 208, the mobile phone gives a voice prompt to the driver.

6. A system for controlling a vehicle based on a mobile phone, using the method for controlling a vehicle based on a mobile phone as claimed in claim 1, characterized in that: The system comprises: A mobile phone driver monitoring module, comprising a mobile phone front camera and a monitoring processor, wherein the mobile phone front camera is used to capture the driver's head posture and sight direction, and the monitoring processor determines whether the driver is concentrating according to the captured information of the mobile phone front camera; A mobile phone human-computer interaction module, comprising a vehicle information display device, a command input device and a voice prompt device, wherein the vehicle information display device is used to display a real-time video of the vehicle and its surroundings, the command input device is used to obtain command operations on the mobile phone, and the voice prompt device is used to prompt the driver; A vehicle-side driving decision module generates a driving decision instruction according to a judgment result of whether the driver is concentrating by the mobile phone-side driver monitoring module and an instruction operation obtained by the instruction input device; A vehicle-side execution module, used for receiving and executing the driving decision instruction to realize the control of the vehicle; The vehicle-side environment perception module communicates with the mobile phone-side driver monitoring module, the mobile phone-side human-computer interaction module and the vehicle-side driving decision-making module. The vehicle-side environment perception module is used to obtain dynamic information of the vehicle and its surrounding environment.

7. The system for controlling a vehicle based on a mobile phone as claimed in claim 6, characterized in that: The vehicle-side environmental perception module includes an environmental perception camera, a millimeter-wave radar and a bus data set. The environmental perception camera obtains real-time video of the vehicle and its surrounding environment, the millimeter-wave radar is used to detect the vehicle's surrounding environment, and the bus data set is used to record the vehicle's condition.

8. A device for controlling a vehicle based on a mobile phone, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images