Parking method, device, system, storage medium, vehicle and mobile terminal

Abstract

Embodiments of the present application provide a parking method, device, system, storage medium, vehicle and mobile terminal. The parking method comprises: acquiring real-time environment information of a vehicle; the real-time environment information comprises real-time environment images at multiple observation points of the vehicle; determining a real-time automatic driving state of the vehicle according to the real-time environment information; when it is determined that the real-time automatic driving state is an abnormal state, controlling the vehicle to stop moving, and sending abnormal reminding information to a mobile terminal, so that the mobile terminal displays the abnormal reminding information; when a viewing request of the mobile terminal for the abnormal reminding information is received, sending the multiple real-time environment images to the mobile terminal after being synthesized into a real-time panoramic image, so that the mobile terminal displays the real-time panoramic image. Embodiments of the present application optimize the hardware resources of the vehicle, effectively reduce the program lag probability in the automatic driving process of the vehicle, and improve the accuracy and safety of automatic parking.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and more specifically, to a parking method, apparatus, system and storage medium, vehicle and mobile terminal. Background Technology

[0002] Automated parking is one direction for the commercial application of autonomous driving technology in specific parking lots, such as enabling one-click automatic car retrieval or return in specific locations and scenarios.

[0003] In real-world applications, automated parking systems often encounter situations that are difficult for them to handle. For example, a vehicle ahead may pull over and stop in a parking lot, or temporary traffic cones may be placed along the path. These situations can cause the vehicle in autonomous driving mode to become unresponsive and enter an abnormal state, thus requiring human intervention.

[0004] Panoramic images are valuable reference information for human intervention, but generating panoramic images requires a large proportion of hardware resources. This can cause program lag during the vehicle's autonomous driving process, leading to untimely or even erroneous responses, affecting the efficiency and safety of automatic parking. Summary of the Invention

[0005] This application addresses the shortcomings of existing methods by proposing a parking method, device, system, storage medium, vehicle, and mobile terminal. The aim is to solve the technical problem that generating panoramic images requires a large proportion of hardware resources, which can cause program lag during autonomous driving, affecting the efficiency and safety of automatic parking.

[0006] In a first aspect, embodiments of this application provide a parking method, including:

[0007] Acquire real-time environmental information of the vehicle; real-time environmental information includes real-time environmental images located at multiple observation points on the vehicle;

[0008] The real-time autonomous driving status of the vehicle is determined based on real-time environmental information;

[0009] When the real-time autonomous driving status is determined to be abnormal, the vehicle is controlled to stop moving, and an abnormality alert is sent to the mobile terminal so that the mobile terminal can display the abnormality alert.

[0010] When a request to view an anomaly alert is received from a mobile terminal, multiple real-time environmental images are combined into a real-time panoramic image and sent to the mobile terminal, allowing the mobile terminal to display the real-time panoramic image.

[0011] Secondly, embodiments of this application provide a parking device, including:

[0012] The environmental information acquisition module is used to acquire real-time environmental information of the vehicle; the real-time environmental information includes real-time environmental images located at multiple observation points on the vehicle.

[0013] The environmental information analysis and vehicle control module is used to determine the real-time autonomous driving status of the vehicle based on real-time environmental information; when the real-time autonomous driving status is determined to be abnormal, the module controls the vehicle to stop moving and sends an abnormality reminder to the mobile terminal; the mobile terminal displays the abnormality reminder; when a request to view the abnormality reminder is received from the mobile terminal, the module combines multiple real-time environmental images into a real-time panoramic image and sends it to the mobile terminal, so that the mobile terminal displays the real-time panoramic image.

[0014] The panoramic image processing module is used to synthesize multiple real-time environmental images into a real-time panoramic image and send it to the mobile terminal when it receives a viewing request for abnormal reminder information from the mobile terminal, so that the mobile terminal can display the real-time panoramic image.

[0015] Thirdly, embodiments of this application provide a vehicle, including: a vehicle body, a power system, a steering system, an on-board controller, a plurality of image acquisition devices, and an on-board signal transceiver;

[0016] Several image acquisition devices are respectively set up at multiple observation points on the vehicle body;

[0017] Vehicle-mounted transceivers are used for communication connections with mobile terminals;

[0018] The powertrain, steering system, several image acquisition units, and vehicle signal transceivers are all connected to the vehicle controller for communication.

[0019] The vehicle controller is configured with a computer program, which, when executed by the vehicle controller, implements the parking method provided in the first aspect above.

[0020] Fourthly, embodiments of this application provide a parking method, including:

[0021] When an abnormal alert is received from the vehicle, the abnormal alert is displayed. The abnormal alert is sent when the vehicle determines that the real-time autonomous driving state of the vehicle is abnormal based on real-time environmental information, which includes real-time environmental images located at multiple observation points of the vehicle.

[0022] Upon receiving a viewing instruction for an anomaly alert, the instruction is converted into a viewing request and sent to the vehicle.

[0023] When a real-time panoramic image of the vehicle is received, it is displayed; the real-time panoramic image is synthesized and sent by the vehicle based on multiple real-time environmental images.

[0024] Fifthly, embodiments of this application provide a parking device, comprising:

[0025] The display module is used to display abnormal alert information when it receives abnormal alert information from the vehicle. The abnormal alert information is sent when the vehicle determines that the real-time autonomous driving state of the vehicle is abnormal based on real-time environmental information, which includes real-time environmental images from multiple observation points of the vehicle. When it receives a real-time panoramic image of the vehicle, it displays the real-time panoramic image. The real-time panoramic image is synthesized and sent by the vehicle based on multiple real-time environmental images.

[0026] The human-computer interaction module is used to convert viewing instructions into viewing requests and send them to the vehicle when it receives viewing instructions for abnormal alert information.

[0027] In a sixth aspect, embodiments of this application provide a mobile terminal, including: a terminal processor, a display screen, a human-computer interaction device, and a terminal signal transceiver;

[0028] The terminal transceiver is used for communication with the vehicle.

[0029] The display screen, human-computer interaction device, and terminal signal transceiver are all communicatively connected to the terminal processor.

[0030] The terminal processor configures a computer program, which, when executed by the terminal processor, implements the parking method provided in the fourth aspect above.

[0031] In a seventh aspect, embodiments of this application provide a parking system, including: a vehicle as provided in the third aspect above, a mobile terminal as provided in the sixth aspect above, and a cloud;

[0032] Both the mobile terminal and the vehicle are connected to the cloud for communication.

[0033] Eighthly, embodiments of this application provide a computer-readable storage medium storing a computer program that is executed by a processor to implement the parking method provided in the first aspect above, or the parking method provided in the fourth aspect above.

[0034] The beneficial technical effects of the technical solutions provided in this application include:

[0035] During automatic parking, real-time environmental images acquired from multiple observation points of the vehicle do not need to be synthesized into a panoramic image, which can effectively reduce the occupation of vehicle hardware resources. The vehicle's hardware resources can be used more for machine perception functions required for autonomous driving, such as obstacle ranging, vehicle positioning, and scene recognition. Furthermore, each real-time environmental image can be used to assist the vehicle's perception, improve the accuracy of the vehicle in determining the real-time autonomous driving status, and thus improve the safety of automatic parking.

[0036] The vehicle only initiates the synthesis of panoramic images upon user request and displays real-time panoramic images on the mobile terminal, providing a useful reference for manual parking intervention. Furthermore, with real-time panoramic images provided, the importance of machine perception functions such as the distance between the vehicle and obstacles, vehicle positioning, and scene recognition decreases, and some or all of these functions can be turned off. Therefore, synthesizing panoramic images at this time does not increase the additional burden on the vehicle's hardware.

[0037] Additional aspects and advantages of this application will be set forth in part in the description which follows, and will become apparent from the description or may be learned by practice of this application. Attached Figure Description

[0038] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0039] Figure 1 This is a schematic diagram of a parking system provided in an embodiment of this application;

[0040] Figure 2 A schematic diagram of the structure of a vehicle provided in this application embodiment;

[0041] Figure 3 A schematic flowchart illustrating a parking method for a vehicle provided in an embodiment of this application;

[0042] Figure 4 This is a schematic diagram of the structure of a parking device provided in an embodiment of this application;

[0043] Figure 5 This is a schematic diagram of the structure of a mobile terminal provided in an embodiment of this application;

[0044] Figure 6 A schematic flowchart illustrating a parking method applied to a mobile terminal, as provided in an embodiment of this application;

[0045] Figure 7 This is a schematic diagram of a parking device applied to a mobile terminal, provided as an embodiment of this application.

[0046] In the picture:

[0047] 100 - Vehicles;

[0048] 110 - Vehicle body; 120 - Power system; 130 - Steering system; 140 - On-board controller; 150 - Image acquisition unit; 160 - On-board signal transceiver;

[0049] 141-Environmental information acquisition module; 142-Environmental information analysis and vehicle control module; 143-Panoramic image processing module;

[0050] 200 - Mobile Terminal;

[0051] 210 - Terminal processor; 220 - Display screen; 230 - Human-computer interaction device; 240 - Terminal signal transceiver;

[0052] 211 - Display module; 212 - Human-computer interaction module;

[0053] 300-Cloud. Detailed Implementation

[0054] This application is described in detail below. Examples of embodiments of this application are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar components or components having the same or similar functions throughout. Furthermore, detailed descriptions of known technologies that are unnecessary for the features of this application are omitted. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0055] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0056] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in the specification of this application means the presence of the stated features, steps, operations, and / or components, but does not exclude the presence or addition of one or more other features, steps, operations, components, and / or groups thereof. The term “and / or” as used herein includes all or any units and all combinations thereof of one or more associated listed items.

[0057] The inventors of this application discovered through research that synthesizing environmental images collected from various observation points on a vehicle into a panoramic image can provide valuable reference information for manual intervention. However, conventionally, panoramic image synthesis is constantly underway regardless of whether the vehicle is in automatic parking or manual intervention mode. This process consumes a significant proportion of hardware resources, which can cause program lag during autonomous driving, leading to delayed or even erroneous responses, thus affecting the efficiency and safety of automatic parking.

[0058] The parking method, apparatus, system, storage medium, vehicle, and mobile terminal provided in this application are intended to solve the above-mentioned technical problems of the prior art.

[0059] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments.

[0060] This application provides a parking system, the structural schematic diagram of which is shown below. Figure 1 As shown, it includes: vehicle 100, mobile terminal 200, and cloud 300.

[0061] Mobile terminal 200 and vehicle 100 are respectively connected to cloud 300 for communication.

[0062] In this embodiment, the vehicle 100 can achieve automatic parking through autonomous driving, and the vehicle 100 can communicate with the mobile terminal 200 through the cloud 300. When the vehicle 100 encounters a situation that it cannot handle during autonomous driving, it can send an abnormal reminder message to the mobile terminal 200.

[0063] Vehicle 100 can also synthesize real-time environmental images collected from multiple observation points of vehicle 200 into a real-time panoramic image based on the investigation request sent by mobile terminal 200, and then send it to the mobile terminal.

[0064] The vehicle 100 can also exit the autonomous driving system and move according to the control information received from the mobile terminal 200 after receiving a takeover request.

[0065] The specific structure of the vehicle 100 will be described in detail later. During the automatic parking process, the vehicle 100 executes a parking method provided in this application embodiment, which will also be described in detail later.

[0066] The mobile terminal 200 communicates with the vehicle 100 through the cloud 300, and can receive abnormal reminder information sent by the vehicle 100 and then display the abnormal reminder information to the user.

[0067] When the mobile terminal 200 receives a user's instruction to view the abnormal alert information, it converts the instruction into a viewing request and sends it to the vehicle 100.

[0068] When the mobile terminal 200 receives a real-time panoramic image from the vehicle 100, it can also display the real-time panoramic image to the user.

[0069] When the mobile terminal 200 receives a takeover command from the user regarding the abnormal alert information, it converts the takeover command into a takeover request and sends it to the vehicle 100. While continuing to receive real-time panoramic images of the vehicle 100, it continues to display the real-time panoramic images. At the same time, when it receives a control command from the user regarding the vehicle 100, it converts the control command into control information and sends it to the vehicle 100, thereby enabling manual intervention to control the movement of the vehicle 100.

[0070] The specific structure of the mobile terminal 200 will be described in detail later. A parking method for the mobile terminal 200 provided in this application embodiment will also be executed, and this method will be described in detail later.

[0071] It is understandable that vehicle 100 and mobile terminal 200 can communicate directly. For example, if mobile terminal 200 is held by the parking lot manager and the communication distance between vehicle 100 and mobile terminal 200 is not large, the information exchange between vehicle 100 and mobile terminal 200 does not need to be relayed through cloud 300, which can reduce the construction and use costs of the parking system.

[0072] Based on the same inventive concept, this application provides a vehicle 100, the structural frame of which is shown in the schematic diagram below. Figure 2 As shown, it includes: vehicle body 110, power system 120, steering system 130, vehicle controller 140, several image acquisition units 150, and vehicle signal transceiver 160.

[0073] Several image acquisition devices 150 are respectively set at multiple observation points on the vehicle body 110.

[0074] The vehicle-mounted signal transceiver 160 is used for communication connection with the mobile terminal 200.

[0075] The power system 120, steering system 130, several image acquisition units 150, and vehicle signal transceiver 160 are respectively connected to the vehicle controller 140 for communication.

[0076] The vehicle controller 140 is configured with a computer program, which, when executed by the vehicle controller 140, implements a parking method for a vehicle 100 provided in this embodiment.

[0077] In this embodiment, several image acquisition devices 150 can be deployed along the perimeter of the vehicle body 110 to acquire real-time environmental images of the vehicle 100 from different directions. Specifically, the image acquisition device 150 can be a camera.

[0078] The vehicle controller 140 can synthesize multiple real-time environmental images acquired by several image acquisition devices 150 into a real-time panoramic image, and then send the real-time panoramic image to the mobile terminal 200 through the vehicle signal transceiver 160.

[0079] The vehicle controller 140 can receive control information sent by the mobile terminal 200 through the vehicle signal transceiver 160, and control the power system 120 and steering system 130 according to the control information, thereby realizing manual intervention control of the movement of the vehicle 100.

[0080] Optionally, the vehicle controller 140 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 2001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0081] In some possible implementations, vehicle 100 may also include several ranging radars, positioning devices, and scene recognition systems. The ranging radars may be, for example, millimeter-wave radars; the positioning devices may be, for example, GPS (Global Positioning System).

[0082] Based on the same inventive concept, this application provides a parking method applicable to vehicle 100, and the flowchart of the method is shown below. Figure 3 As shown, the method includes steps S101-S107:

[0083] S101: Acquire real-time environmental information of vehicle 100. The real-time environmental information includes real-time environmental images located at multiple observation points on vehicle 100.

[0084] In this step, real-time environmental images of the vehicle 100 in different directions can be collected by several image acquisition devices 150 deployed around the vehicle body 110.

[0085] Optionally, real-time environmental information may also include vehicle location information, distance between the vehicle and obstacles, autonomous driving scenario, and other information.

[0086] S102: Determine the real-time autonomous driving status of vehicle 100 based on real-time environmental information, and then execute step S103 or S104.

[0087] In this step, the real-time autonomous driving state of vehicle 100 can be determined based on at least one real-time environmental image from the real-time environmental information.

[0088] That is, each image of the implementation environment can be used by the vehicle to perceive the current environment and thus use it as a reference to determine the current autonomous driving state.

[0089] Optionally, the implementation environment information may also include the current dwell time of the vehicle 100 in autonomous driving mode. This step can determine the real-time autonomous driving status of the vehicle by comparing the current dwell time of the vehicle 100 with a set threshold. Specifically, the current dwell time of the vehicle 100 is determined. If the current dwell time is less than the set threshold, the real-time autonomous driving status is confirmed to be normal; if the current dwell time is not less than the set threshold, the real-time autonomous driving status is confirmed to be abnormal.

[0090] S103: When it is determined that the real-time automatic driving status is normal, control vehicle 100 to continue automatic parking.

[0091] During the automatic parking process in this step, the real-time environmental images obtained from multiple observation points of vehicle 100 do not need to be synthesized into a panoramic image, which can effectively reduce the occupation of vehicle 100's hardware resources. Vehicle 100's hardware resources can be used more for machine perception functions required for autonomous driving, such as obstacle ranging, vehicle positioning, and scene recognition. In addition, each real-time environmental image can be used to assist the vehicle's perception and improve the accuracy of vehicle 100 in determining the real-time autonomous driving status.

[0092] S104: When it is determined that the real-time autonomous driving state is abnormal, the vehicle 100 is controlled to stop moving and an abnormal reminder message is sent to the mobile terminal 200, so that the mobile terminal 200 displays the abnormal reminder message.

[0093] In this step, when the real-time autonomous driving state is determined to be abnormal, controlling vehicle 100 to stop moving ensures the safety of vehicle 100. Sending an abnormality alert to mobile terminal 200 can draw the user's attention.

[0094] S105: When a request to view an abnormality alert is received from the mobile terminal 200, multiple real-time environmental images are combined into a real-time panoramic image and sent to the mobile terminal 200, so that the mobile terminal 200 can display the real-time panoramic image.

[0095] In this step, vehicle 100 initiates the synthesis of panoramic images only upon user request and displays the real-time panoramic images on mobile terminal 200, providing a useful reference for manual parking intervention.

[0096] S106: When a takeover request for the panoramic image is received from the mobile terminal 200, the system continues to synthesize multiple real-time environmental images into a real-time panoramic image and sends it to the mobile terminal 200, so that the mobile terminal 200 continues to display the real-time panoramic image; at the same time, after exiting the autonomous driving system, the system controls the vehicle 100 to move according to the control information received from the mobile terminal 200.

[0097] In this step, the user determines whether manual parking intervention is needed based on the real-time panoramic image. The user can directly obtain information about obstacles around vehicle 100, the vehicle's location, and scene information from the panoramic image. That is, when vehicle 100 provides a real-time panoramic image, the importance of machine perception functions such as the distance between vehicle 100 and obstacles, vehicle positioning, and scene recognition decreases, and some or all of these functions can be turned off. Therefore, synthesizing a panoramic image at this time does not increase the additional burden on the vehicle's hardware.

[0098] Optionally, the control information includes: forward control information, braking control information, reverse control information, left turn control information, and right turn control information.

[0099] In this step, controlling the vehicle 100 to move according to the control information received from the mobile terminal 200 may include:

[0100] Based on one of the received forward control information, braking control information, and reverse control information, the power system 120 of the vehicle 100 is controlled to perform one of the actions of forward movement, braking, and reversing.

[0101] Based on one of the received left-turn control information and right-turn control information, the steering system 130 of the vehicle 100 is controlled to perform one of the actions of left turn and right turn.

[0102] S107: When a termination takeover request is received from the mobile terminal 200, the synthesis of multiple real-time environmental images into a real-time panoramic image is stopped, and the autonomous driving system is restarted to continue to determine the real-time autonomous driving status of the vehicle 100 based on the real-time environmental information.

[0103] In this step, after manual intervention in parking and resolution of the abnormal state of vehicle 100, vehicle 100 can resume automatic parking upon receiving a termination of takeover request from mobile terminal 200. At this point, the synthesis of multiple real-time environmental images into a real-time panoramic image is stopped, and the hardware resources of vehicle 100 are freed up for use by the restarted autonomous driving system and the mechanical perception functions of vehicle 100, thereby optimizing the utilization of vehicle 100's hardware resources.

[0104] Based on the same inventive concept, this application provides a parking device that can be applied to a vehicle 100. A schematic diagram of the device's structural framework is shown below. Figure 4 As shown, it includes: an environmental information acquisition module 141, an environmental information analysis and vehicle control module 142, and a panoramic image processing module 143.

[0105] The environmental information acquisition module 141 is used to acquire real-time environmental information of the vehicle 100. The real-time environmental information includes real-time environmental images located at multiple observation points on the vehicle 100.

[0106] The environmental information analysis and vehicle control module 142 is used to determine the real-time autonomous driving status of vehicle 100 based on real-time environmental information. When the real-time autonomous driving status is determined to be abnormal, the module controls vehicle 100 to stop moving and sends an abnormality alert to mobile terminal 200, causing mobile terminal 200 to display the abnormality alert. When a request to view the abnormality alert is received from mobile terminal 200, multiple real-time environmental images are combined into a real-time panoramic image and sent to mobile terminal 200, causing mobile terminal 200 to display the real-time panoramic image.

[0107] The panoramic image processing module 143 is used to synthesize multiple real-time environmental images into a real-time panoramic image and send it to the mobile terminal 200 when it receives a viewing request from the mobile terminal 200 for abnormal reminder information, so that the mobile terminal 200 can display the real-time panoramic image.

[0108] In some possible implementations, the panoramic image processing module 143 is also used to continue to synthesize multiple real-time environmental images into a real-time panoramic image and send it to the mobile terminal 200 after receiving a takeover request from the mobile terminal 200 for the panoramic image, so that the mobile terminal 200 continues to display the real-time panoramic image.

[0109] The environmental information analysis and vehicle control module 142 is also used to control the vehicle 100 to move according to the control information received from the mobile terminal 200 after exiting the autonomous driving system when it receives a takeover request from the mobile terminal 200 for the panoramic image.

[0110] Specifically, the control information includes: forward control information, braking control information, reverse control information, left turn control information, and right turn control information.

[0111] The environmental information analysis and vehicle control module 142 can control the power system 120 of the vehicle 100 to perform one of the following actions: forward, braking, or reverse, based on one of the received forward control information, braking control information, or reverse control information.

[0112] The environmental information analysis and vehicle control module 142 can control the steering system 130 of the vehicle 100 to perform one of the following actions: left turn or right turn, based on the received left turn control information and right turn control information.

[0113] In some possible implementations, the panoramic image processing module 143 is also configured to stop combining multiple real-time environmental images into a real-time panoramic image when it receives a termination takeover request from the mobile terminal 200.

[0114] The environmental information analysis and vehicle control module 142 is also used to restart the autonomous driving system when it receives a termination of takeover request from the mobile terminal 200, and continue to determine the real-time autonomous driving status of the vehicle 100 based on real-time environmental information.

[0115] In some possible implementations, when the environmental information analysis and vehicle control module 142 determines the real-time autonomous driving state of the vehicle 100 based on the real-time environmental information, it may do so in the following manner: determine the real-time autonomous driving state of the vehicle 100 based on at least one real-time environmental image in the real-time environmental information.

[0116] In some possible implementations, when the environmental information analysis and vehicle control module 142 determines the real-time autonomous driving status of the vehicle 100 based on real-time environmental information, it may do so in the following manner: determine the current dwell time of the vehicle 100; if the current dwell time is less than a set threshold, then confirm that the real-time autonomous driving status is normal; if the current dwell time is not less than the set threshold, then confirm that the real-time autonomous driving status is abnormal.

[0117] The parking device of this embodiment can execute a parking method provided in the embodiments of this application. This parking method can be applied to vehicle 100, and its implementation principle is similar, so it will not be described again here.

[0118] Based on the same inventive concept, this application provides a mobile terminal 200, the structural framework of which is shown in the schematic diagram below. Figure 5 As shown, it includes: a terminal processor 210, a display screen 220, a human-computer interaction device 230, and a terminal signal transceiver 240.

[0119] The terminal signal transceiver 240 is used for communication connection with the vehicle 100.

[0120] The display screen 220, the human-computer interaction device 230, and the terminal signal transceiver 240 are respectively connected to the terminal processor 210 for communication.

[0121] The terminal processor 210 is configured with a computer program, which, when executed by the terminal processor 210, implements any of the above-described parking methods as applied in the mobile terminal 200 according to the embodiments of this application.

[0122] In this embodiment, the human-computer interaction device 230 is used by the user to input commands to the mobile terminal 200. The human-computer interaction device 230 can be any one of a keyboard, mouse, microphone, or touchpad. The human-computer interaction device 230 can also be integrated with the display screen 220, such as a touch display screen.

[0123] When the terminal processor 210 receives an abnormality alert from the vehicle 100, it can display the abnormality alert to the user to remind the user that the vehicle 100 is in an abnormal state.

[0124] When the terminal processor 210 receives a user's instruction to view the abnormal alert information, it converts the instruction into a viewing request and sends it to the vehicle 100.

[0125] When the terminal processor 210 receives a real-time panoramic image of the vehicle 100, it can also display the real-time panoramic image to the user so that the user can determine whether manual intervention is needed for parking based on the real-time panoramic image.

[0126] When the terminal processor 210 receives a takeover instruction from the user in response to an abnormality alert, it converts the takeover instruction into a takeover request and sends it to the vehicle 100; when it continues to receive real-time panoramic images of the vehicle 100, it continues to display the real-time panoramic images to the user; when it receives a control instruction from the user for the vehicle 100, it converts the control instruction into control information and sends it to the vehicle 100, so that the vehicle 100 moves according to the control information after exiting the autonomous driving system.

[0127] When the terminal processor 210 receives a user input command to end management, it can convert the command into a command to end takeover request and send it to the vehicle 100.

[0128] Optionally, the terminal processor 210 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 2001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0129] Based on the same inventive concept, this application provides another parking method applicable to mobile terminal 200. A flowchart of this method is shown below. Figure 6 As shown, the method includes S201-S205:

[0130] S201: When receiving an abnormality alert from vehicle 100, display the abnormality alert; the abnormality alert is sent by vehicle 100 when it determines that the real-time autonomous driving state of vehicle 100 is abnormal based on real-time environmental information, which includes real-time environmental images located at multiple observation points of vehicle 100.

[0131] In this step, the mobile terminal 200 can display abnormal reminder information in the form of text, sound, light, or animation to alert the user.

[0132] S202: When a viewing instruction for an abnormality alert is received, the viewing instruction is converted into a viewing request and sent to vehicle 100.

[0133] In this step, the mobile terminal 200 receives the user's viewing instruction through the human-computer interaction device 230, and converts the viewing instruction into machine language (i.e., viewing request) and sends it to the vehicle 100, so that the vehicle 100 starts the synthesis of panoramic images.

[0134] Specifically, the user's viewing command can be a physical or virtual button on the human-computer interaction device 230.

[0135] S203: When a real-time panoramic image of vehicle 100 is received, the real-time panoramic image is displayed; the real-time panoramic image is synthesized and sent by vehicle 100 based on multiple real-time environmental images.

[0136] In this step, when the mobile terminal 200 receives the real-time panoramic image of the vehicle 100, it displays the real-time panoramic image to the user so that the user can determine whether manual intervention is needed for parking based on the real-time panoramic image.

[0137] S204: When a takeover instruction for an abnormality alert is received, the takeover instruction is converted into a takeover request and sent to vehicle 100; when a real-time panoramic image of vehicle 100 is received, the real-time panoramic image is continued to be displayed; when a control instruction for vehicle 100 is received, the control instruction is converted into control information and sent to vehicle 100, so that vehicle 100 moves according to the control information after exiting the autonomous driving system.

[0138] In this step, when the mobile terminal 200 receives a takeover instruction from the user regarding the abnormality alert, it converts the takeover instruction into machine language (i.e., a takeover request) and sends it to the vehicle 100, so that the vehicle exits the autonomous driving system and moves according to the control information.

[0139] Specifically, the takeover command can be a physical or virtual button on the human-computer interaction device 230.

[0140] Users can directly obtain information about obstacles around vehicle 100, the location of vehicle 100, and scene information from the panoramic image. That is, when vehicle 100 provides a real-time panoramic image, the importance of machine perception functions such as the distance between vehicle 100 and obstacles, vehicle localization, and scene recognition decreases, and some or all of these functions can be turned off. Therefore, synthesizing a panoramic image at this time does not increase the additional burden on the vehicle's hardware.

[0141] Optionally, the control information includes: forward control information, braking control information, reverse control information, left turn control information, and right turn control information.

[0142] Specifically, the control information can be a series of physical or virtual buttons on the human-computer interaction device 230.

[0143] S205: Upon receiving the end management instruction, the end management instruction is converted into an end takeover request and sent to vehicle 100, causing vehicle 100 to stop combining multiple real-time environmental images into a real-time panoramic image and restart the autonomous driving system to continue determining the real-time autonomous driving status of vehicle 100 based on real-time environmental information.

[0144] In this step, after manual intervention in parking and resolution of the abnormal state of vehicle 100, mobile terminal 200 can convert the user's input request to end takeover into machine language (i.e., end management command) and send it to vehicle 100, so that vehicle 100 can resume automatic parking.

[0145] Specifically, the end management command can be a physical or virtual button on the human-computer interaction device 230.

[0146] Based on the same inventive concept, another parking device provided in this application embodiment can be applied to a mobile terminal 200. A schematic diagram of the device's structural framework is shown below. Figure 7 As shown, it includes: a display module 211 and a human-computer interaction module 212.

[0147] The display module 211 is used to display an anomaly alert message received from the vehicle 100. The anomaly alert message is sent by the vehicle 100 when it determines that its real-time autonomous driving state is abnormal based on real-time environmental information, which includes real-time environmental images from multiple observation points on the vehicle 100. When a real-time panoramic image from the vehicle 100 is received, the module displays that image. The real-time panoramic image is synthesized and transmitted by the vehicle 100 based on multiple real-time environmental images.

[0148] The human-machine interaction module 212 is used to convert a viewing instruction into a viewing request and send it to the vehicle 100 when it receives a viewing instruction for an abnormality reminder.

[0149] In some possible implementations, the human-machine interaction module 212 is also used to convert the takeover instruction into a takeover request and send it to the vehicle 100 when it receives a takeover instruction for an abnormality alert.

[0150] The display module 211 is also used to continue displaying the real-time panoramic image when it continues to receive the real-time panoramic image of the vehicle 100.

[0151] The human-machine interaction module 212 is also used to convert control commands into control information and send them to the vehicle 100 when it receives control commands for the vehicle 100, so that the vehicle 100 can move according to the control information after exiting the autonomous driving system.

[0152] Specifically, the control information includes: forward control information, braking control information, reverse control information, left turn control information, and right turn control information.

[0153] In some possible implementations, the human-machine interaction module 212 is also used to convert the end management instruction into an end takeover request and send it to the vehicle 100 when it receives the end management instruction, so that the vehicle 100 stops combining multiple real-time environmental images into a real-time panoramic image and restarts the autonomous driving system to continue to determine the real-time autonomous driving status of the vehicle 100 based on the real-time environmental information.

[0154] Another parking device in this embodiment can execute another parking method provided in the embodiments of this application. The implementation principle is similar, and will not be described again here.

[0155] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program, which is executed by a processor to implement any of the parking methods provided in this embodiment, including parking methods applicable to vehicles and parking methods applicable to mobile terminals.

[0156] Computer-readable storage media can be ROM (Read-Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (CompactDisc Read-Only Memory) or other optical disc storage, optical disk storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0157] This application provides a computer-readable storage medium applicable to the above-described method embodiments. Further details will not be elaborated upon here.

[0158] By applying the embodiments of this application, at least the following beneficial effects can be achieved:

[0159] 1. During automatic parking, the real-time environmental images obtained from multiple observation points of the vehicle do not need to be synthesized into a panoramic image, which can effectively reduce the occupation of vehicle hardware resources. The vehicle's hardware resources can be used more for machine perception functions required for autonomous driving, such as obstacle ranging, vehicle positioning, and scene recognition. In addition, each real-time environmental image can be used to assist the vehicle's perception and improve the accuracy of the vehicle in determining the real-time autonomous driving status.

[0160] 2. The vehicle only initiates the synthesis of panoramic images upon user request and displays real-time panoramic images on the mobile terminal, providing a useful reference for manual parking intervention. Furthermore, with real-time panoramic images provided, the importance of machine perception functions such as the distance between the vehicle and obstacles, vehicle positioning, and scene recognition decreases, and some or all of these functions can be turned off. Therefore, synthesizing panoramic images at this time will not increase the additional burden on the vehicle's hardware.

[0161] Those skilled in the art will understand that the steps, measures, and solutions in the various operations, methods, and processes discussed in this application can be alternated, modified, combined, or deleted. Furthermore, other steps, measures, and solutions in the various operations, methods, and processes discussed in this application can also be alternated, modified, rearranged, decomposed, combined, or deleted. Furthermore, steps, measures, and solutions in the prior art that are similar to those disclosed in this application can also be alternated, modified, rearranged, decomposed, combined, or deleted.

[0162] In the description of this application, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0163] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0164] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "joining" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0165] In the description of this specification, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0166] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0167] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A parking method applied to a vehicle, characterized in that, include: Obtain real-time environmental information for the vehicle; The real-time environmental information includes real-time environmental images located at multiple observation points on the vehicle; The real-time autonomous driving status of the vehicle is determined based on the real-time environmental information. When the real-time autonomous driving state is determined to be abnormal, the vehicle is controlled to stop moving, and an abnormality reminder message is sent to the mobile terminal, so that the mobile terminal displays the abnormality reminder message; When a request to view the abnormal alert information is received from the mobile terminal, multiple real-time environmental images are combined into a real-time panoramic image and sent to the mobile terminal, so that the mobile terminal can display the real-time panoramic image. When a takeover request for the real-time panoramic image is received from the mobile terminal, the system continues to synthesize multiple real-time environmental images into a real-time panoramic image and sends it to the mobile terminal, so that the mobile terminal continues to display the real-time panoramic image; at the same time, after exiting the autonomous driving system, the system controls the vehicle to move according to the control information received from the mobile terminal.

2. The parking method according to claim 1, characterized in that, The control information includes: forward control information, braking control information, reverse control information, left turn control information, and right turn control information; The step of controlling the vehicle to move according to the control information received from the mobile terminal includes: Based on one of the received forward control information, braking control information, and reverse control information, the vehicle's power system is controlled to perform one of the forward, braking, and reverse actions. Based on one of the received left-turn control information and right-turn control information, the vehicle's steering system is controlled to perform one of the actions of turning left or turning back.

3. The parking method according to claim 1, characterized in that, After exiting the autonomous driving system, the following is also included: When the mobile terminal receives a request to end takeover, the system stops combining multiple real-time environmental images into the real-time panoramic image, restarts the autonomous driving system, and continues to determine the real-time autonomous driving status of the vehicle based on the real-time environmental information.

4. The parking method according to any one of claims 1-3, characterized in that, Determining the real-time autonomous driving state of the vehicle based on the real-time environmental information includes: The real-time autonomous driving state of the vehicle is determined based on at least one of the real-time environmental images in the real-time environmental information.

5. The parking method according to claim 4, characterized in that, Determining the real-time autonomous driving state of the vehicle based on the real-time environmental information further includes: Determine the current stationary time of the vehicle; If the current pause time is less than a set threshold, then the real-time autonomous driving status is confirmed to be normal. If the current pause time is not less than a set threshold, then the real-time autonomous driving state is confirmed to be abnormal.

6. A parking device, characterized in that, include: The environmental information acquisition module is used to acquire real-time environmental information of the vehicle. The real-time environmental information includes real-time environmental images located at multiple observation points on the vehicle; The environmental information analysis and vehicle control module is used to determine the real-time autonomous driving status of the vehicle based on the real-time environmental information; when the real-time autonomous driving status is determined to be abnormal, the module controls the vehicle to stop moving and sends an abnormality reminder message to the mobile terminal. This causes the mobile terminal to display the abnormality alert information; When a request to view the abnormal alert information is received from the mobile terminal, multiple real-time environmental images are combined into a real-time panoramic image and sent to the mobile terminal, so that the mobile terminal can display the real-time panoramic image. When a takeover request for the real-time panoramic image is received from the mobile terminal, the system continues to synthesize multiple real-time environmental images into a real-time panoramic image and sends it to the mobile terminal, so that the mobile terminal continues to display the real-time panoramic image; at the same time, after exiting the autonomous driving system, the system controls the vehicle to move according to the control information received from the mobile terminal. The panoramic image processing module is used to synthesize multiple real-time environmental images into a real-time panoramic image and send it to the mobile terminal when it receives a viewing request from the mobile terminal for the abnormal reminder information, so that the mobile terminal can display the real-time panoramic image.

7. A vehicle, characterized in that, include: Vehicle body, power system, steering system, on-board controller, several image acquisition devices, and on-board signal transceivers; Several of the aforementioned image acquisition devices are respectively installed at multiple observation points on the vehicle body; The vehicle-mounted signal transceiver is used for communication connection with the mobile terminal; The power system, the steering system, the plurality of image acquisition units, and the vehicle signal transceiver are respectively communicatively connected to the vehicle controller; The vehicle controller is configured with a computer program, which, when executed by the vehicle controller, implements the parking method as described in any one of claims 1-5.

8. A parking method applied to a mobile terminal, characterized in that, include: When an abnormal vehicle alert is received, the abnormal alert information is displayed. The abnormality alert is sent by the vehicle when it determines that the real-time autonomous driving state of the vehicle is abnormal based on real-time environmental information. The real-time environmental information includes real-time environmental images located at multiple observation points of the vehicle. Upon receiving a viewing instruction for an anomaly alert, the viewing instruction is converted into a viewing request and sent to the vehicle; Upon receiving a real-time panoramic image of the vehicle, the real-time panoramic image is displayed. The real-time panoramic image is synthesized and transmitted by the vehicle based on multiple real-time environmental images; Upon receiving a takeover command for the real-time panoramic image, the takeover command is converted into a takeover request and sent to the vehicle. When receiving more real-time panoramic images of the vehicle, continue to display the real-time panoramic images; When a control command is received for the vehicle, the control command is converted into control information and sent to the vehicle, so that the vehicle moves according to the control information after exiting the autonomous driving system.

9. The parking method according to claim 8, characterized in that, The control information includes: forward control information, braking control information, reverse control information, left turn control information, and right turn control information.

10. The parking method according to claim 8, characterized in that, Upon receiving a control command for the vehicle, the method further includes converting the control command into control information and sending it to the vehicle, so that the vehicle moves according to the control information after exiting the autonomous driving system. Upon receiving a termination management instruction, the termination management instruction is converted into a termination takeover request and sent to the vehicle, causing the vehicle to stop combining multiple real-time environmental images into the real-time panoramic image and restart the autonomous driving system to continue determining the real-time autonomous driving state of the vehicle based on the real-time environmental information.

11. A parking device, characterized in that, include: The display module is used to display the abnormal alert information when it receives an abnormal alert information from the vehicle; The abnormality alert is sent when the vehicle determines that its real-time autonomous driving state is abnormal based on real-time environmental information, which includes real-time environmental images from multiple observation points of the vehicle; when the vehicle's real-time panoramic image is received, the real-time panoramic image is displayed. The real-time panoramic image is synthesized and transmitted by the vehicle based on multiple real-time environmental images; when a takeover command is received for the real-time panoramic image, the takeover command is converted into a takeover request and sent to the vehicle; when the real-time panoramic image of the vehicle is received again, the real-time panoramic image continues to be displayed; when a control command is received for the vehicle, the control command is converted into control information and sent to the vehicle, so that the vehicle moves according to the control information after exiting the autonomous driving system; The human-computer interaction module is used to convert a viewing instruction for an abnormality alert into a viewing request and send it to the vehicle.

12. A mobile terminal, characterized in that, include: Terminal processor, display screen, human-computer interaction device, and terminal signal transceiver; The terminal transceiver is used for communication connection with the vehicle; The display screen, the human-computer interaction device, and the terminal transceiver are all communicatively connected to the terminal processor. The terminal processor is configured with a computer program that, when executed by the terminal processor, implements the parking method as described in any one of claims 8-10.

13. A parking system, characterized in that, include: The vehicle as described in claim 7, the mobile terminal as described in claim 12, and the cloud; The mobile terminal and the vehicle are respectively connected to the cloud for communication.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is executed by a processor to implement the parking method as described in any one of claims 1-5 and 8-10.

Images