Vehicle control method and device, vehicle, electronic equipment and storage medium

Abstract

The invention provides a vehicle control method and device, a vehicle, electronic equipment and a storage medium, and relates to the technical field of automatic driving. The method comprises the steps of obtaining a parking instruction of a user; determining surrounding environment information of the vehicle; a scene recognition result corresponding to the surrounding environment information is obtained, and the scene recognition result is obtained by processing the surrounding environment information of the vehicle based on a deep learning model; and determining a parking path plan according to the scene recognition result and the parking instruction of the user, and controlling the vehicle to be automatically parked according to the parking path plan. Therefore, in the process that the vehicle arrives at the parking space, finds the parking space and parks in the parking space during cruising, the surrounding environment information of the vehicle can be recognized in real time, the vehicle is accurately controlled in combination with the scene recognition result and the parking instruction of the user, the safety and accuracy in the automatic parking process of the vehicle are improved, and the user experience is improved. And the vehicle can be controlled to be automatically parked in the parking space selected by the user or can be automatically parked in the parking space selected by the vehicle, so that the parking efficiency and the user experience are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of autonomous driving technology, and in particular to a vehicle control method, device, vehicle, electronic equipment and storage medium. Background Art

[0002] With the development of vehicle technology, more and more vehicles are equipped with automatic parking functions, which improves the user experience. However, the current automatic parking function can only control the vehicle to automatically park in the target storage location when it reaches the vicinity of the user's selected target storage location. Summary of the invention

[0003] In order to overcome the problems existing in the related art, the present disclosure provides a vehicle control method, device, vehicle, electronic device and storage medium.

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided a vehicle control method, comprising:

[0005] Get user parking instructions;

[0006] In response to the user's parking instruction, determining the surrounding environment information of the vehicle;

[0007] Obtaining a scene recognition result corresponding to the surrounding environment information of the vehicle, wherein the scene recognition result is obtained by processing the surrounding environment information of the vehicle based on a deep learning model;

[0008] A parking path plan is determined according to the scene recognition result and the user parking instruction, and the vehicle is controlled to perform automatic parking according to the parking path plan.

[0009] According to a second aspect of an embodiment of the present disclosure, there is provided a vehicle control method, comprising:

[0010] Obtaining the surrounding environment information of the vehicle;

[0011] Inputting the surrounding environment information into a deep learning model to obtain a scene recognition result;

[0012] The scene recognition result is sent.

[0013] According to a third aspect of an embodiment of the present disclosure, there is provided a vehicle control device, comprising:

[0014] A first acquisition module is used to acquire a user's parking instruction;

[0015] A determination module, configured to respond to the user's parking instruction and determine the surrounding environment information of the vehicle;

[0016] A second acquisition module is used to acquire a scene recognition result corresponding to the surrounding environment information of the vehicle, wherein the scene recognition result is obtained by processing the surrounding environment information of the vehicle based on a deep learning model;

[0017] A control module is used to determine a parking path plan according to the scene recognition result and the user parking instruction, and control the vehicle to perform automatic parking according to the parking path plan.

[0018] According to a fourth aspect of an embodiment of the present disclosure, there is provided a vehicle control device, comprising:

[0019] An acquisition module, used to acquire the surrounding environment information of the vehicle;

[0020] A processing module, used to input the surrounding environment information into a deep learning model to obtain a scene recognition result;

[0021] A sending module is used to send the scene recognition result.

[0022] According to a fifth aspect of an embodiment of the present disclosure, a vehicle is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to: implement the steps of the vehicle control method proposed in the embodiment of the first aspect of the present disclosure.

[0023] According to the sixth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to: implement the steps of the vehicle control method proposed in the embodiment of the second aspect of the present disclosure.

[0024] According to the seventh aspect of the embodiment of the present disclosure, a computer-readable storage medium is provided. When the instructions in the storage medium are executed by the processor of the mobile terminal, the steps of the vehicle control method proposed in the embodiment of the first aspect of the present disclosure are implemented.

[0025] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0026] In the disclosed embodiment, the user's parking instruction is first obtained, and then the user's parking instruction is responded to, the vehicle's surrounding environment information is determined, and then the scene recognition result corresponding to the vehicle's surrounding environment information is obtained, wherein the scene recognition result is obtained by processing the vehicle's surrounding environment information based on a deep learning model, and finally the parking path planning is determined according to the scene recognition result and the user's parking instruction, and the vehicle is controlled to automatically park according to the parking path planning. As a result, in the process of the vehicle cruising to a parking space, finding a parking space, and parking in a parking space, the scene recognition of the vehicle's surrounding environment information can be performed in real time, and the vehicle can be accurately controlled in combination with the scene recognition result and the user's parking instruction, thereby improving the safety and accuracy of the vehicle's automatic parking process, and the vehicle can be controlled to automatically park in the storage space selected by the user, or the parking space that the vehicle selects autonomously, thereby improving parking efficiency and enhancing user experience.

[0027] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure:

[0029] Figure 1 is a flow chart of a vehicle control method according to some embodiments of the present disclosure;

[0030] Figure 2 is a flow chart of a vehicle control method according to some embodiments of the present disclosure;

[0031] Figure 3 is a flow chart of a vehicle control method according to some embodiments of the present disclosure;

[0032] Figure 4 is a flow chart of a vehicle control method according to some embodiments of the present disclosure;

[0033] Figure 5 A schematic diagram of a sign provided in accordance with an embodiment of the present disclosure;

[0034] Figure 6 is a structural schematic diagram of a vehicle control device according to some embodiments of the present disclosure;

[0035] Figure 7 is a structural schematic diagram of a vehicle control device according to some embodiments of the present disclosure;

[0036] Figure 8 is a schematic diagram of a functional block diagram of a vehicle shown in an exemplary embodiment;

[0037] Fig. 9 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0038] Some embodiments of the present disclosure will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. Various changes, modifications and equivalents of the methods, devices and / or systems described herein will become apparent after understanding the present disclosure. For example, the order of operations described herein is merely an example and is not limited to those orders set forth herein, but can be changed as becomes apparent after understanding the present disclosure, except for operations that must be performed in a specific order. In addition, for clarity and brevity, descriptions of features known in the art may be omitted.

[0039] The embodiments described in some embodiments of the present disclosure below do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0040] It should be noted that the vehicle control method of this embodiment can be applied to a vehicle control device. In some possible embodiments, the device can be configured in a vehicle or a chip so that the vehicle or the chip can perform a vehicle control function. In addition, in some possible embodiments, the vehicle control device can also be software in the vehicle, etc. Among them, the software, for example, is vehicle control software, etc.

[0041] Figure 1 is a flow chart of a vehicle control method according to some embodiments of the present disclosure. Figure 1 As shown, the following steps are included:

[0042] Step 101, obtaining a user's parking instruction.

[0043] Among them, the user parking instruction can be an instruction for the user to instruct the vehicle to park automatically.

[0044] For example, the user's parking instruction may be "please park the car in the parking space on the B2 floor", "please park the car in the parking space on the third underground floor", "please notify me immediately if there is a risk of collision during the entire parking process", "please park the car in the parking space close to the left of the vehicle", "please park the vehicle and turn right after parking", etc. The present disclosure does not limit this.

[0045] In some embodiments, the user's voice and / or body movements may be recognized to obtain the user's parking instructions. The vehicle may obtain the user's voice or body movements in real time, and then recognize and determine whether the user's voice or body movements contain a pre-set parking instruction.

[0046] Alternatively, the user's parking instruction may also be obtained based on the startup text operation input by the user. In some embodiments, the user may input text on the vehicle screen, and the vehicle may determine to execute the parking instruction based on the recognition of the text input by the user.

[0047] Alternatively, the user's parking instruction can also be obtained based on the user's click on the vehicle's control operation. In some embodiments, the automatic parking control can be displayed on the vehicle screen, or a physical control for automatic parking can be set in the vehicle, so that the user can obtain the user's parking instruction after clicking the automatic parking control on the vehicle screen or the physical control. In some embodiments, the physical control can be set on the steering wheel.

[0048] Step 102, responding to the user's parking instruction, determining the surrounding environment information of the vehicle.

[0049] In the disclosed embodiment, after obtaining the user's parking instruction, the surrounding environment information of the vehicle can be further determined, and then the scene recognition result corresponding to the surrounding environment information can be determined.

[0050] The surrounding environment information of the vehicle may be an image collected in real time by an image sensor on the vehicle. The image sensor may be a panoramic camera, a language camera, a laser radar, etc. The present disclosure does not limit this.

[0051] Step 103, obtaining a scene recognition result corresponding to the surrounding environment information of the vehicle, wherein the scene recognition result is obtained by processing the surrounding environment information of the vehicle based on a deep learning model.

[0052] The deep learning model may be a scene recognition model. In some embodiments, the deep learning model may be deployed on the vehicle side or on the cloud side. This disclosure does not limit this.

[0053] In some embodiments, the deep learning model may be a visual language model (VLM). It should be noted that the VLM generally has billions of parameters, and it is difficult to complete the deployment of edge computing on the vehicle side. Therefore, the VLM model may be deployed on the cloud side, and the surrounding environment information may be sent to the cloud side, and the cloud side may perform scene recognition on the second image through the VLM, and then the scene recognition result may be fed back to the vehicle side through the vehicle-cloud link.

[0054] It should be noted that, due to the large number of training samples of the vision-language model, it can recognize various scenarios that cannot be recognized by the vehicle terminal, such as shopping carts, cardboard boxes, etc. Therefore, by deploying the vision-language model in the cloud to perform scene recognition on the surrounding environment information, it can help the vehicle better understand the environment around the vehicle, and thus control the vehicle more accurately.

[0055] Step 104, determine the parking path planning according to the scene recognition result and the user's parking instruction, and control the vehicle to perform automatic parking according to the parking path planning.

[0056] In the embodiments of the present disclosure, first obtain the user's parking instruction, then respond to the user's parking instruction to determine the surrounding environment information of the vehicle, and further obtain the scene recognition result corresponding to the surrounding environment information of the vehicle. The scene recognition result is obtained by processing the surrounding environment information of the vehicle based on a deep learning model. Finally, determine the parking path planning according to the scene recognition result and the user's parking instruction, and control the vehicle to perform automatic parking according to the parking path planning. Thus, during the process of the vehicle cruising to the parking space, searching for the parking space, and parking into the parking space, the scene recognition of the surrounding environment information of the vehicle can be performed in real time. Combining the scene recognition result and the user's parking instruction, the vehicle can be accurately controlled, which can not only improve the safety and accuracy during the automatic parking process of the vehicle, but also control the vehicle to automatically park into the parking space selected by the user or the available parking space autonomously selected by the vehicle, thereby improving the parking success rate and enhancing the user experience.

[0057] Figure 2 is a flowchart of a vehicle control method shown according to some embodiments of the present disclosure, as Figure 2 shown, including the following steps:

[0058] Step 201, obtain the user's parking instruction.

[0059] Step 202, respond to the user's parking instruction and determine the current state of the vehicle.

[0060] Among them, the current state of the vehicle can be a cruising state, a parking state, and other states. The present disclosure does not limit this.

[0061] Among them, the cruising state can be a state where the vehicle travels at a certain speed and searches for a target parking space. For example, in the scenario of cross-floor parking, the driving state of the vehicle from the current position to the target parking space on the target floor is the cruising state.

[0062] Among them, the parking state can be a state where the vehicle parks into the target parking space after driving around the target parking space.

[0063] In some embodiments, the current state of the vehicle can be determined according to the state machine in the vehicle.

[0064] In some embodiments, after determining the current state of the vehicle, the current state of the vehicle can also be synchronized to the user, so that the user in the vehicle can know the real-time state of the vehicle. In some embodiments, the current state of the vehicle can be synchronized by means of display on the in-vehicle screen, or, the current state of the vehicle can also be synchronized by means of voice broadcast.

[0065] Step 203, determine the surrounding environment information according to the current state of the vehicle.

[0066] In some embodiments, when the current state of the vehicle is the cruise state, the surrounding environment information may include image information containing a sign. The sign includes one or more of a parking lot entrance / exit sign, a bay number sign, a no-parking area sign, a driving direction sign, a bay status reminder sign, a bay pointing sign, a ground / wall indication sign, a road indication sign, and an elevator sign.

[0067] The image information containing the sign can be collected by a target image acquisition device. The target image acquisition device can be an in-vehicle camera.

[0068] In some embodiments, when the current state of the vehicle is the cruise state, perform target recognition on the vehicle perimeter image collected by the target image acquisition device to obtain a target recognition result; when the target recognition result is that the vehicle perimeter image contains a sign, perform cropping on the vehicle perimeter image based on the target position of the sign in the vehicle perimeter image in the target recognition result to obtain the image information containing the sign.

[0069] Among them, performing target recognition on the vehicle perimeter image can identify whether the vehicle perimeter image contains a sign, and when the vehicle perimeter image contains a sign, determine the target position of the sign in the vehicle perimeter image. Thus, the target recognition of the vehicle perimeter image can be performed quickly and accurately.

[0070] In some embodiments, a pre-trained target recognition model can be used to perform target recognition on the vehicle perimeter image. The target recognition model is trained and generated according to the first sample image containing a sign and its corresponding label, and the second sample image not containing a sign and its corresponding label.

[0071] In some embodiments, when the current state of the vehicle is the parking state, the surrounding environment information includes a target image of the target bay to which the vehicle is to be parked.

[0072] In some embodiments, the position information of the target storage location in the target image may also be marked. Thereby, the deep learning model can be informed of the position of the target storage location in the target image, so as to prompt the deep learning model to perform scene recognition near the target storage location, thereby reducing the computational amount of scene recognition and improving the scene recognition efficiency.

[0073] In some embodiments, the position information may be the positions of the four corner points of the target storage location. The present disclosure does not limit this.

[0074] In some embodiments, when the current state of the vehicle is the parking state, the vehicle performs storage location recognition on the vehicle perimeter image collected currently, and then determines the vehicle perimeter image including the target storage location as the target image.

[0075] In some embodiments, the vehicle perimeter image including the target storage location may also be cropped based on the position information to obtain the target image. The target image includes the target storage location and the area within the first distance around the target storage location. The first distance may be 5 meters, 3 meters, etc. Thereby, the content of scene recognition can be further reduced and the scene recognition efficiency can be improved.

[0076] In some embodiments, when a candidate storage location is detected, the candidate storage location is displayed, and the driver is prompted to select the target storage location from the candidate storage locations, and then the target storage location selected by the driver by clicking the screen and / or voice from the candidate storage locations is obtained. Thereby, the target storage location selected by the user can be obtained before the vehicle parks into the target storage location.

[0077] In some embodiments, after the vehicle detects a candidate storage location, the vehicle may also autonomously select a storage location from the candidate storage locations as the target storage location. In some embodiments, the user may also specify the target storage location in the user parking instruction. The present disclosure does not limit this.

[0078] In the embodiments of the present disclosure, the vehicle perimeter image collected currently by the vehicle is processed based on the current state of the vehicle, so that the important information required in the current state of the vehicle is included in the vehicle perimeter environment information, and the efficiency and accuracy of scene recognition are improved.

[0079] Step 204, obtaining a scene recognition result corresponding to the vehicle perimeter environment information, where the scene recognition result is obtained by processing the vehicle perimeter environment information based on a deep learning model.

[0080] In some embodiments, when the parking state is the cruise state, the scene recognition result may include vehicle driving suggestions and / or the meanings corresponding to each identifier in the sign.

[0081] Among them, the meaning corresponding to each identifier in the sign, for example, the floor where the current vehicle is located, the destination when driving left, the destination when driving right, etc. The present disclosure does not limit this.

[0082] Among them, the vehicle driving suggestion can be determined by the visual language model according to the sign, and it is a suggestion on which direction the vehicle can drive.

[0083] In some embodiments, when the vehicle state is the parking state, the scene recognition result may include the position information of the target parking space and the surrounding obstacles. Among them, the parking state refers to the state where the vehicle is parked in the target parking space; the position information of the surrounding obstacles refers to the position information of moving objects such as pedestrians, carts, other vehicles, etc., and / or stationary objects (such as stone piers, stationary suitcases) that may affect the vehicle's entry into the target parking space.

[0084] Step 205, determine the parking path planning according to the scene recognition result and the user's parking instruction, and control the vehicle to perform automatic parking according to the parking path planning.

[0085] In some embodiments, during the vehicle parking process, if a preset event of the vehicle is detected, at least one of stopping parking, determining a new target parking space, and prompting the user to take over is performed.

[0086] In some embodiments, the preset event includes any one of the following situations: the vehicle has a collision risk, the target parking space is too narrow for the vehicle to park in, and a user instruction to stop parking is received. Among them, receiving a user instruction to stop parking includes the user gently stepping on the brake, or voice control, or clicking to stop parking on the in-vehicle screen, or stopping parking by clicking a physical control on the vehicle.

[0087] In some embodiments, it can be determined that the vehicle has a collision risk when the distance between the vehicle and the obstacle is less than the distance threshold.

[0088] In the embodiments of the present disclosure, a user parking instruction is obtained, and the current state of the vehicle is determined; according to the current state of the vehicle, the surrounding environment information is determined, and the scene recognition result corresponding to the surrounding environment information of the vehicle is obtained, where the scene recognition result is obtained by processing the surrounding environment information of the vehicle based on a deep learning model. Finally, the parking path planning is determined according to the scene recognition result and the user parking instruction, and the vehicle is controlled to perform automatic parking according to the parking path planning. Thus, based on the current state of the vehicle, the surrounding environment information of the vehicle can be determined, so that the important information required in the current state of the vehicle is included in the determined surrounding environment information of the vehicle, improving the efficiency and accuracy of scene recognition, and further improving the efficiency of automatic parking while ensuring the safety and success rate of automatic parking.

[0089] Figure 3It is a flowchart of a vehicle control method shown according to some embodiments of the present disclosure, including the following steps:

[0090] Step 301, obtain a user parking instruction.

[0091] Step 302, in response to the user parking instruction, determine the surrounding environment information of the vehicle.

[0092] Step 303, obtain the scene recognition result corresponding to the surrounding environment information of the vehicle, where the scene recognition result is obtained by processing the surrounding environment information of the vehicle based on a deep learning model.

[0093] Step 304, determine the road information corresponding to the vehicle's location and the location information of the surrounding objects of the vehicle.

[0094] Among them, the road information can be determined through a road topology map, so as to tell the vehicle which areas or sections the vehicle can currently pass through.

[0095] Among them, the location information of the surrounding objects of the vehicle may include the motion data of moving obstacles (such as vehicles, people, etc.), and also includes the location information of fixed obstacles (such as stone piers, stationary suitcases, etc.).

[0096] Among them, the motion data may include the spatial coordinates, distance, speed, acceleration information, etc. of the moving obstacle. The present disclosure does not limit this.

[0097] In some embodiments, the road topology map corresponding to the vehicle's location and the location information of the surrounding objects of the vehicle can be determined by the vehicle itself. In some embodiments, the road topology map corresponding to the vehicle's location and the location information of the surrounding objects of the vehicle can be obtained from the cloud. The present disclosure does not limit this.

[0098] Step 305, determine the parking path planning according to the scene recognition result, the user parking instruction, the road information and the location information of the surrounding objects of the vehicle.

[0099] In the embodiments of the present disclosure, after determining the scene recognition result, the user parking instruction, the road information and the location information of the surrounding objects of the vehicle, the parking path planning can be determined through comprehensive judgment according to the scene recognition result, the user parking instruction, the road information and the location information of the surrounding objects of the vehicle, so as to improve the accuracy of the parking path planning and the parking success rate.

[0100] Step 306, control the vehicle to perform automatic parking according to the parking path planning.

[0101] In the embodiments of the present disclosure, the parking path planning can be determined according to the scene recognition result corresponding to the surrounding environment information of the vehicle, the user parking instruction, the road information, and the position information of the surrounding objects of the vehicle, so that the vehicle can be controlled more accurately, and the safety and reliability of automatic parking can be further improved.

[0102] In some embodiments, the vehicle can also send the current state of the vehicle and the first instruction, where the first instruction is the user parking instruction or the vehicle control instruction corresponding to the user parking instruction.

[0103] In some embodiments, the vehicle control instruction can include vehicle driving instructions, parking space selection instructions, and parking action instructions. Among them, the vehicle driving instructions include forward, backward, left turn ahead, right turn ahead, etc.; the parking space selection instructions include selecting the parking space on the left side of the vehicle, selecting the parking space on the right side of the vehicle, selecting to park on the first basement floor (B1), selecting to park on the second basement floor (B2), etc.; the parking action instructions include parking in the current parking space, parking out of the current parking space, etc.

[0104] For example, if the user parking instruction is "Please park the car in the parking space on the B2 floor", the corresponding vehicle control instruction is "Select to park on the second basement floor (B2)".

[0105] In some embodiments, the user parking instruction can be input into the large language model to obtain the corresponding vehicle control instruction.

[0106] In the embodiments of the present disclosure, after sending the current state of the vehicle, the first instruction, and the surrounding environment information of the vehicle to the cloud, the cloud can determine the prompt word according to the parking state and the first instruction, and input the prompt word and the surrounding environment information of the vehicle into the deep learning model, so that the deep learning model can perform scene recognition on the surrounding environment information of the vehicle based on the prompt word, making the scene recognition result more in line with the user's intention and the current state of the vehicle, improving the accuracy of scene recognition, and further improving the accuracy of vehicle control.

[0107] It should be noted that in the embodiments of the present disclosure, the current state of the vehicle and the first instruction can be sent first, and then the surrounding environment information of the vehicle can be sent; or the surrounding environment information of the vehicle can be sent first, and then the current state of the vehicle and the first instruction can be sent; or, the surrounding environment information of the vehicle, the current state of the vehicle, and the first instruction can be sent at the same time. The present disclosure does not make any limitations in this regard.

[0108] In some embodiments, the vehicle can also be controlled according to the vehicle control instruction corresponding to the user parking instruction, combined with the scene recognition result, the road information, and the position information of the surrounding objects of the vehicle, so that the vehicle can better understand the user's intention and realize the control of the vehicle.

[0109] Figure 4 is a flowchart of a vehicle control method shown according to some embodiments of the present disclosure, which is executed by the cloud, as Figure 4 shown, and includes the following steps:

[0110] Step 401, obtain the surrounding environment information of the vehicle.

[0111] In some embodiments, receive the surrounding environment information of the vehicle sent by the vehicle.

[0112] In some embodiments, the surrounding environment information of the vehicle can be obtained by the vehicle processing the vehicle's current collected vehicle perimeter images based on the current state.

[0113] Among them, the current state of the vehicle can be a cruise state, a parking state, and other states. The present disclosure does not limit this.

[0114] Among them, the cruise state is the state where the vehicle travels at a certain speed and searches for the target parking space. For example, in the cross-floor parking scenario, the parking state of the vehicle before driving to the target parking space on the target floor is the cruise state.

[0115] Among them, the parking state is the state where the vehicle parks in the target parking space after driving around the target parking space.

[0116] In some embodiments, when the parking state of the vehicle is the cruise state, the surrounding environment information includes signs.

[0117] In some embodiments, when the parking state of the vehicle is the parking state, the surrounding environment information includes a target image including the target parking space and the surrounding obstacles.

[0118] Step 402, input the surrounding environment information into the deep learning model to obtain a scene recognition result.

[0119] In some embodiments, the surrounding environment information and the first prompt word can be input into the deep learning model to obtain a scene recognition result.

[0120] Among them, the first prompt word is used to instruct the vision-language model to perform scene recognition on the surrounding environment information to obtain a scene recognition result corresponding to the surrounding environment information.

[0121] In some embodiments, the first prompt word can be used to instruct the vision-language model to recognize signs in the surrounding environment information, recognize obstacles, etc. The present disclosure does not limit this.

[0122] Among them, the scene recognition result can be the information included in the surrounding environment information.

[0123] In some embodiments, the current parking state of the vehicle and the first instruction may also be received, and then, based on the current parking state of the vehicle and the first instruction, a first prompt word is determined. The first instruction is a user parking instruction or a vehicle control instruction corresponding to the user parking instruction.

[0124] In some embodiments, the vehicle control instruction may include vehicle driving instructions, parking space selection instructions, and parking operation instructions. Among them, the vehicle driving instructions include forward, backward, left turn ahead, right turn ahead, etc.; the parking space selection instructions include select the parking space on the left side of the vehicle, select the parking space on the right side of the vehicle, select to park on the first basement floor (B1), select to park on the second basement floor (B2), etc.; the parking operation instructions include park in the current parking space, park out of the current parking space, etc.

[0125] For example, if the user parking instruction is "Please park the car in the parking space on the B2 floor", the corresponding vehicle control instruction is "Select to park on the second basement floor (B2)".

[0126] In some embodiments, according to the current state of the vehicle, a prompt word template may be determined, keywords of the first instruction are extracted to obtain target keywords, and the target keywords are filled into the prompt word template to obtain the first prompt word.

[0127] In some embodiments, when the first instruction is a user parking instruction, the vehicle control instruction corresponding to the user parking instruction may be determined first, and then keywords of the vehicle control instruction are extracted to obtain target keywords.

[0128] It should be noted that for different parking states, the concerned scenario information is different. For example, in the cruise state, the vehicle is more concerned about where to go; in the parking state, the vehicle is more concerned about whether there are obstacles near the target parking space, which affects parking in. Therefore, different prompt word templates may be set for different parking states, so that a scene recognition result more suitable for the current parking state can be obtained.

[0129] In some embodiments, the target keywords may be the floor where the vehicle stops, the target parking space where the vehicle stops, etc. The present disclosure does not limit this.

[0130] For example, if the first instruction is "Please park the car in the parking space on the B1 floor", the corresponding target keyword may be "B1 (the first basement floor)". The corresponding first prompt word may be "You are a driver who is driving. After seeing the sign information as shown in the figure and wanting to drive to B1 (the first basement floor), how should you drive the vehicle?"

[0131] For example, if the first instruction is "Please park the vehicle in the target parking space", the corresponding keyword can be "target parking space". The corresponding first prompt can be "You are a parking intelligent assistant. You need to briefly analyze the obstacle information inside and around the parking space based on the position information of the target parking space in the image." Or, "You are a parking intelligent assistant. Briefly analyze the obstacle information inside and around the parking space."

[0132] In some embodiments, when the current state of the vehicle is in a cruise state, the scene recognition result can include driving suggestions for the vehicle and / or the meaning corresponding to each identifier in the signpost.

[0133] For example, if the first prompt is "You are a driver who is driving. After seeing the signpost shown in the figure and wanting to drive to B1 (the first basement floor), how should you drive the vehicle?", and the image information including the signpost is Figure 5 , then the corresponding scene recognition result can include "Parking lot sign: Clearly indicates that this is the path to the parking lot; Arrow direction: The arrow points to the right, indicating that the vehicle needs to drive to the right; B1 (the first basement floor): Indicates that the destination is the first basement floor. Therefore, you should drive the vehicle to the right and enter the parking lot on the first basement floor (B1) along the direction indicated by the sign."

[0134] In some embodiments, when it is possible to give driving suggestions based on the image information including the signpost and the first prompt, the deep learning model outputs driving suggestions for the vehicle. When it is not possible to give driving suggestions based on the signpost image and the first prompt, it outputs the meaning corresponding to each identifier in the signpost.

[0135] In some embodiments, the deep learning model can also directly output the meaning corresponding to each identifier in the signpost, and the vehicle performs path planning based on the meaning corresponding to each identifier in the signpost.

[0136] In some embodiments, when the current state of the vehicle is in a parking state, the scene recognition result can include the position information of the target parking space and the obstacles around it.

[0137] For example, if the first prompt is "You are a parking intelligent assistant. You need to briefly analyze the obstacle information inside and around the parking space based on the position information of the target parking space in the image.", then the corresponding scene recognition result can include "Inside the parking space enclosed by the four corner points: None; Outside the line from corner point 2 to corner point 3: None; Outside the line from corner point 0 to corner point 1: None; Outside the line from corner point 3 to corner point 0: There is a red fire box and cardboard boxes; Outside the line from corner point 1 to corner point 2: There is a cat."

[0138] In the embodiments of the present disclosure, according to the current state of the vehicle and the first instruction, a first prompt word is determined. Based on the first prompt word, a deep learning model is prompted to perform scene recognition on the surrounding environment information, so that the determined scene recognition result is more in line with the user's intention and the current state of the vehicle.

[0139] Step 403: Send the scene recognition result.

[0140] In the embodiments of the present disclosure, the surrounding environment information of the vehicle is obtained; the surrounding environment information is input into the deep learning model to obtain a scene recognition result; the scene recognition result is sent. Thus, by using the deep learning model to perform scene recognition on the surrounding environment information of the vehicle, a more comprehensive and accurate scene recognition result can be obtained, providing data support for accurately controlling the vehicle.

[0141] To implement the above embodiments, the present disclosure also proposes a vehicle control device.

[0142] Figure 6 is a block diagram of a vehicle control device shown according to some embodiments of the present disclosure. Referring to Figure 6 , the device includes:

[0143] A first acquisition module 601, configured to acquire a user parking instruction;

[0144] A determination module 602, configured to respond to the user parking instruction and determine the surrounding environment information of the vehicle;

[0145] A second acquisition module 603, configured to acquire a scene recognition result corresponding to the surrounding environment information of the vehicle, where the scene recognition result is obtained by processing the surrounding environment information of the vehicle based on a deep learning model;

[0146] A control module 604, configured to determine a parking path plan according to the scene recognition result and the user parking instruction, and control the vehicle to perform automatic parking according to the parking path plan.

[0147] In some embodiments, the first acquisition module 601 is configured to:

[0148] Recognize the user's voice and / or body movements to acquire the user parking instruction;

[0149] Acquire the user parking instruction according to the start text operation input by the user;

[0150] Acquire the user parking instruction according to the control operation of the user clicking on the vehicle.

[0151] In some embodiments, the determination module 602 is configured to:

[0152] Determine the current state of the vehicle;

[0153] Determine the surrounding environment information according to the current state of the vehicle.

[0154] In some embodiments, the determining module 602 is configured to:

[0155] When the current state of the vehicle is the cruise state, the surrounding environment information includes image information containing signs.

[0156] In some embodiments, the determining module 602 is configured to:

[0157] When the current state of the vehicle is the parking state, the surrounding environment information includes a target image containing the target parking space where the vehicle is to be parked and the surrounding obstacles.

[0158] In some embodiments, the control module 604 is configured to:

[0159] During the process of the vehicle parking into the target parking space, if a preset event is detected for the vehicle, stop parking into the target parking space and perform at least one of determining a new target parking space and prompting the user to take over.

[0160] In some embodiments, the preset event includes any one of the following situations:

[0161] The vehicle has a collision risk, the target parking space is too narrow for the vehicle to park in, or a command to stop parking is received from the user.

[0162] In some embodiments, it further includes a synchronization module, which is configured to:

[0163] Synchronize the current state of the vehicle by means of display on the in-vehicle screen;

[0164] Synchronize the current state of the vehicle by means of voice playback.

[0165] In some embodiments, it further includes a processing module, which is configured to determine the road information corresponding to the location of the vehicle and the location information of the surrounding targets of the vehicle. Determine the parking path planning according to the scene recognition result and the user's parking instruction, including the control module 604, which is configured to:

[0166] Determine the parking path planning according to the scene recognition result, the user's parking instruction, the road information and the location information of the surrounding targets of the vehicle.

[0167] Regarding the device in the above embodiments, the specific ways for each module to perform operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0168] The vehicle control device according to the embodiments of the present disclosure first obtains a user's parking instruction, then responds to the user's parking instruction to determine the surrounding environment information of the vehicle, and further obtains a scene recognition result corresponding to the surrounding environment information of the vehicle, where the scene recognition result is obtained by processing the surrounding environment information of the vehicle based on a deep learning model. Finally, a parking path plan is determined according to the scene recognition result and the user's parking instruction, and the vehicle is controlled to park automatically according to the parking path plan. Thus, during the process of the vehicle cruising to the parking space, searching for the parking space, and parking into the parking space, the surrounding environment information of the vehicle can be recognized in real time, and the vehicle can be accurately controlled by combining the scene recognition result and the user's parking instruction, improving the safety and accuracy during the vehicle's automatic parking process. Moreover, the vehicle can be controlled to park automatically into the parking space selected by the user or the available parking space autonomously selected by the vehicle, thereby improving the parking efficiency and enhancing the user experience.

[0169] Figure 7 is a block diagram of a vehicle control device shown according to some embodiments of the present disclosure. Referring to Figure 7 ,the device includes:

[0170] An acquisition module 701, configured to acquire the surrounding environment information of the vehicle;

[0171] A processing module 702, configured to input the surrounding environment information into a deep learning model to obtain a scene recognition result;

[0172] A sending module 703, configured to send the scene recognition result.

[0173] In some embodiments, when the current state of the vehicle is the cruising state, the scene recognition result includes driving suggestions for the vehicle and / or the meaning corresponding to each identifier in the sign; or,

[0174] when the current state of the vehicle is the parking state, the scene recognition result includes the position information of the target parking space and the surrounding obstacles.

[0175] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0176] The vehicle control device according to the embodiments of the present disclosure acquires the surrounding environment information of the vehicle; inputs the surrounding environment information into a deep learning model to obtain a scene recognition result; and sends the scene recognition result. Thus, by using the deep learning model to perform scene recognition on the surrounding environment information of the vehicle, a more comprehensive and accurate scene recognition result can be obtained, providing data support for accurately controlling the vehicle.

[0177] Figure 8FIG. 0 is a block diagram of a vehicle 800 shown in accordance with an exemplary embodiment. For example, vehicle 800 may be a hybrid vehicle, or a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicles. Vehicle 800 may be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.

[0178] Referring Figure 8 , vehicle 800 may include various subsystems. For example, an infotainment system 810, a perception system 820, a decision control system 830, a drive system 840, and a computing platform 850. Among them, vehicle 800 may also include more or fewer subsystems, and each subsystem may include multiple components. In addition, each subsystem and each component of vehicle 800 may be interconnected by wired or wireless means.

[0179] In some embodiments, the infotainment system 810 may include a communication system, an entertainment system, and a navigation system, etc. The perception system 820 may include several types of sensors for sensing information about the environment around vehicle 800. For example, the perception system 820 may include a global positioning system (the global positioning system may be a GPS system, or a Beidou system, or other positioning systems), an inertial measurement unit (IMU), lidar, millimeter wave radar, ultrasonic radar, and a camera device.

[0180] The decision control system 830 may include a computing system, a vehicle controller, a steering system, an accelerator, and a braking system. The drive system 840 may include components that provide motive power for vehicle 800. In one embodiment, the drive system 840 may include an engine, an energy source, a transmission system, and wheels. The engine may be one or a combination of an internal combustion engine, an electric motor, and an air compression engine. The engine can convert the energy provided by the energy source into mechanical energy.

[0181] Some or all of the functions of vehicle 800 are controlled by the computing platform 850. The computing platform 850 may include at least one processor 851 and a memory 852. The processor 851 may execute instructions 853 stored in the memory 852.

[0182] The processor 851 can be any conventional processor, such as a commercially available CPU. The processor may also include, for example, a Graphic Process Unit (GPU), a Field Programmable Gate Array (FPGA), a System on Chip (SOC), an Application Specific Integrated Circuit (ASIC), or a combination thereof.

[0183] The memory 852 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk.

[0184] In addition to the instructions 853, the memory 852 can also store data, such as road maps, route information, data on the position, direction, speed, etc. of the vehicle. The data stored in the memory 852 can be used by the computing platform 850. In the embodiments of the present disclosure, the processor 851 can execute the instructions 853 to complete all or part of the steps of the above-described vehicle control method.

[0185] To implement the above embodiments, the present disclosure also proposes an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the vehicle control method proposed in the foregoing embodiments of the present disclosure.

[0186] In some embodiments, a server of the vehicle is deployed in the electronic device, where the server can also become the cloud of the vehicle.

[0187] Fig. 9 A block diagram of an exemplary electronic device suitable for implementing the embodiments of the present disclosure is shown. Fig. 9 The shown electronic device 12 is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0188] As Fig. 9 shown, the electronic device 12 is presented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).

[0189] Bus 18 represents one or more of several types of bus architectures, including a memory bus or memory controller, a peripheral bus, an Accelerated Graphics Port, a processor, or a local bus using any of the various bus architectures. By way of example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnection (PCI) bus.

[0190] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and nonvolatile media, removable and non-removable media.

[0191] Memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and / or cache memory 32. Electronic device 12 may further include other removable / non-removable, volatile / nonvolatile computer system storage media. By way of example only, storage system 34 can be used for reading and writing on non-removable, nonvolatile magnetic media ( Fig. 9 not shown, typically referred to as a "hard disk drive"). Although Fig. 9 not shown in the figure, a disk drive for reading and writing on a removable nonvolatile disk (such as a "floppy disk") and an optical disk drive for reading and writing on a removable nonvolatile optical disk (such as Compact Disc Read Only Memory (CD-ROM), Digital Video Disc Read Only Memory (DVD-ROM), or other optical media) can be provided. In these cases, each drive can be connected to bus 18 through one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the various embodiments of the present disclosure.

[0192] A program / utility 40 having a set (at least one) of program modules 42 can be stored, for example, in a memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. The program modules 42 generally execute the functions and / or methods in the embodiments described in the present disclosure.

[0193] The electronic device 12 can also communicate with one or more external devices 14 (such as a keyboard, a pointing device, a display 24, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device 12, and / or communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (such as a network card, a modem, etc.). Such communication can be carried out through an input / output (I / O) interface 22. Moreover, the electronic device 12 can also communicate with one or more networks (such as a Local Area Network (LAN), a Wide Area Network (WAN), and / or a public network, such as the Internet) through a network adapter 20. As shown in the figure, the network adapter 20 communicates with other modules of the electronic device 12 through a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0194] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the methods mentioned in the foregoing embodiments.

[0195] To implement the above embodiments, the present disclosure also proposes a chip, including: The chip includes a processing circuit configured to execute the vehicle control method provided in the foregoing embodiments.

[0196] The present disclosure also provides a computer-readable storage medium, on which computer program instructions are stored, and when the program instructions are executed by a processor, the steps of the vehicle control method provided by the present disclosure are implemented.

[0197] In addition, as used herein, the word "exemplary" is used to mean serving as an example, instance, or illustration. Any aspect or design described herein as "exemplary" is not necessarily to be construed as advantageous over other aspects or designs. Rather, the word exemplary is intended to present concepts in a concrete fashion. As used herein, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or clear from the context, "X applies A or B" is intended to mean any of the natural inclusive permutations. That is, if X applies A; X applies B; or X applies both A and B, then "X applies A or B" is satisfied under any one of the foregoing instances. Additionally, unless otherwise specified or clear from the context that it is referring to the singular form, the articles "a" and "an" as used in this application and the appended claims are generally understood to mean "one or more".

[0198] Likewise, although the present disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding this specification and the drawings. The present disclosure includes all such modifications and variations and is limited only by the scope of the claims. Specifically with respect to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, the terms used to describe such components are intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if not structurally equivalent to the disclosed structure. Additionally, although certain features of the present disclosure may have been disclosed with respect to only one of several implementations, such features may, as may be desired and advantageous for any given or particular application, be combined with one or more other features of other implementations. Further, with respect to the terms "comprising", "possessing", "having", "with", or variants thereof as used in the detailed description or claims, such terms are intended to be inclusive in a manner similar to the term "including".

[0199] Other embodiments of the present disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known or customary techniques in the art that are not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0200] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes may be made without departing from its scope. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A vehicle control method, characterized in that: The method comprises: Get user parking instructions; In response to the user's parking instruction, determining the surrounding environment information of the vehicle; Obtaining a scene recognition result corresponding to the surrounding environment information of the vehicle, wherein the scene recognition result is obtained by processing the surrounding environment information of the vehicle based on a deep learning model; A parking path plan is determined according to the scene recognition result and the user parking instruction, and the vehicle is controlled to perform automatic parking according to the parking path plan.

2. The method according to claim 1, characterized in that: The obtaining of the user's parking instruction includes any of the following: Recognize the user's voice and / or body language to obtain the user's parking instruction; According to the start text operation input by the user, obtaining the user's parking instruction; The user's parking instruction is obtained according to the user clicking on the control operation of the vehicle.

3. The method according to claim 1, characterized in that The determining of the surrounding environment information of the vehicle includes: determining a current state of the vehicle; The surrounding environment information is determined according to the current state of the vehicle.

4. The method according to claim 3, characterized in that The determining the surrounding environment information according to the current state of the vehicle includes: When the current state of the vehicle is a cruising state, the surrounding environment information includes image information including a sign.

5. The method according to claim 3, characterized in that: The determining the surrounding environment information according to the current state of the vehicle includes: When the current state of the vehicle is a parking state, the surrounding environment information includes a target image containing a target parking location where the vehicle is to be parked.

6. The method according to claim 5, characterized in that During the process of the vehicle parking in the target storage location, if it is detected that a preset event occurs to the vehicle, the vehicle stops parking in the target storage location, and at least one of determining a new target storage location and prompting a user to take over is performed.

7. The method according to claim 6, characterized in that The preset event includes any of the following situations: The vehicle is at risk of collision, the target storage space is too small to allow the vehicle to park, or a user's instruction to stop parking is received.

8. The method according to claim 3, characterized in that The method further comprises any of the following: Synchronize the current status of the vehicle by displaying it on the vehicle screen; The current status of the vehicle is synchronized by voice playback.

9. The method according to any one of claims 1 to 8, characterized in that: The method further includes: determining road information corresponding to the location of the vehicle and location information of objects around the vehicle, and determining parking path planning according to the scene recognition result and the user parking instruction, including: The parking path planning is determined according to the scene recognition result, the user parking instruction, the road information and the position information of the target objects around the vehicle.

10. A vehicle control method, characterized in that: The method comprises: Obtaining the surrounding environment information of the vehicle; Inputting the surrounding environment information into a deep learning model to obtain a scene recognition result; The scene recognition result is sent.

11. The method according to claim 10, characterized in that When the current state of the vehicle is a cruising state, the scene recognition result includes a vehicle driving suggestion and / or a meaning corresponding to each mark in the sign; or, When the current state of the vehicle is a parking state, the scene recognition result includes location information of the target storage location and its surrounding obstacles.

12. A vehicle control method and device, characterized in that: The device comprises: A first acquisition module is used to acquire a user's parking instruction; A determination module, configured to respond to the user's parking instruction and determine the surrounding environment information of the vehicle; A second acquisition module is used to acquire a scene recognition result corresponding to the surrounding environment information of the vehicle, wherein the scene recognition result is obtained by processing the surrounding environment information of the vehicle based on a deep learning model; A control module is used to determine a parking path plan according to the scene recognition result and the user parking instruction, and control the vehicle to perform automatic parking according to the parking path plan.

13. A vehicle control method and device, characterized in that: The device comprises: An acquisition module, used to acquire the surrounding environment information of the vehicle; A processing module, used to input the surrounding environment information into a deep learning model to obtain a scene recognition result; A sending module is used to send the scene recognition result.

14. A vehicle, characterized in that: include: processor; A memory for storing processor-executable instructions; wherein the processor is configured to: implement the steps of the method described in any one of claims 1-9.

15. An electronic device, characterized in that: include: processor; A memory for storing processor-executable instructions; wherein the processor is configured to: implement the steps of the method described in any one of claims 10-11.

16. A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a mobile terminal, it is capable of executing the steps of the method described in any one of claims 1 to 9, or executing the steps of the method described in any one of claims 10 to 11.

Images