Vehicle control method and device, vehicle and storage medium

Abstract

The invention provides a vehicle control method and device, a vehicle and a storage medium. The method comprises the steps that action fusion information of the vehicle in a current operation mode is acquired; and controlling the vehicle to park or park according to the action fusion information. According to the method, the auxiliary driving information is extracted from the data acquired by the multiple sensors on the vehicle in the same processing flow according to the current operation mode, the auxiliary driving information is fused to obtain the action fusion information in the previous operation mode, and the vehicle is controlled to run or park according to the action fusion information. According to the method, driving or parking of the vehicle is controlled to be combined into one processing flow, data sharing between driving and parking is achieved, the extraction accuracy of auxiliary driving information is improved, driving and parking of the vehicle are more intelligent, safer and more reliable, and therefore the vehicle using safety and experience of a user are guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicles, and in particular, to a control method for a vehicle, a control device for a vehicle, a vehicle, and a computer-readable storage medium. Background Art

[0002] In related technologies, most vehicle driving or parking adopts a single time-division multiplexing / multi-SoC (System on Chip) full-time integrated driving and parking solution. Taking the single-SoC time-division multiplexing integrated driving and parking solution as an example, the driving sensors are turned off in the parking mode, and the parking sensors are turned off in the driving mode. The cameras and radars in each mode are relied on to collect and process the video data in each mode, and the assisted driving information is extracted to control the vehicle driving or parking.

[0003] However, controlling the vehicle driving or parking in the above manner will result in incomplete or costly extraction of assisted driving information, reducing the user's vehicle use safety and experience. For example, using a single SoC, relying only on the perception of the parking surround-view camera and ultrasonic radar cannot detect distant targets or small objects, while using multiple SoCs will result in a more complex solution and higher deployment costs. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art.

[0005] To this end, an object of the present invention is to propose a control method for a vehicle, which improves the extraction accuracy of assisted driving information, makes the vehicle driving and parking more intelligent, safe and reliable, thereby ensuring the user's vehicle use safety and experience.

[0006] To this end, a second object of the present invention is to propose a control device for a vehicle.

[0007] To this end, a third object of the present invention is to propose a vehicle.

[0008] To this end, a fourth object of the present invention is to propose a computer-readable storage medium.

[0009] To achieve the above object, an embodiment of the first aspect of the present invention proposes a control method for a vehicle, the method includes: obtaining the action fusion information of the vehicle in the current operation mode; controlling the vehicle to drive or park according to the action fusion information.

[0010] The control method of a vehicle according to an embodiment of the present invention extracts auxiliary driving information from the data obtained by multiple sensors on the vehicle in the same processing flow according to the current operating mode, fuses the auxiliary driving information to obtain action fusion information in the previous operating mode, and controls the vehicle to drive or park according to the action fusion information, so as to realize the integration of controlling the vehicle to drive or park into one processing flow, realize data sharing between driving and parking, improve the extraction accuracy of auxiliary driving information, make the driving and parking of the vehicle more intelligent, safe and reliable, and thus ensure the user's vehicle use safety and experience.

[0011] In some embodiments, the action fusion information includes parking obstacle fusion information. Obtaining the action fusion information of the vehicle in the current operating mode includes: when determining that the current operating mode is the first operating mode, obtaining panoramic video stream data; determining the first obstacle information of the vehicle and the parking space information of the vehicle according to the panoramic video stream data; fusing the first obstacle information, the parking space information and the second obstacle information of the vehicle in the first operating mode to obtain the parking obstacle fusion information.

[0012] In some embodiments, the action fusion information includes driving obstacle fusion information. Obtaining the action fusion information of the vehicle in the current operating mode includes: when determining that the current operating mode is the second operating mode, obtaining forward-looking video stream data; determining the third obstacle information of the vehicle and the identification information of the vehicle according to the forward-looking video stream data; fusing the third obstacle information, the identification information and the fourth obstacle information of the vehicle in the driving operating mode to obtain driving obstacle fusion information.

[0013] In some embodiments, before fusing the first obstacle information, the parking space information and the second obstacle information of the vehicle in the first operating mode to obtain the parking obstacle fusion information, it further includes: obtaining forward-looking video stream data; determining the second obstacle information of the vehicle in the first operating mode according to the forward-looking video stream data.

[0014] In some embodiments, before fusing the third obstacle information, the identification information and the fourth obstacle information of the vehicle in the driving operating mode to obtain driving obstacle fusion information, it further includes: obtaining rear-view video stream data; determining the fourth obstacle information of the vehicle in the second operating mode according to the rear video stream data.

[0015] In some embodiments, before obtaining the action fusion data of the vehicle in the current operating mode, it further includes: determining that the vehicle performs parking data collection or the vehicle performs driving data collection.

[0016] In some embodiments, before obtaining the action fusion information of the vehicle in the current operation mode, the method further includes: establishing an operation module corresponding to the current operation mode of the vehicle.

[0017] To achieve the above object, an embodiment of the second aspect of the present invention provides a control device for a vehicle. The control device for the vehicle includes: an acquisition module configured to acquire action fusion information of the vehicle in the current operation mode; and a control module configured to control the vehicle to stop or park according to the action fusion information.

[0018] According to the control device for a vehicle of the embodiment of the present invention, data acquired by multiple sensors on the vehicle is used to extract assisted driving information in the same processing flow according to the current operation mode, and the assisted driving information is fused to obtain action fusion information in the previous operation mode. The vehicle is controlled to drive or park according to the action fusion information, so as to integrate the control of the vehicle to drive or park into one processing flow, realize data sharing between driving and parking, improve the extraction accuracy of assisted driving information, make the driving and parking of the vehicle more intelligent, safe and reliable, and thus ensure the vehicle use safety and experience of users.

[0019] To achieve the above object, an embodiment of the third aspect of the present invention provides a vehicle, which includes: the control device for a vehicle as required in the above embodiment.

[0020] According to the vehicle of the embodiment of the present invention, data acquired by multiple sensors on the vehicle is used to extract assisted driving information in the same processing flow according to the current operation mode, and the assisted driving information is fused to obtain action fusion information in the previous operation mode. The vehicle is controlled to drive or park according to the action fusion information, so as to integrate the control of the vehicle to drive or park into one processing flow, realize data sharing between driving and parking, improve the extraction accuracy of assisted driving information, make the driving and parking of the vehicle more intelligent, safe and reliable, and thus ensure the vehicle use safety and experience of users.

[0021] To achieve the above object, an embodiment of the fourth aspect of the present invention provides a computer-readable storage medium, on which a control program for a vehicle is stored. When the control program for the vehicle is executed by a processor, an apparatus installed with the control program for the vehicle implements the vehicle control method described in the above embodiment.

[0022] The additional aspects and advantages of the present invention will be partially given in the following description, partially become apparent from the following description, or be understood through the practice of the present invention. Description of the Drawings

[0023] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of embodiments in conjunction with the following drawings, wherein: Figure 1 is a control schematic diagram of an integrated driving and parking system according to an embodiment of the present invention; Figure 2 is a control flowchart of an integrated driving and parking system according to an embodiment of the present invention; Figure 3 is a flowchart of a vehicle control method according to an embodiment of the present invention; Figure 4 is an inference flowchart of an integrated driving and parking system according to an embodiment of the present invention; Figure 5 is a flowchart of a vehicle control method according to another embodiment of the present invention; Figure 6 is a block diagram of a vehicle control device according to an embodiment of the present invention; Figure 7 is a vehicle block diagram according to an embodiment of the present invention.

[0024] Reference numerals: Control message 60; Mode acquisition 61; Millimeter-wave radar 62; Forward-looking camera 63; First detection model 64; First visual perception and radar fusion 65; Surround-view camera 66; Second detection model 67; Second visual perception and radar fusion 68; Vehicle control 69; Ultrasonic radar 71; Integrated driving and parking system 72; System on chip 73; Acquisition module 98; Control module 99; Vehicle control device 100; Vehicle 101. Detailed implementation manners

[0025] The embodiments described with reference to the drawings are exemplary. Embodiments of the present invention are described in detail below.

[0026] Integrated driving and parking means integrating the driving and parking functions in a domain controller, realizing deep reuse of sensors and computing resources, so as to simultaneously provide functions such as high-speed driving assistance, urban driving assistance, and low-speed parking assistance. Integrating the originally separate driving and parking systems into one set improves the performance while bringing consumers an intelligent driving experience with seamless connection between multiple different scenarios.

[0027] In the related art, for example, the integrated driving and parking single SoC solution based on automotive-grade vision processor chips is divided into single SOC full-time operation and single SoC time-division multiplexing. Due to limited computing power, the automotive-grade vision processor chip cannot simultaneously call and process the data of all sensors for driving and parking. Therefore, most of the mass-produced integrated driving and parking solutions on the market are time-division multiplexing solutions. The driving sensors are turned off during parking, and the parking sensors are turned off during driving. Among them, the sensors are configured with 4 surround-view cameras, 1 front-view camera, 4 ultrasonic radars, and 1 millimeter-wave radar.

[0028] However, when using the above time-division multiplexing solution, there are problems that some functions cannot be realized or the experience is not good. For example, in the parking mode, to implement functions such as HPA (Home-zone Parking Assist) or AVP (Automated Valet Parking), during the process of entering the parking lot to find a parking space, relying only on the perception of the surround-view cameras and ultrasonic radars in the parking mode cannot detect distant targets or small objects, so as to avoid or bypass them in advance and solve the problem of emergency obstacle avoidance. Similarly, in the driving mode, in scenarios such as adjacent lane vehicles overtaking / merging in, relying only on the front-view camera in the driving mode cannot detect vehicles cutting in at close range, resulting in poor accuracy in predicting the cutting-in of the following vehicle.

[0029] Another example is that adopting the integrated driving and parking multi-SoC solution can improve the acquisition degree of auxiliary driving information. However, the deployment cost and complexity will be greatly increased.

[0030] Therefore, by using the vehicle control method of the embodiments of the present invention, the data obtained by multiple sensors on the vehicle is used to extract auxiliary driving information in the same processing flow according to the current operating mode, and the auxiliary driving information is fused to obtain the action fusion information in the previous operating mode. The vehicle is controlled to drive or park according to the action fusion information, so as to realize the combination of controlling the vehicle to drive or park into one processing flow, realize data sharing between driving and parking, improve the extraction accuracy of auxiliary driving information, make the driving and parking of the vehicle more intelligent, safe and reliable, and thus ensure the user's vehicle use safety and experience.

[0031] The above vehicle control method controls the driving or parking of the vehicle based on the integrated driving and parking system. The integrated driving and parking system will be described by way of example below.

[0032] First, refer to Figure 1 the integrated driving and parking system described in the embodiments of the present invention.

[0033] As Figure 1As shown, it is a control schematic diagram of the integrated driving and parking system according to an embodiment of the present invention. The control of the integrated driving and parking system according to the embodiment of the present invention is based on a single SoC of an automotive-grade vision processor chip. Among them, the automotive-grade vision processor chip is mainly used for autonomous driving and advanced driver assistance systems (ADAS). For example, it uses a C66x DSP (Digital Signal Processor) and an ARM Cortex-R5F processor, as well as multiple hardware accelerators, including an image processing unit (IPU), a video processing unit (VPU), and a deep learning accelerator (DLA), which can achieve efficient image processing and deep learning calculations.

[0034] The control schematic diagram of the integrated driving and parking system includes: System on Chip 73, that is, SoC. SoC integrates the main functions of the system into one chip. Essentially, it is doing a complex IC (Integrated Circuit Design) design. SoC is the product of the development of integrated circuit design and manufacturing technology. It can integrate the entire system on one chip, mainly composed of multiple main processors, multiple processing engines (PE, Processing Engine), multiple peripherals, and a main memory unit, with the characteristic of high parallelism, and can complete multiple functions at the same time. Now, various circuits such as CPU (Central Processing Unit), DSP (Digital Signal Processor), digital circuits, analog circuits, memories, and on-chip programmable logic can be integrally realized on the SoC chip, comprehensively realizing functions such as image processing, voice processing, communication protocols, communication functions, and data processing.

[0035] Integrated driving and parking system 72 is a system that controls the vehicle to drive and park by extracting auxiliary driving information. Among them, the auxiliary driving information includes obstacle information, lane line information, and parking space information, etc.; parking includes HPA and AVP. HPA is the memory parking function, which can learn, record, and store the user's frequently used getting-off position, parking location, and parking driving path through the vehicle's sensors, so as to park the vehicle in the designated parking space on behalf of the user in certain specific scenarios; AVP is the "autonomous valet parking system". As an application of autonomous driving in the parking scenario, AVP realizes the full-automatic valet parking function. The ultimate goal is to replace the traditional manual valet parking, helping users save a lot of parking time and solving the pain point of queuing for parking during peak hours.

[0036] The integrated driving and parking system 72 obtains the operating mode of the vehicle in the mode acquisition 61 through the control message 60 in the system on chip 73. The operating mode includes, for example, the parking mode and the driving mode. For the driving mode, the network model detection of obstacles, traffic lights, lane lines, etc. is performed in the first detection model 64 by combining the image data collected by the front view camera 63. The detection results are sent to the first vision perception and radar fusion 65. The first vision perception and radar fusion 65 will fuse the information detected by the millimeter wave radar 62 with the detection results, and send the results after fusing the obstacles and the millimeter wave radar to the second vision perception and radar fusion 68 for fusion. For the parking mode, the network model detection of obstacle vehicles, parking spaces, etc. is performed in the second detection model 67 by combining the image data collected by the surround view camera 66 (such as a 4-channel surround view camera). The detection results are sent into the second vision perception and radar fusion 68. The second vision perception and radar fusion 68 will fuse the information detected by the ultrasonic radar 71 with the detection results, and send the results of detecting obstacles and vehicles, etc. from the image data collected by the rear view camera to the first vision perception and radar fusion 65 for fusion. The parking or driving of the vehicle is controlled in the vehicle control 69 according to the fusion results in the first vision perception and radar fusion 65 and the second vision perception and radar fusion 68.

[0037] The above combines parking and driving and runs them in one image processing process, realizing data interaction and sharing between driving and parking. The controller sends control messages to the SOC through inter-core communication according to different scenario settings, and obtains the current latest operating mode of the vehicle every time a frame of image is inferred, and performs different logical processing according to the driving and parking modes.

[0038] Correspondingly, the control flowchart corresponding to the control schematic diagram of the above integrated driving and parking system is as Figure 2 shown Figure 2 This is the control flowchart of the integrated driving and parking system according to an embodiment of the present invention. The control process of the integrated driving and parking system at least includes steps S10 - S17.

[0039] S10, start.

[0040] S11, resource initialization and kernel registration, and initialization of inter-core IPC (Inter-Process Communication).

[0041] S12, initialize the driving camera, parking camera, driving vision perception module and parking vision perception module, and the default startup mode is the parking mode.

[0042] S13, create the driving module and the parking module.

[0043] S14. Obtain the current video for network model inference.

[0044] S15. Determine whether the program ends. If so, execute step S16; otherwise, execute step S17.

[0045] S16. End.

[0046] S17. Update the current mode and execute step S14.

[0047] Based on the above vehicle collision avoidance system 1, the control method of the vehicle according to the embodiments of the present invention will be described below in conjunction with Figures 1-5 Describe the control method of the vehicle according to the embodiments of the present invention.

[0048] As Figure 3 shown, it is a flowchart of the control method of the vehicle according to an embodiment of the present invention. The control method of the vehicle according to the embodiments of the present invention at least includes step S1 and step S2.

[0049] Step S1. Obtain the action fusion information of the vehicle in the current operating mode.

[0050] In the embodiment, as Figure 1 shown, the current operating mode is a mode adapted to the current driving demand, including a parking mode, a driving mode, etc. The parking mode and the driving mode will be used as examples for subsequent description; the action fusion information is the information obtained by performing corresponding fault, lane line, and parking space detections on the data acquired by multiple sensors on the vehicle and fusing the detection results. Obtain the action fusion information of the vehicle in the current operating mode, for example: If the current operating mode is the parking mode, perform obstacle detection on the panoramic video stream data collected by the panoramic camera, and fuse the detection result with the detection result of the ultrasonic radar to obtain panoramic obstacle information; perform obstacle detection on the front video stream data collected by the front camera, and fuse the detection result with the detection result of the millimeter wave radar to obtain front obstacle information; splice the panoramic image according to the image data collected by the panoramic camera, and perform parking space information detection on the spliced panoramic image to obtain the parking space information of the vehicle; fuse the panoramic obstacle information, the front obstacle information, and the parking space information of the vehicle at the parking fusion node (the second vision perception and radar fusion 68) to obtain the action fusion information in the parking mode. Among them, in the acquisition of the action fusion information, every time a frame of image is passed to complete the acquisition of the fusion information, the current operating mode is updated, and then the fusion information is continuously obtained according to the next frame, and the cycle continues until the acquisition ends.

[0051] If the current operating mode is the driving mode, detect obstacle information from the front-view video stream data, and fuse the detection results with the detection results of the millimeter-wave radar to obtain the front-view obstacle information; detect traffic lights, lane lines, traffic signs, lane lines, and semantic segmentation, etc. from the front-view video stream data to obtain the identification information of the vehicle; perform obstacle detection on the rear-view video stream data collected by the rear-view camera in the panoramic camera, and fuse the detection results with the detection results of the ultrasonic radar to obtain the rear-view obstacle information; fuse the front-view obstacle information, the vehicle identification information, and the rear-view obstacle information at the driving fusion node (First Visual Perception and Radar Fusion 65) to obtain the action fusion information in the driving mode. Among them, in the acquisition of the action fusion information, every time a frame of image is passed to complete the acquisition of the fusion information, the current operating mode is updated, and then the fusion information is continuously obtained according to the next frame, and the cycle continues until the acquisition is completed.

[0052] Obtain the action fusion information of the vehicle in the current operating mode, so as to realize that the obstacle information obtained in the parking mode is more accurate and comprehensive obstacle information obtained by fusing the panoramic video stream data, the front-view video stream data, the detection results of the millimeter-wave radar, and the detection results of the ultrasonic radar, avoiding the problem in the prior art that only the panoramic video stream data or the detection results of the ultrasonic radar are relied on for obstacle detection in the parking mode, resulting in the inability to obtain long-distance or tiny obstacle information, and thus unable to perform early avoidance or emergency obstacle avoidance. Also, realize that the obstacle information obtained in the driving mode is more accurate and comprehensive obstacle information obtained by fusing the rear-view video stream data, the front-view video stream data, the detection results of the millimeter-wave radar, and the detection results of the ultrasonic radar, avoiding the problem in the prior art that only the front-view video stream data or the detection results of the millimeter-wave radar are relied on for obstacle detection in the driving mode, resulting in the inability to detect vehicles cutting in closely, and thus unable to perform early avoidance.

[0053] Step S2, control the vehicle to drive or park according to the action fusion information.

[0054] In the embodiment, as Figure 1 shown, control the vehicle to drive or park according to the action fusion information. For example: If the current operating mode is the parking mode, control the vehicle to park in the vehicle control 69 according to the action fusion information in the parking mode; if the current operating mode is the driving mode, control the vehicle to drive in the vehicle control 69 according to the action fusion information in the driving mode.

[0055] Control the vehicle for driving or parking according to the action fusion information, so as to realize the idea of using multi-sensors on the vehicle for full-time operation plus time-sharing control, combine parking and driving into one processing flow for operation, realize data interaction and sharing between driving and parking, solve the problem of incomplete auxiliary driving information extracted under time-sharing multiplexing of driving and parking, and solve the problem of high cost under full-time operation of multiple SoCs, improve the extraction accuracy of auxiliary driving information, make the driving and parking of the vehicle more intelligent, safe and reliable, and thus ensure the user's vehicle use safety and experience.

[0056] According to the vehicle control method of an embodiment of the present invention, the data acquired by multi-sensors on the vehicle is used to extract auxiliary driving information in the same processing flow according to the current operation mode, the auxiliary driving information is fused to obtain the action fusion information in the previous operation mode, and the vehicle is controlled to drive or park according to the action fusion information, so as to realize combining the control of the vehicle for driving or parking into one processing flow, realize data sharing between driving and parking, improve the extraction accuracy of auxiliary driving information, make the driving and parking of the vehicle more intelligent, safe and reliable, and thus ensure the user's vehicle use safety and experience.

[0057] In some embodiments, the action fusion information includes parking obstacle fusion information. Obtaining the action fusion information of the vehicle in the current operation mode includes: when it is determined that the current operation mode is the first operation mode, obtaining the panoramic video stream data; determining the first obstacle information of the vehicle and the parking space information of the vehicle according to the panoramic video stream data; fusing the first obstacle information, the parking space information and the second obstacle information of the vehicle in the first operation mode to obtain the parking obstacle fusion information.

[0058] In the embodiment, the current operation mode of the vehicle is a mode adapted to the current driving requirement, including a parking mode, a driving mode, etc.; the first operation mode is, for example, a parking mode. In the prior art, the first operation mode is to perform image and video acquisition using the panoramic camera on the vehicle and then perform corresponding processing to control obstacle avoidance and parking space detection of the vehicle in the first operation mode, while the first operation mode in this embodiment will use the front camera and the panoramic camera for image and video acquisition; the first obstacle information is the information obtained by performing obstacle detection on the panoramic video stream data acquired by the panoramic camera and fusing the detection result with the detection result of the ultrasonic radar; the second obstacle information is the information obtained by performing obstacle detection on the front video stream data acquired by the front camera and fusing the detection result with the detection result of the millimeter wave radar; such as Figure 1As shown, in mode acquisition 61, the current operating mode of the vehicle is acquired. When it is determined that the current operating mode of the vehicle is the parking mode, the panoramic video stream data collected by the surround-view cameras on the vehicle is acquired, preparing data for extracting auxiliary driving information in the parking control inference based on the panoramic video stream data; in the second detection model 67, the corresponding detection model is selected for detection according to the scenario of the current parking mode. The detection models include models for obstacle information detection and vehicle parking space information detection. The obstacle information in the panoramic video stream data is detected, and the detection result is fused with the detection result of the ultrasonic radar to obtain the first obstacle information. The panoramic image is stitched based on the image data collected by the surround-view cameras, and the parking space information of the vehicle is detected from the stitched panoramic image, preparing for controlling the vehicle to park; in the second visual perception and radar fusion 68, that is, the parking fusion node, the first obstacle information, the parking space information, and the second obstacle information of the vehicle in the first operating mode are fused, and the obstacle recognition result after multi-sensor fusion is output to obtain the parking obstacle fusion information, so as to achieve that the obstacle information obtained in the parking mode is more accurate and comprehensive obstacle information fused from the panoramic video stream data, the forward video stream data, the millimeter-wave radar detection result, and the ultrasonic radar detection result, avoiding the problem in the prior art that only the panoramic video stream data or the detection result of the ultrasonic radar is relied on for obstacle detection in the parking mode, resulting in the inability to obtain long-distance or tiny obstacle information, and thus unable to perform early avoidance or emergency obstacle avoidance.

[0059] In some embodiments, the action fusion information includes the driving obstacle fusion information. Obtaining the action fusion information of the vehicle in the current operating mode includes: when it is determined that the current operating mode is the second operating mode, acquiring the forward video stream data; determining the third obstacle information of the vehicle and the identification information of the vehicle according to the forward video stream data; fusing the third obstacle information, the identification information, and the fourth obstacle information of the vehicle in the driving operating mode to obtain the driving obstacle fusion information.

[0060] In an embodiment, the current operating mode of the vehicle is a mode adapted to the current driving demand, including the parking mode, the driving mode, etc.; the second operating mode is, for example, the driving mode. In the prior art, the second operating mode is to perform image video acquisition using the forward camera on the vehicle and then perform corresponding processing to control the detection of obstacles, traffic lights, lane lines, etc. of the vehicle in the second operating mode, while the second operating mode in this embodiment will use both the forward camera and the surround-view cameras for image video acquisition; the identification information of the vehicle is the effective identification information required during the driving of the vehicle, including information such as traffic lights, lane lines, traffic signs, and semantic segmentation; the fourth obstacle information is the information obtained by detecting the obstacle in the video stream data collected by the rear-view camera in the surround-view cameras and fusing the detection result with the detection result of the ultrasonic radar; such as Figure 1As shown in the figure, in mode acquisition 61, the current operating mode of the vehicle is acquired. When it is determined that the current operating mode of the vehicle is the driving mode, the forward video stream data collected by the forward-looking camera on the vehicle is acquired, which prepares data for extracting auxiliary driving information in the driving control inference based on the forward video stream data. In the first detection model 64, the corresponding detection model is selected for detection according to the scenario of the current driving mode. The detection models include models for detecting obstacles, traffic lights, and lane lines. The forward video stream data is used to detect obstacle information, and the detection result is fused with the detection result of the millimeter wave radar to obtain the third obstacle information. The forward video stream data is also used to detect traffic lights, lane lines, traffic signs, lane lines, and semantic segmentation, etc., to obtain the identification information of the vehicle, which prepares for controlling the vehicle to park. In the first vision perception and radar fusion 65, that is, the driving fusion node, the third obstacle information, the identification information, and the fourth obstacle information of the vehicle in the driving operation mode are fused, and the obstacle recognition result after multi-sensor fusion is output to obtain the driving obstacle fusion information, so as to realize that the obstacle information obtained in the driving mode is more accurate and comprehensive obstacle information fused from the rear-view video stream data, the forward-view video stream data, the detection result of the millimeter wave radar, and the detection result of the ultrasonic radar, avoiding the problem in the prior art that only the forward-view video stream data or the detection result of the millimeter wave radar is relied on for obstacle detection in the driving mode, making it impossible to detect a vehicle cutting in at a close distance and thus unable to make an early avoidance.

[0061] For example, as Figure 4 shown, it is the inference flowchart of the integrated driving and parking system according to an embodiment of the present invention. The inference flowchart of the integrated driving and parking system at least includes steps S20 - step S31.

[0062] Step S20, parking video acquisition.

[0063] Step S21, determine whether the current is the parking mode. If so, execute step S22; otherwise, execute step S24.

[0064] Step S22, vehicle obstacle and parking space detection.

[0065] Step S23, parking fusion node.

[0066] Step S24, rear-view vehicle detection.

[0067] Step S25, fuse the detection result with driving and execute step S31.

[0068] Step S26, driving video acquisition.

[0069] Step S27, determine whether the current is the driving mode. If so, execute step S30; otherwise, execute step S28.

[0070] Step S28, obstacle detection.

[0071] Step S29, fuse the detection result with parking. Execute step S23.

[0072] Step S30, detect obstacles, traffic lights, and lane lines.

[0073] Step S31, driving fusion node.

[0074] In some embodiments, before fusing the first obstacle information, parking space information, and the second obstacle information of the vehicle in the first operating mode to obtain the parking obstacle fusion information, it further includes: obtaining the front view video stream data; determining the second obstacle information of the vehicle in the first operating mode according to the front view video stream data.

[0075] In an embodiment, as Figure 1 shown, obtaining the front view video stream data collected by the front view camera on the vehicle prepares data for extracting auxiliary driving information in the parking control inference based on the front view video stream data; if the current operating mode of the vehicle obtained in mode acquisition 61 is the parking mode, then perform obstacle detection on the front view video stream data in the first detection model 64, and at the same time fuse the detection result with the detection result of the millimeter wave radar 62 to obtain the second obstacle information, and send the second obstacle information to the parking fusion node for fusion processing to prepare for controlling the vehicle to park according to the obstacle detection result extracted from the front view video stream.

[0076] In some embodiments, before fusing the third obstacle information, identification information, and the fourth obstacle information of the vehicle in the driving operating mode to obtain the driving obstacle fusion information, it further includes: obtaining the rear view video stream data; determining the fourth obstacle information of the vehicle in the second operating mode according to the rear video stream data.

[0077] In an embodiment, as Figure 1 shown, obtaining the rear view video stream data collected by the rear view camera in the surround view camera of the vehicle prepares data for extracting auxiliary driving information in the driving control inference based on the rear view video stream data; if the current operating mode of the vehicle obtained in mode acquisition 61 is the driving mode, then perform obstacle detection on the rear view video stream data in the second detection model 67, and at the same time fuse the detection result with the detection result of the ultrasonic radar 71 to obtain the fourth obstacle information, and send the fourth obstacle information to the driving fusion node for fusion processing to prepare for controlling the vehicle to drive according to the obstacle detection result extracted from the rear view video stream.

[0078] In some embodiments, before obtaining the action fusion data of the vehicle in the current operating mode, it further includes: determining whether the vehicle performs parking data collection or driving data collection.

[0079] In an embodiment, parking data collection, for example, 4-channel surround cameras on the vehicle collect surround video stream data; driving data collection, for example, a front-view camera on the vehicle collects front-view video stream data; it is determined that the vehicle performs parking data collection or the vehicle performs driving data collection. For example, as Figure 1 shown, in the system-on-chip 73, modules and functional nodes related to parking and driving are all started, and the control inference for parking or driving is started. Specifically, it is determined whether the vehicle is in parking data collection or driving data collection. When in parking data collection, 4-channel surround cameras on the vehicle are used to collect surround video stream data, and when in driving data collection, a front-view camera on the vehicle is used to collect front-view video stream data, so as to prepare data for the control inference of parking or driving, such that the control inference of parking or driving is based on both parking data collection and driving data collection, rather than based on a single parking data collection or driving data collection for the control inference of parking or driving, thereby making the data information included in the inference process more comprehensive.

[0080] In some embodiments, before obtaining the action fusion information of the vehicle in the current operating mode, it further includes: establishing an operating module corresponding to the current operating mode of the vehicle.

[0081] In an embodiment, the operating module includes modules and functional nodes related to parking and driving, etc.; establishing an operating module corresponding to the current operating mode of the vehicle, for example, performing application resource initialization, registering all functional algorithms for driving and parking, then initializing the camera modules for driving and parking, and then initializing the visual perception modules for driving and parking, and setting the mode to the default parking mode; creating all functional nodes for driving and parking. In order to achieve the smoothness of the switch between driving and parking, all nodes are created here first; by establishing an operating module corresponding to the current operating mode of the vehicle, the software and hardware resources for precisely controlling the vehicle to park or drive are achieved.

[0082] Next, refer to Figure 5 to specifically describe the control method of the vehicle according to the embodiments of the present invention.

[0083] As Figure 5 shown, it is a flowchart of the control method of the vehicle according to another embodiment of the present invention. The control method of the vehicle according to the embodiments of the present invention includes at least steps S40 - S55.

[0084] Step S40, establish an operating module corresponding to the current operating mode of the vehicle.

[0085] Step S41, determine that the vehicle performs parking data collection or the vehicle performs driving data collection.

[0086] Step S42, parking video collection.

[0087] Step S43, determine whether it is the first operation mode currently. If so, execute Step S44; otherwise, execute Step S53.

[0088] Step S44, obtain the panoramic video stream data.

[0089] Step S45, determine the first obstacle information of the vehicle and the parking space information of the vehicle according to the panoramic video stream data.

[0090] Step S46, collect driving videos.

[0091] Step S47, determine whether it is the first operation mode currently. If so, execute Step S47; otherwise, execute Step S48.

[0092] Step S48, obtain the front view video stream data.

[0093] Step S49, determine the second obstacle information of the vehicle in the first operation mode according to the front view video stream data.

[0094] Step S50, fuse the first obstacle information, the parking space information and the second obstacle information of the vehicle in the first operation mode to obtain the parking obstacle fusion information.

[0095] Step S51, obtain the front view video stream data.

[0096] Step S52, determine the third obstacle information of the vehicle and the identification information of the vehicle according to the front view video stream data.

[0097] Step S53, obtain the rear view video stream data.

[0098] Step S54, determine the fourth obstacle information of the vehicle in the second operation mode according to the video stream data.

[0099] Step S55, fuse the third obstacle information, the identification information and the fourth obstacle information of the vehicle in the second operation mode to obtain the driving obstacle fusion information.

[0100] According to the vehicle control method of the embodiment of the present invention, by extracting the assisted driving information from the data obtained by multiple sensors on the vehicle in the same processing flow according to the current operation mode, fusing the assisted driving information to obtain the action fusion information in the front operation mode, and controlling the vehicle to drive or park according to the action fusion information, so as to realize combining the control of the vehicle to drive or park into one processing flow, realizing data sharing between driving and parking, improving the extraction accuracy of the assisted driving information, making the driving and parking of the vehicle more intelligent, safe and reliable, thereby ensuring the vehicle use safety and experience of the user.

[0101] Next, refer to Figure 6Describe the control device of the vehicle according to an embodiment of the present invention.

[0102] As Figure 6 shown, it is a block diagram of the control device of the vehicle according to an embodiment of the present invention. The control device 100 of the vehicle according to the embodiment of the present invention includes: an acquisition module 98, configured to acquire the action fusion information of the vehicle in the current operation mode; and a control module 99, configured to control the vehicle to park or stop according to the action fusion information.

[0103] According to the control device 100 of the vehicle according to the embodiment of the present invention, the acquisition module 98 extracts the assisted driving information from the data acquired by multiple sensors on the vehicle in the same processing flow according to the current operation mode, fuses the assisted driving information to obtain the action fusion information in the previous operation mode, and the control module 99 controls the vehicle to drive or park according to the action fusion information, so as to realize the integration of controlling the vehicle to drive or park into one processing flow, realize data sharing between driving and parking, improve the extraction accuracy of assisted driving information, make the driving and parking of the vehicle more intelligent, safe and reliable, and thus ensure the user's vehicle use safety and experience.

[0104] In some embodiments, the acquisition module 98 is configured to: the action fusion information includes parking obstacle fusion information. Acquiring the action fusion information of the vehicle in the current operation mode includes: when determining that the current operation mode is the first operation mode, acquiring the panoramic video stream data; determining the first obstacle information of the vehicle and the parking space information of the vehicle according to the panoramic video stream data; fusing the first obstacle information, the parking space information and the second obstacle information of the vehicle in the first operation mode to obtain the parking obstacle fusion information.

[0105] In some embodiments, the acquisition module 98 is configured to: the action fusion information includes driving obstacle fusion information. Acquiring the action fusion information of the vehicle in the current operation mode includes: when determining that the current operation mode is the second operation mode, acquiring the front view video stream data; determining the third obstacle information of the vehicle and the identification information of the vehicle according to the front view video stream data; fusing the third obstacle information, the identification information and the fourth obstacle information of the vehicle in the driving operation mode to obtain the driving obstacle fusion information.

[0106] In some embodiments, the acquisition module 98 is configured to: before fusing the first obstacle information, the parking space information and the second obstacle information of the vehicle in the first operation mode to obtain the parking obstacle fusion information, it further includes: acquiring the front view video stream data; determining the second obstacle information of the vehicle in the first operation mode according to the front view video stream data.

[0107] In some embodiments, the obtaining module 98 is configured to: fuse the third obstacle information, the identification information, and the fourth obstacle information of the vehicle in the driving operation mode. Before obtaining the driving obstacle fusion information, it further includes: obtaining the rear-view video stream data; determining the fourth obstacle information of the vehicle in the second operation mode according to the rear video stream data.

[0108] In some embodiments, the obtaining module 98 is configured to: before obtaining the action fusion data of the vehicle in the current operation mode, it further includes: determining that the vehicle performs parking data acquisition or the vehicle performs driving data acquisition.

[0109] In some embodiments, the obtaining module 98 is configured to: before obtaining the action fusion information of the vehicle in the current operation mode, it further includes: establishing an operation module corresponding to the current operation mode of the vehicle.

[0110] According to the vehicle control device 100 of the embodiments of the present invention, the data obtained by multiple sensors on the vehicle through the obtaining module 98 extracts the assisted driving information in the same processing flow according to the current operation mode, fuses the assisted driving information to obtain the action fusion information in the previous operation mode, and the control module 99 controls the vehicle to drive or park according to the action fusion information, so as to implement merging the control of the vehicle to drive or park into one processing flow, realizing data sharing between driving and parking, improving the extraction accuracy of the assisted driving information, making the driving and parking of the vehicle more intelligent, safe and reliable, thereby ensuring the user's vehicle use safety and experience.

[0111] Next, refer to Figure 7 Describe the vehicle of the embodiments of the present invention.

[0112] As Figure 7 shown, it is a block diagram of a vehicle according to an embodiment of the present invention. The vehicle 101 includes: the vehicle control device 100 as described in the above embodiments.

[0113] According to the vehicle 101 of the embodiments of the present invention, the vehicle 101 uses the vehicle control device 100 of the above embodiments. By extracting the assisted driving information in the same processing flow according to the data obtained by multiple sensors on the vehicle, fusing the assisted driving information to obtain the action fusion information in the previous operation mode, and controlling the vehicle to drive or park according to the action fusion information, so as to implement merging the control of the vehicle to drive or park into one processing flow, realizing data sharing between driving and parking, improving the extraction accuracy of the assisted driving information, making the driving and parking of the vehicle more intelligent, safe and reliable, thereby ensuring the user's vehicle use safety and experience.

[0114] Next, describe the computer-readable storage medium of the embodiments of the present invention.

[0115] The computer-readable storage medium according to an embodiment of the present invention stores a control program for a vehicle. When the control program for the vehicle is executed by a processor, the device installed with the control program for the vehicle implements the vehicle control method according to the above embodiment.

[0116] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "schematic embodiments", "examples", "specific examples", or "some examples", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example.

[0117] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the claims and their equivalents.

Claims

1. A vehicle control method, characterized in that: include: Acquire motion fusion information of the vehicle in the current operation mode; The vehicle is controlled to drive or park according to the action fusion information.

2. The vehicle control method according to claim 1, characterized in that: The action fusion information includes parking obstacle fusion information, and the acquiring the action fusion information of the vehicle in the current operation mode includes: When determining that the current operating mode is the first operating mode, acquiring surround view video stream data; Determining first obstacle information of the vehicle and parking space information of the vehicle according to the surround view video stream data; The first obstacle information, the parking space information, and the second obstacle information of the vehicle in the first operating mode are integrated to obtain the parking obstacle integration information.

3. The vehicle control method according to claim 1, characterized in that: The motion fusion information includes driving obstacle fusion information, and the acquiring of the motion fusion information of the vehicle in the current operation mode includes: When determining that the current operating mode is the second operating mode, acquiring forward-view video stream data; Determining third obstacle information of the vehicle and identification information of the vehicle according to the forward-looking video stream data; The third obstacle information, the identification information and the fourth obstacle information of the vehicle in the driving operation mode are integrated to obtain driving obstacle integration information.

4. The vehicle control method according to claim 2, characterized in that: Before fusing the first obstacle information, the parking space information, and the second obstacle information of the vehicle in the first operating mode to obtain the parking obstacle fusion information, the method further includes: Get the forward-looking video stream data; The second obstacle information of the vehicle in the first operating mode is determined based on the forward-looking video stream data.

5. The vehicle control method according to claim 3, characterized in that: Before fusing the third obstacle information, the identification information and the fourth obstacle information of the vehicle in the driving operation mode to obtain the driving obstacle fusion information, the method further includes: Get rear view video stream data; The fourth obstacle information of the vehicle in the second operating mode is determined according to the rear video stream data.

6. The vehicle control method according to claim 1, characterized in that: Before obtaining the motion fusion data of the vehicle in the current operation mode, the method further includes: It is determined that the vehicle performs parking data collection or the vehicle performs driving data collection.

7. The vehicle control method according to claim 1, characterized in that: Before obtaining the action fusion information of the vehicle in the current operation mode, the method further includes: An operation module corresponding to the current operation mode of the vehicle is established.

8. A vehicle control device, characterized in that: include: An acquisition module, used for acquiring the motion fusion information of the vehicle in the current operation mode; A control module is used to control the vehicle to stop or park according to the action fusion information.

9. A vehicle, characterized in that: include: A vehicle control device as claimed in claim 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a vehicle control program, and when the vehicle control program is executed by a processor, a device installed with the vehicle control program implements the vehicle control method as described in any one of claims 1-7.

Images