Vehicle control system and method and intelligent vehicle

Abstract

The invention provides a vehicle control system and method and an intelligent vehicle, relates to the technical field of intelligent vehicle safety, and aims to relieve the technical problem that in the prior art, an intelligent vehicle cannot autonomously avoid when encountering sudden dangerous situations. A hazard source monitoring device detects current hazard source data in a surrounding environment area and then sends a hazard source avoidance triggering instruction, so that a path planning device can be triggered to send a hazard source avoidance execution instruction after determining a current hazard source avoidance path of an intelligent vehicle; therefore, the whole vehicle control device can be further triggered to control the intelligent vehicle to execute the danger source avoiding action according to the current danger source avoiding path, mutual cooperation of the danger source monitoring device, the path planning device and the whole vehicle control device is achieved, and full-link connection from sudden danger source detection to danger source autonomous avoiding is completed. The bottleneck of'sensing-decision-execution 'chain breakage in the prior art is broken through, and the safety of the intelligent vehicle is improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent vehicle safety technology, and in particular to a vehicle control system, method and intelligent vehicle. Background Art

[0002] In intelligent vehicle technology, when an intelligent vehicle encounters a sudden dangerous situation, it often triggers local video recording and simple alarms to issue danger warnings. It lacks the ability to respond to dangerous situations independently and relies on human intervention to take risk avoidance actions, which can easily lead to safety problems due to human operation delays or negligence. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a vehicle control system, method and intelligent vehicle to enable the intelligent vehicle to autonomously avoid sudden dangerous situations and improve the safety of the intelligent vehicle.

[0004] In a first aspect, the present invention provides a vehicle control system comprising a hazard source monitoring device, a path planning device, and a vehicle control device; A hazard source monitoring device is configured to obtain current environmental data within a surrounding environment area corresponding to the intelligent vehicle; extract M consecutive frames of environmental images of the same size from an environmental video stream included in the current environmental data; detect the hazard source data in each frame of the environmental image using a hazard source detection model; determine, based on the hazard source data in each frame of the environmental image, when an overlap rate of hazard source areas in N consecutive frames of environmental images is greater than or equal to a first threshold and / or a confidence fluctuation degree is less than or equal to a second threshold, determine current hazard source data based on the hazard source data in each frame of the environmental image; wherein M is a positive integer greater than 1 and N is less than or equal to M; and send a hazard source avoidance trigger instruction based on the current hazard source data; A path planning device is configured to, upon receiving a hazard avoidance trigger instruction, determine a current hazard avoidance path corresponding to the intelligent vehicle based on the current hazard source data in the hazard avoidance trigger instruction; and send a hazard avoidance execution instruction based on the current hazard avoidance path; The vehicle control device is used to control the intelligent vehicle to perform hazard avoidance actions according to the current hazard avoidance path in the hazard avoidance execution instruction when receiving the hazard avoidance execution instruction.

[0005] Optionally, the vehicle control system provided by the present invention further includes a vehicle-mounted communication device; The vehicle-mounted communication device is configured to, upon receiving a hazard avoidance trigger instruction, forward the hazard avoidance trigger instruction to a user client, so that the user client performs a hazard avoidance prompt based on the current hazard source data in the hazard avoidance trigger instruction; and upon receiving a confirmation avoidance instruction returned by the user client, forward the hazard avoidance trigger instruction to the vehicle control device; The vehicle control device is also used to obtain the current vehicle status data of the intelligent vehicle when receiving a hazard avoidance trigger instruction, and when it is determined based on the current vehicle status data that the intelligent vehicle meets the intelligent driving conditions, send the hazard avoidance trigger instruction to the path planning device.

[0006] Optionally, the vehicle-mounted communication device is further configured to send a first wake-up instruction to the vehicle control device upon receiving a sentry mode activation instruction sent by a user client; wherein the sentry mode activation instruction is initiated by the user client when the smart vehicle is in a stationary state; The vehicle control device is further configured to wake up the in-vehicle network upon receiving the first wake-up instruction and send a second wake-up instruction to the hazard source monitoring device; The hazard source monitoring device is also used to obtain current environmental data in the surrounding environment area corresponding to the smart vehicle when receiving the second wake-up instruction.

[0007] Optionally, a path planning device is used to determine the current hazard level based on the current hazard source data, and determine the current hazard source avoidance action based on the current hazard level; based on the current hazard source avoidance action, determine the current hazard source avoidance path.

[0008] Optionally, a path planning device is used to determine the danger level corresponding to the trigger condition reached by the current danger source data as the current danger level based on the correspondence between the danger level and the trigger condition; and to determine the danger source avoidance action corresponding to the current danger level as the current danger source avoidance action based on the correspondence between the danger level and the danger source avoidance action.

[0009] Optionally, the vehicle control device is also used to detect an abnormal situation during the vehicle control process according to the abnormality detection method corresponding to various abnormal situations, and determine the detected abnormal situation as an abnormal situation to be processed; based on the correspondence between the abnormal situation and the processing strategy, determine the processing strategy corresponding to the abnormal situation to be processed as the target processing strategy; and process the abnormal situation to be processed based on the target processing strategy.

[0010] In a second aspect, the present invention provides a vehicle control method, which is applied to the hazard source monitoring device in the above-mentioned vehicle control system, comprising: Obtain current environmental data within the surrounding area corresponding to the smart vehicle; Extracting M consecutive frames of environmental images of the same size from an environmental video stream included in current environmental data; detecting the hazard source data in each frame of the environmental image using a hazard source detection model; determining, based on the hazard source data in each frame of the environmental image, when an overlap rate of hazard source regions in N consecutive frames of environmental images is greater than or equal to a first threshold and / or a confidence fluctuation degree is less than or equal to a second threshold, determining current hazard source data based on the hazard source data in each frame of the environmental image; wherein M is a positive integer greater than 1, and N is less than or equal to M; Based on the current hazard source data, a hazard source avoidance trigger instruction is sent to trigger the path planning device to determine the current hazard source avoidance strategy corresponding to the intelligent vehicle based on the current hazard source data in the hazard source avoidance trigger instruction, and then trigger the vehicle control device to control the intelligent vehicle to perform the hazard source avoidance action based on the current hazard source avoidance strategy in the hazard source avoidance execution instruction.

[0011] In a third aspect, the present invention provides another vehicle control method, which is applied to a path planning device in the above-mentioned vehicle control system, comprising: Receive a hazard source avoidance trigger instruction sent by a hazard source monitoring device; wherein, the hazard source avoidance trigger instruction is that the hazard source monitoring device extracts M frames of continuous environmental images of the same size from the environmental video stream contained in the current environmental data based on the current environmental data in the surrounding environment area corresponding to the intelligent vehicle; uses a hazard source detection model to detect the hazard source data in each frame of the environmental image; based on the hazard source data in each frame of the environmental image, when it is determined that the overlap rate of the hazard source area in N consecutive frames of environmental images is greater than or equal to a first threshold and / or the confidence fluctuation degree is less than or equal to a second threshold, the current hazard source data is determined based on the hazard source data in each frame of the environmental image; wherein, M is a positive integer greater than 1, and N is less than or equal to M; based on the current hazard source data, send a hazard source avoidance trigger instruction; Determine the current hazard avoidance strategy corresponding to the intelligent vehicle based on the current hazard source data in the hazard source avoidance trigger instruction; Based on the current hazard avoidance strategy, a hazard avoidance execution instruction is sent to trigger the vehicle control device to control the intelligent vehicle to perform hazard avoidance actions based on the current hazard avoidance strategy in the hazard avoidance instruction.

[0012] In a fourth aspect, the present invention provides an intelligent vehicle comprising the above-mentioned vehicle control system.

[0013] The beneficial effects of the present invention are as follows: In the vehicle control system, method and intelligent vehicle provided by the present invention, the hazard source monitoring device detects the current hazard source data in the surrounding environment area and then sends a hazard source avoidance trigger instruction, which can trigger the path planning device to send a hazard source avoidance execution instruction after determining the current hazard source avoidance path of the intelligent vehicle, thereby further triggering the whole vehicle control device to control the intelligent vehicle to perform hazard source avoidance actions according to the current hazard source avoidance path, thereby realizing the mutual coordination of the hazard source monitoring device, the path planning device and the whole vehicle control device, completing the full link from sudden hazard source detection to autonomous hazard source avoidance, breaking through the bottleneck of the "perception-decision-execution" chain in the existing technology, and improving the safety of the intelligent vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0015] Figure 1 A schematic diagram showing the composition framework of a vehicle control system provided by an embodiment of the present invention is shown; Figure 2 A schematic diagram of an interactive process of a vehicle control method provided by an embodiment of the present invention is shown; Figure 3 A schematic diagram showing an overview of the hazard source detection model training method provided by an embodiment of the present invention is shown; Figure 4 A functional structure diagram of a vehicle control device provided by an embodiment of the present invention is shown; Figure 5 A functional structure diagram of another vehicle control device provided by an embodiment of the present invention is shown; Figure 6 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0017] In order to facilitate those skilled in the art to better understand this application, the technical terms involved in this application are briefly introduced below.

[0018] The hazard source monitoring device is a device used to monitor whether there are hazards in the surrounding area of ​​the intelligent vehicle. In this application, the hazard source monitoring device can be but is not limited to a sentry mode controller.

[0019] A path planning device is a device used to determine the driving path of an intelligent vehicle. In this application, the path planning device can be but is not limited to an intelligent driving control; wherein, the hazard source monitoring device and the path planning device can be two independent controllers, or the hazard source monitoring device and the path planning device can be integrated into a single domain controller.

[0020] The vehicle control device is a device that controls the intelligent vehicle. In this application, the vehicle control device can be but is not limited to a vehicle controller, an electronic control unit (ECU), and a drive system.

[0021] It should be noted that the term "and / or" used in this application describes the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0022] The embodiment of the present invention provides a vehicle control system, referring to Figure 1 As shown, the vehicle control system provided by the embodiment of the present invention includes a hazard source monitoring device 110, a path planning device 120 and a vehicle control device 130; The hazard source monitoring device 110 is used to obtain current environmental data in the surrounding environment area corresponding to the intelligent vehicle; extract M frames of continuous environmental images of the same size from the environmental video stream contained in the current environmental data; use a hazard source detection model to detect the hazard source data in each frame of the environmental image; based on the hazard source data in each frame of the environmental image, determine that when the overlap rate of the hazard source area in N consecutive frames of environmental images is greater than or equal to a first threshold and / or the confidence fluctuation degree is less than or equal to a second threshold, determine the current hazard source data based on the hazard source data in each frame of the environmental image; wherein M is a positive integer greater than 1, and N is less than or equal to M; based on the current hazard source data, send a hazard avoidance trigger instruction; wherein the current hazard source data includes at least any one of fire, flood, smoke and abnormal vehicles; the hazard source monitoring device includes any one of a temperature sensor, a smoke sensor and a surround-view camera.

[0023] The path planning device 120 is configured to, upon receiving a hazard avoidance trigger instruction, determine a current hazard avoidance path corresponding to the intelligent vehicle based on the current hazard data in the hazard avoidance trigger instruction; and send a hazard avoidance execution instruction based on the current hazard avoidance path; The vehicle control device 130 is used to control the intelligent vehicle to perform a hazard avoidance action according to the current hazard avoidance path in the hazard avoidance execution instruction when receiving the hazard avoidance execution instruction.

[0024] During specific implementation, the hazard source monitoring device 110 can call environmental detection sensors to obtain current environmental data within the surrounding environment area corresponding to the smart vehicle. For example, environmental detection sensors include but are not limited to temperature sensors, smoke sensors, and surround-view cameras. Correspondingly, the hazard source monitoring device 110 can call temperature sensors to collect temperature data, call smoke sensors to collect smoke concentration data, and call surround-view camera sensors to collect environmental video stream data. Furthermore, the hazard source monitoring device 110 can input current environmental data into a hazard source detection model to obtain conventional hazard source data such as pedestrians and obstacles within the surrounding environment area corresponding to the smart vehicle as current hazard source data, thereby enabling detection and autonomous avoidance of different conventional hazard sources. In addition, after inputting current environmental data into the hazard source detection model, the hazard source monitoring device 110 can also obtain fire, flood, smoke, abnormal vehicles, etc. as current hazard source data, thereby enabling detection and autonomous avoidance of various types of conventional hazard sources while enabling detection and autonomous avoidance of various types of sudden hazard sources, thereby improving the performance of the vehicle control system and the safety of the smart vehicle.

[0025] In specific implementation, the hazard source monitoring device is used to extract M frames of environmental images of the same size and continuous from the environmental video stream contained in the current environmental data through the input layer of the hazard source detection model; detect the hazard source data in each frame of the environmental image through the detection layer of the hazard source detection model; and determine the current hazard source data based on the hazard source data in each frame of the environmental image through the output layer of the hazard source detection model, when the overlap rate of the hazard source area in N consecutive frames of environmental images is greater than or equal to a first threshold and / or the confidence fluctuation degree is less than or equal to a second threshold; wherein, M is a positive integer greater than 1, and N is less than or equal to M.

[0026] Among them, the input layer of the hazard source detection model continuously receives the environmental video stream and intercepts M frames of continuous environmental images of the same size at a fixed frequency to ensure temporal coherence and spatial consistency; the detection layer of the hazard source detection model detects the hazard source category (fire, flood, smoke, abnormal vehicles, etc.) in each frame of the environmental image, the position coordinates of the hazard source boundary box (that is, the hazard source area), and the confidence level (0-1 range) that represents the degree of certainty of the hazard source within the hazard source boundary box, and obtains the hazard source data in each frame of the environmental image; the output layer of the hazard source detection model calculates the hazard source in N consecutive frames of environmental images. The intersection over union (IoU) of the source area is used as the overlap rate. When the overlap rate is ≥ the first threshold (for example, 70%), it is determined that the same hazard source continues to exist in N consecutive frames of environmental images, and the spatial consistency verification is confirmed to have passed. The difference between the maximum and minimum values ​​of the confidence of the hazard source in the hazard source area in the N consecutive frames of environmental images is counted as the degree of confidence fluctuation. When the degree of confidence fluctuation is ≤ the second threshold (for example, 0.15), it is confirmed that the reliability verification of the detection result has passed, and the hazard source data that has passed the spatial consistency verification and / or the detection result reliability verification is determined as the current hazard source data.

[0027] In this embodiment of the present invention, the hazard source detection model can be a deep learning model such as YOLOv8, Faster R-CNN, Transformer (such as DETR), or lightweight MobileNet. After being trained on a cloud server, the hazard source detection model is loaded into the hazard source monitoring device for hazard source detection. The hazard source detection model comprises an input layer, a detection layer, and an output layer. In this embodiment of the present invention, an adaptive scaling layer is introduced in the input layer to uniformly scale environmental images of different resolutions to a reference size (such as 640×640). Images with significantly different aspect ratios are edge-padding to preserve the original hazard source morphological features. A Feature Pyramid Network (FPN) is integrated in the detection layer to enhance the sensitivity of detecting small hazard sources (such as 10×10 pixels) through multi-scale feature fusion. This allows the hazard source detection model to simultaneously capture details of both large near-field hazards and small distant ones, improving the accuracy of hazard source detection. At the output layer, the hazard sources detected by the detection layer are screened through spatial consistency verification and / or detection result reliability verification. Hazard source data that passes spatial consistency verification and / or detection result reliability verification is identified as the current hazard source data, further improving hazard source detection accuracy. Furthermore, at the detection layer, on the one hand, hazard sources in M ​​frames of environmental imagery can be dynamically tracked, and their motion trajectories can be predicted using an LSTM network or Kalman filter to eliminate transient misdetections of hazard sources (such as those caused by reflections, interference from moving light sources, etc.). On the other hand, a sliding window algorithm can be used to calculate the dynamic IOU change rate of the hazard source target box (i.e., a rectangular box defining the specific location and range of the hazard source in the image). When the dynamic IOU change rate falls below a threshold, the hazard source target box is filtered out as an invalid target box. In this way, through dynamic hazard source tracking and / or IOU change rate filtering, invalid hazard sources can be filtered out, thereby improving hazard source detection accuracy.

[0028] In a specific implementation, the vehicle control system provided by the embodiment of the present invention further includes a vehicle-mounted communication device 140; The vehicle-mounted communication device is configured to, upon receiving a hazard avoidance trigger instruction, forward the hazard avoidance trigger instruction to a user client, so that the user client performs a hazard avoidance prompt based on the current hazard source data in the hazard avoidance trigger instruction; and upon receiving a confirmation avoidance instruction returned by the user client, forward the hazard avoidance trigger instruction to the vehicle control device; The vehicle control device is also used to obtain the current vehicle status data of the intelligent vehicle when receiving a hazard avoidance trigger instruction, and when it is determined based on the current vehicle status data that the intelligent vehicle meets the intelligent driving conditions, send the hazard avoidance trigger instruction to the path planning device.

[0029] Furthermore, the vehicle-mounted communication device is also used to upload the current environmental data and hazard source data in the surrounding environment area corresponding to the smart vehicle obtained by the hazard source monitoring device to the cloud.

[0030] In the above-mentioned vehicle-mounted communication device and vehicle control device provided by the embodiments of the present invention, the hazard source avoidance trigger instruction is received by the vehicle-mounted communication device, and the hazard source avoidance trigger instruction is forwarded to the user client for processing, so that the user can make a hazard source avoidance prompt based on the received data; and when a confirmation avoidance instruction returned by the user client is received, the hazard source avoidance trigger instruction is forwarded to the vehicle control device; the vehicle control device obtains the current vehicle status data of the intelligent vehicle, and when it is determined that the vehicle has the conditions for automatic driving, it is forwarded to the path planning device based on the received hazard source avoidance trigger instruction, so that the path planning device plans the optimal avoidance path according to the received hazard source avoidance trigger instruction and vehicle status data, thereby realizing the active avoidance of the intelligent vehicle when encountering a hazard source, thereby improving driving safety and convenience.

[0031] In a specific implementation, the vehicle communication device is further configured to send a first wake-up instruction to the vehicle control device upon receiving a sentry mode activation instruction sent by a user client; wherein the sentry mode activation instruction is initiated by the user client when the smart vehicle is in a stationary state; The vehicle control device is further configured to wake up the in-vehicle network upon receiving the first wake-up instruction and send a second wake-up instruction to the hazard source monitoring device; The hazard source monitoring device is also used to obtain current environmental data in the surrounding environment area corresponding to the smart vehicle when receiving the second wake-up instruction.

[0032] In the above-mentioned on-vehicle communication device provided in the embodiment of the present application, after the on-vehicle communication device receives a sentry mode activation command initiated by the user through the client when the smart vehicle is stationary, it sends a first wake-up command (i.e., activates sentry mode) to the vehicle control device, thereby waking up the network system inside the vehicle; after receiving the first wake-up command, the vehicle control device wakes up and activates the hazard source monitoring device in the smart vehicle; after receiving the second wake-up command, the hazard source monitoring device can obtain the current environmental data in the surrounding area of ​​the smart vehicle, such as temperature, humidity, light intensity, etc., to provide real-time information for vehicle safety protection and prevent potential safety hazards. When the user activates the sentry mode, the on-vehicle communication device transmits the command to the vehicle control device, which then wakes up the hazard source monitoring device to perform environmental monitoring to realize the safety monitoring function of the smart vehicle when it is stationary.

[0033] In specific implementation, the path planning device is used to determine the current hazard level based on the current hazard source data, and determine the current hazard source avoidance action based on the current hazard level; based on the current hazard source avoidance action, determine the current hazard source avoidance path.

[0034] In an embodiment of the present invention, the path planning device can determine the danger level corresponding to the trigger condition reached by the current danger source data as the current danger level based on the correspondence between the danger level and the trigger condition; and determine the danger source avoidance action corresponding to the current danger level as the current danger source avoidance action based on the correspondence between the danger level and the danger source avoidance action.

[0035] Specifically, for example, if the current hazard source data is that the distance between the smart vehicle and the flame is less than or equal to 1 meter, the hazard level is Level 1. If the current hazard source data is that the smoke concentration is greater than 20%, the hazard level is Level 2. If the current hazard source data is that an abnormal vehicle is approaching the smart vehicle, the hazard level is Level 3. When the danger level is Level 1, the danger avoidance action is forced emergency reverse (the vehicle can move at a maximum speed of 8km / h); when the danger level is Level 2, the danger avoidance action is low-speed departure (the vehicle can move at a speed less than or equal to 5km / m); when the danger level is Level 3, the danger avoidance action is lateral translation (that is, the intelligent vehicle performs lateral avoidance with an avoidance distance of 0.5m).

[0036] Furthermore, when the danger level is Level 2 or Level 3, the on-board communication device forwards the danger source avoidance trigger instruction to the user client for processing, so that the user can make danger source avoidance prompts based on the received data. When the confirmation avoidance instruction returned by the user client is received, the danger source avoidance trigger instruction is forwarded to the vehicle control device, and the vehicle control device controls the intelligent vehicle to perform danger avoidance actions.

[0037] Furthermore, when the vehicle is in sentry mode in the parking lot, the path planning device determines the current hazard avoidance path based on the current hazard avoidance action according to a preset planning algorithm, specifically including the following steps: a. Safe area search: Search for all drivable areas, including open areas, exits, and open corners.

[0038] The search method for open locations includes: discretizing the parking lot map into a grid map or a topological map, with each grid marked as "passable" or "inpassable"; using the A* algorithm or the SPFA algorithm to calculate the shortest path from the vehicle's current location to each passable grid, and selecting areas with accessible and unobstructed paths as candidate open locations; performing a connectivity analysis on the candidate open locations, eliminating isolated candidate open locations that are not connected to the main channel, and selecting candidate open locations with an area larger than the square of the vehicle's minimum turning radius as the open location area.

[0039] Among them, the search method for the exit location includes: obtaining a set of candidate exits containing all exits pre-marked in the parking lot map; based on the path length, number of turns, and congestion probability (such as historical data or real-time sensor feedback) from the current location of the vehicle to each candidate exit, using the Dijkstra or A* algorithm to evaluate the path cost to obtain the path cost from the current location of the vehicle to each candidate exit; determining the candidate exit with the smallest path cost among all candidate exits as the exit location; the candidate exit with the smallest path cost is the exit with the shortest path, the fewest turns, and the fewest obstacles.

[0040] Among them, the search method for open corner areas includes: identifying all corner areas in the parking lot map (such as the four vertices of a rectangular area and the area surrounded by an L-shaped wall); performing a sector scan on each corner area to detect whether there are obstacles or occupation in the sector area, and after eliminating the corner areas with obstacles and occupied areas from each corner area, using the geometric expansion method to expand the vehicle outline by a set distance (for example, 0.5 to 1 meter) as a safe accommodation area. The corner areas in the remaining corner areas whose area is not less than the area of ​​the safe accommodation area are determined as corner areas that can accommodate vehicle parking, that is, open corner areas.

[0041] In the embodiment of the present application, it is also possible to search for each drivable area in a sector-shaped area or a circular area with a set distance (for example, 50 meters) as a radius; in this case, the passable grids, candidate exits, and corner areas in the sector-shaped area or the circular area are first screened out according to the parking map, and then the above-mentioned method is used to search for open position areas, exit positions, and open corner areas from the passable grids, candidate exits, and corner areas in the sector-shaped area or the circular area, which will not be repeated here.

[0042] The target location is determined based on the priority order of each drivable area. In this embodiment of the present application, the drivable area with the highest priority can be determined as the target location. For example, the order of priority from high to low is exit location - open area - open corner area. If an open area, exit location, and open corner area are found, the exit location can be determined as the target location. If an open area and an open corner area are found, the open area can be determined as the target location. Determining the target location by priority ensures that the smart vehicle is parked in a location that is easy to leave, reducing the impact on other vehicles and pedestrians.

[0043] b. Path feasibility verification: The minimum turning radius is greater than or equal to 2.2m, and obstacles with a height of less than 1.5m are avoided.

[0044] The feasibility of the intelligent vehicle's driving path is verified to ensure that the path has enough space for turns and avoids obstacles such as fire hydrants or low-hanging objects on the driving path.

[0045] c. Generate trajectory: B-spline curve smoothing, speed curve limited to 0 to 5 km / h.

[0046] In the embodiment of the present application, a smooth and continuous driving path can be obtained based on the B-spline curve, which helps to improve driving comfort and safety, and keep the vehicle speed within a moderate range to avoid instability and safety hazards caused by excessive driving.

[0047] Furthermore, when the vehicle control device controls the intelligent vehicle to perform hazard avoidance actions, the environmental detection sensor obtains the current environmental data in the surrounding environment area corresponding to the intelligent vehicle every 200 milliseconds to ensure that there are no sudden dangers on the driving path, such as pedestrians suddenly entering the driving path; when the path planning fails, the on-the-spot hazard avoidance strategy is triggered, that is, the hazard warning lights of the intelligent vehicle are activated to remind surrounding vehicles and pedestrians to pay attention to avoid or honk the horn briefly and continuously to remind the surrounding factors that may bring safety hazards. The above methods can ensure that the vehicle can take evasive measures in time when encountering an emergency to ensure driving safety.

[0048] Furthermore, the current hazard avoidance action can be determined through a data table (that is, the corresponding relationship is pre-stored in the data table) or through a deep learning model (that is, the corresponding relationship is learned by the deep learning model during the model training process).

[0049] In the path planning device provided in the embodiments of the present application, a three-level risk avoidance strategy system is constructed by determining the current risk level based on current risk source data and determining the current risk avoidance action based on the current risk level. When the risk level is Level 1, forced avoidance is triggered, breaking through the traditional manual confirmation process; when the risk levels are Level 2 and Level 3, a user confirmation mechanism is used. The three-level risk avoidance strategy system ensures automatic emergency avoidance when the safety level is high, and provides a user confirmation mechanism when the risk is controllable but requires user confirmation, thereby improving the safety and user experience of smart vehicles.

[0050] In specific implementation, the vehicle control device is also used to detect abnormal situations in the vehicle control process according to the abnormality detection methods corresponding to various abnormal situations, and determine the detected abnormal situations as abnormal situations to be processed; based on the correspondence between abnormal situations and processing strategies, the processing strategy corresponding to the abnormal situation to be processed is determined as the target processing strategy; and the abnormal situation to be processed is processed based on the target processing strategy.

[0051] Specifically, for example, when the on-board communication device times out (such as the TBOX heartbeat packet timeout is greater than 5 seconds), it is necessary to switch to local decision-making (that is, execute forced emergency reversing avoidance action when the danger level is Level 1); when there are continuous A-frame invalid environmental images in the environmental video stream collected by the environmental detection sensor (A is greater than or equal to 10), it is necessary to activate the redundant sensor; when the control device in the vehicle control system responds to the instruction delay for more than 500 milliseconds, it is necessary to trigger the vehicle control device to perform a safe parking.

[0052] The vehicle control device provided in the embodiment of the present application can also handle abnormal situations such as switching local decisions when the network is interrupted, activating redundant cameras when the sensor fails, and initiating safe parking when the control timeout exceeds 500ms, so as to improve the reliability of the vehicle control system.

[0053] Based on the above embodiments, an embodiment of the present invention provides an intelligent vehicle, including the above vehicle control system.

[0054] In the above-mentioned vehicle control system and intelligent vehicle provided by the embodiments of the present invention, the hazard source monitoring device detects the current hazard source data in the surrounding environment area and then sends a hazard source avoidance trigger instruction, which can trigger the path planning device to send a hazard source avoidance execution instruction after determining the current hazard source avoidance path of the intelligent vehicle, thereby further triggering the whole vehicle control device to control the intelligent vehicle to perform hazard source avoidance actions according to the current hazard source avoidance path, thereby realizing the mutual coordination of the hazard source monitoring device, the path planning device and the whole vehicle control device, completing the full link from sudden hazard source detection to autonomous hazard source avoidance, breaking through the bottleneck of the "perception-decision-execution" chain in the existing technology, and improving the safety of the intelligent vehicle.

[0055] Based on the same inventive concept, the embodiment of the present invention further provides a vehicle control method, see Figure 2 As shown, the interaction process of the vehicle control method provided by the embodiment of the present invention is as follows: Step 201: When the vehicle-mounted communication device receives the sentry mode activation instruction sent by the user client, it sends a first wake-up instruction to the vehicle control device; wherein, the sentry mode activation instruction is initiated by the user client when the smart vehicle is in a stationary state.

[0056] Step 202: When the vehicle control device receives the first wake-up instruction, it wakes up the in-vehicle network and sends a second wake-up instruction to the hazard source monitoring device.

[0057] Step 203: When the hazard source monitoring device receives the second wake-up instruction, it obtains the current environmental data in the surrounding environment area corresponding to the smart vehicle; uses the hazard source detection model to detect the current hazard source data in the surrounding environment area; and based on the current hazard source data, sends a hazard source avoidance trigger instruction to the vehicle-mounted communication device.

[0058] In an embodiment of the present application, in one embodiment, the hazard source monitoring device extracts M frames of environmental images of the same size and continuous from the environmental video stream contained in the current environmental data; uses a hazard source detection model to detect the hazard source data in each frame of the environmental image; based on the hazard source data in each frame of the environmental image, when the overlap rate of the hazard source area in the consecutive N frames of environmental images is greater than or equal to a first threshold and / or the confidence fluctuation degree is less than or equal to a second threshold, the current hazard source data is determined based on the hazard source data in each frame of the environmental image; wherein, M is a positive integer greater than 1, and N is less than or equal to M.

[0059] Step 204: When the vehicle-mounted communication device receives a hazard source avoidance trigger instruction, it forwards the hazard source avoidance trigger instruction to the user client, so that the user client performs a hazard source avoidance prompt based on the current hazard source data in the hazard source avoidance trigger instruction; when receiving a confirmation avoidance instruction returned by the user client, it forwards the hazard source avoidance trigger instruction to the vehicle control device.

[0060] Step 205: The vehicle control device receives the hazard source avoidance trigger instruction and forwards the hazard source avoidance trigger instruction to the path planning device. When the path planning device receives the hazard source avoidance trigger instruction, it determines the current hazard source avoidance path corresponding to the intelligent vehicle based on the current hazard source data in the hazard source avoidance trigger instruction; based on the current hazard source avoidance path, the hazard source avoidance execution instruction is sent to the vehicle control device.

[0061] In an embodiment of the present application, in one embodiment, the path planning device determines the current danger level based on the current danger source data, and determines the current danger source avoidance action based on the current danger level; based on the current danger source avoidance action, determines the current danger source avoidance path.

[0062] In another embodiment, the path planning device determines the danger level corresponding to the trigger condition reached by the current danger source data as the current danger level based on the correspondence between the danger level and the trigger condition; and determines the danger source avoidance action corresponding to the current danger level as the current danger source avoidance action based on the correspondence between the danger level and the danger source avoidance action.

[0063] Step 206: When the vehicle control device receives the hazard avoidance execution instruction, it controls the intelligent vehicle to execute the hazard avoidance action according to the current hazard avoidance path in the hazard avoidance execution instruction.

[0064] In an embodiment of the present application, in one embodiment, the vehicle control device detects an abnormal situation during the vehicle control process according to the abnormality detection method corresponding to various abnormal situations, and determines the detected abnormal situation as an abnormal situation to be processed; based on the correspondence between the abnormal situation and the processing strategy, the processing strategy corresponding to the abnormal situation to be processed is determined as the target processing strategy; and the abnormal situation to be processed is processed based on the target processing strategy.

[0065] Next, the training method of the hazard source detection model mentioned in the embodiment of the present invention is described in detail. Figure 3 As shown, the general process of the hazard source detection model training method provided by the embodiment of the present invention is as follows: Step 301: Obtain a training sample data set; wherein the training sample data set includes multiple training sample data; each training sample data includes an original video frame image and flame label data, smoke label data, flood label data and abnormal vehicle label data corresponding to the original video frame image.

[0066] In the embodiment of the present application, when obtaining the training sample data set, the following methods may be used but are not limited to: First, a plurality of original video frame images are obtained; wherein each original video frame image contains at least one hazard source selected from the group consisting of flames, smoke, floods, and abnormal vehicles.

[0067] Then, using Labelme software, each original video frame image is labeled with the pixel location information of different hazard sources to generate a standard mask image. Taking the standard mask image as a binary mask image as an example, each pixel in the original video frame image's target hazard source location area is labeled as 1, and each pixel in the location area of ​​the remaining hazard sources is labeled as 0. This generates a binary mask image of the target hazard source with the same image size as the original video frame image. The target hazard source label is added to the binary mask image. For example, if the target hazard source is a flame, the GT-H label is added to the binary mask image to facilitate hazard source identification. When an original video frame image contains multiple different hazard sources, the original video frame image, the standard mask images of the multiple hazard sources, and the hazard source labels constitute a training sample data set.

[0068] Finally, a plurality of training sample data are formed based on the plurality of original video frame images and the plurality of standard mask images of the original video frame images to obtain a training sample data set.

[0069] Step 302: Based on the training sample data set, perform an iterative training operation on the target detection model; wherein the iterative training operation includes: selecting target training sample data from the training sample data set; inputting the original video frame image in the target training sample data into the target detection model, so that the target detection model extracts features from the original video frame image to obtain fire hazard source data, flood hazard source data and abnormal hazard source data; based on the prediction error between the fire hazard source data and the flame label data and smoke label data in the target training sample data, based on the prediction error between the flood hazard source data and the flood label data in the target training sample data, and based on the prediction error between the abnormal hazard source data and the abnormal vehicle label data in the target training sample data, update the weights and thresholds of the target detection model.

[0070] Step 303: Determine whether the iterative training termination condition is met; if so, execute step 304; if not, return to step 302.

[0071] Step 304: Based on the weights and thresholds of the target detection model updated when the iterative training operation was last performed.

[0072] Step 305: Obtain a verification data set and a test data set; wherein the verification data set includes multiple verification sample data; each verification sample data includes an original video frame image and fire hazard source, flood hazard source and abnormal vehicle verification data corresponding to the original video frame image; the test data set includes multiple test sample data; each test sample data includes an original video frame image and fire hazard source, flood hazard source and abnormal vehicle test data corresponding to the original video frame image.

[0073] Step 306: Select target verification sample data from the verification data set, and input the original video frame image in the target verification sample data into the target detection model, so that the target detection model performs feature extraction on the original video frame image to obtain fire hazard source data, flood hazard source data and abnormal hazard source data, and obtain the verification loss curve based on the prediction error between the fire hazard source data and the fire hazard source verification data in the verification sample data, the prediction error between the flood hazard source data and the flood verification data in the verification sample data, and the prediction error between the abnormal hazard source data and the abnormal vehicle verification data in the verification sample data.

[0074] Step 307: Select target verification sample data from the test data set, and input the original video frame image in the target test sample data into the target detection model, so that the target detection model performs feature extraction on the original video frame image to obtain fire hazard source data, flood hazard source data and abnormal hazard source data, and obtain the test loss curve based on the prediction error between the fire hazard source data and the fire hazard source verification data in the test sample data, the prediction error between the flood hazard source data and the flood verification data in the test sample data, and the prediction error between the abnormal hazard source data and the abnormal vehicle verification data in the test sample data.

[0075] Step 308: Optimize the object detection model based on the validation loss curve and the test loss curve.

[0076] In the above-mentioned hazard source detection model training method provided in the embodiment of the present application, the YOLOv8 model is combined with multi-frame verification and confidence fluctuation degree to filter false detections, so that the false alarm rate of fire detection is more accurate, and the detection status is updated in real time and the hazard status is updated every 200ms to realize the active response of stationary vehicles to dynamic threats.

[0077] The functions and structures of the hazard source monitoring device and the path planning device mentioned in the embodiments of the present invention are briefly described below.

[0078] See Figure 4 As shown, the hazard source monitoring device provided by the embodiment of the present invention includes at least: The data acquisition unit 401 is used to acquire current environmental data in the surrounding area corresponding to the smart vehicle; The hazard source detection unit 402 is configured to extract, based on the current environmental data, M consecutive frames of environmental images of the same size from the environmental video stream contained in the current environmental data; detect the hazard source data in each frame of the environmental image using a hazard source detection model; determine, based on the hazard source data in each frame of the environmental image, when the overlap rate of hazard source regions in N consecutive frames of environmental images is greater than or equal to a first threshold and / or the degree of confidence fluctuation is less than or equal to a second threshold, determine the current hazard source data based on the hazard source data in each frame of the environmental image; wherein M is a positive integer greater than 1 and N is less than or equal to M; and send a hazard source avoidance trigger instruction based on the current hazard source data; wherein the current hazard source data includes at least any one of fire, flood, smoke, and abnormal vehicle. The avoidance trigger unit 403 is used to send a hazard source avoidance trigger instruction based on the current hazard source data, so as to trigger the path planning device to determine the current hazard source avoidance strategy corresponding to the intelligent vehicle based on the current hazard source data in the hazard source avoidance trigger instruction, and then trigger the vehicle control device to control the intelligent vehicle to perform the hazard source avoidance action based on the current hazard source avoidance strategy in the hazard source avoidance execution instruction.

[0079] See Figure 5 As shown, the path planning device provided by the embodiment of the present invention at least includes: The instruction receiving unit 501 is used to receive a hazard source avoidance trigger instruction sent by the hazard source monitoring device; wherein, the hazard source avoidance trigger instruction is that the hazard source monitoring device extracts M frames of continuous environmental images of the same size from the environmental video stream contained in the current environmental data based on the current environmental data in the surrounding environment area corresponding to the intelligent vehicle; uses a hazard source detection model to detect the hazard source data in each frame of the environmental image; based on the hazard source data in each frame of the environmental image, determines that when the overlap rate of the hazard source area in the consecutive N frames of environmental images is greater than or equal to a first threshold and / or the confidence fluctuation degree is less than or equal to a second threshold, determines the current hazard source data based on the hazard source data in each frame of the environmental image; wherein M is a positive integer greater than 1, and N is less than or equal to M; based on the current hazard source data, sends a hazard source avoidance trigger instruction; wherein, the current hazard source data includes at least any one of fire, flood, smoke and abnormal vehicle; A strategy determination unit 502 is configured to determine a current hazard avoidance strategy corresponding to the intelligent vehicle based on the current hazard source data in the hazard source avoidance trigger instruction; The execution trigger unit 503 is used to send a hazard avoidance execution instruction based on the current hazard avoidance strategy to trigger the vehicle control device to control the intelligent vehicle to perform the hazard avoidance action based on the current hazard avoidance strategy in the hazard avoidance instruction.

[0080] Based on the above embodiments, an embodiment of the present invention provides an electronic device, which may be a hazard source monitoring device, a path planning device, or a vehicle control device. Figure 6 As shown, the electronic device 600 provided by the embodiment of the present invention includes at least a controller 601, a memory 602 and a computer program stored in the memory 602 and executable on the controller 601. When the controller 601 executes the computer program, the vehicle control method performed by the above-mentioned hazard source monitoring device, path planning device or vehicle control device provided by the embodiment of the present invention is implemented.

[0081] The electronic device 600 provided in this embodiment of the present invention may further include a bus 603 connecting different components (including the controller 601 and the memory 602). The bus 603 represents one or more of several types of bus structures, including a memory bus, a peripheral bus, a local bus, and the like.

[0082] Memory 602 may include a readable storage medium in the form of volatile memory, such as random access memory (RAM) 6021 and / or cache memory 6022, and may further include read-only memory (ROM) 6023. Memory 602 may also include a program tool 6025 having a set (at least one) of program modules 6024. Program modules 6024 include, but are not limited to, an operating subsystem, one or more application programs, other program modules, and program data. Each of these examples, or some combination thereof, may include an implementation of a network environment.

[0083] Controller 601 can be a single processing element or a collective term for multiple processing elements. For example, controller 601 can be a central processing unit (CPU) or one or more integrated circuits configured to implement the vehicle control method performed by the hazard source monitoring device, path planning device, or vehicle control device provided in the embodiments of the present invention. Specifically, controller 601 can be a general-purpose controller, including but not limited to a CPU, an application-specific integrated circuit (ASIC), an off-the-shelf field programmable gate array (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0084] The electronic device 600 can communicate with one or more external devices 604 (e.g., keyboard, remote control, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device 600 (e.g., mobile phone, computer, etc.), and / or communicate with a device that enables the electronic device 600 to communicate with one or more other electronic devices 600 (e.g., router, modem, etc.). Such communication can be performed through an input / output (I / O) interface 605. In addition, the electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) through a network adapter 606. Figure 6 As shown, the network adapter 606 communicates with other modules of the electronic device 600 via the bus 603. Figure 6 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to microcode, device drivers, redundant controllers, external disk drive arrays, disk array (Redundant Arrays of Independent Disks, RAID) subsystems, tape drives, and data backup storage subsystems.

[0085] It should be noted that Figure 6 The electronic device 600 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0086] The following describes the computer-readable storage medium provided in an embodiment of the present invention. The computer-readable storage medium provided in an embodiment of the present invention stores computer instructions, which, when executed by a controller, implement the vehicle control method performed by the hazard source monitoring device, path planning device, or vehicle control device provided in an embodiment of the present invention. In a specific implementation, the computer instructions can be built into or installed in the controller. In this way, the controller can implement the vehicle control method performed by the hazard source monitoring device, path planning device, or vehicle control device provided in an embodiment of the present invention by executing the built-in or installed computer instructions.

[0087] In addition, the vehicle control method performed by the above-mentioned hazard source monitoring device, path planning device or whole vehicle control device provided in the embodiment of the present invention can also be implemented as a computer program product, which includes a program code. When the program code runs on a controller, it implements the vehicle control method performed by the above-mentioned hazard source monitoring device, path planning device or whole vehicle control device provided in the embodiment of the present invention.

[0088] The computer program product provided by the embodiments of the present invention may adopt one or more computer-readable storage media, and the computer-readable storage medium may be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any suitable combination of the above. Specifically, more specific examples (a non-exhaustive list) of computer-readable storage media include an electrical connection with one or more wires, a portable disk, a hard disk, RAM, ROM, Erasable Programmable Read Only Memory (EPROM), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

[0089] The computer program product provided in embodiments of the present invention may utilize a CD-ROM and include program code. It may also be run on electronic devices such as hazard source monitoring devices, path planning devices, or vehicle control devices. However, the computer program product provided in embodiments of the present invention is not limited thereto. In embodiments of the present invention, a computer-readable storage medium may be any tangible medium containing or storing program code, which may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0090] It should be noted that although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more units described above may be embodied in a single unit. Conversely, the features and functions of a single unit described above may be further divided and embodied by multiple units.

[0091] Furthermore, although the operations of the method of the present invention are described in a particular order in the accompanying drawings, this does not require or imply that these operations must be performed in this particular order, or that all illustrated operations must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0092] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic creative concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0093] Obviously, those skilled in the art may make various changes and modifications to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, if such changes and modifications of the embodiments of the present invention fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A vehicle control system, characterized in that: Including hazard source monitoring device, path planning device and vehicle control device; The hazard source monitoring device is used to obtain current environmental data within the surrounding environment area corresponding to the intelligent vehicle; extract M frames of environmental images of the same size and continuous from the environmental video stream contained in the current environmental data; use a hazard source detection model to detect the hazard source data in each frame of the environmental image; based on the hazard source data in each frame of the environmental image, determine that when the overlap rate of the hazard source area in N consecutive frames of environmental images is greater than or equal to a first threshold and / or the confidence fluctuation degree is less than or equal to a second threshold, determine the current hazard source data based on the hazard source data in each frame of the environmental image; wherein M is a positive integer greater than 1, and N is less than or equal to M; based on the current hazard source data, send a hazard source avoidance trigger instruction; The path planning device is configured to, upon receiving the hazard source avoidance trigger instruction, determine a current hazard source avoidance path corresponding to the intelligent vehicle based on the current hazard source data in the hazard source avoidance trigger instruction; and send a hazard source avoidance execution instruction based on the current hazard source avoidance path; The whole vehicle control device is used to control the intelligent vehicle to perform a hazard avoidance action according to the current hazard avoidance path in the hazard avoidance execution instruction when receiving the hazard avoidance execution instruction.

2. The vehicle control system according to claim 1, characterized in that: Also included is an onboard communication device; The vehicle-mounted communication device is configured to, upon receiving the hazard source avoidance trigger instruction, forward the hazard source avoidance trigger instruction to the user client, so that the user client performs a hazard source avoidance prompt based on the current hazard source data in the hazard source avoidance trigger instruction; and upon receiving a confirmation avoidance instruction returned by the user client, forward the hazard source avoidance trigger instruction to the vehicle control device; The whole vehicle control device is also used to obtain the current vehicle status data of the intelligent vehicle when receiving the hazard avoidance trigger instruction, and when determining that the intelligent vehicle meets the intelligent driving conditions based on the current vehicle status data, send the hazard avoidance trigger instruction to the path planning device.

3. The vehicle control system according to claim 2, characterized in that: The vehicle-mounted communication device is further configured to send a first wake-up instruction to the vehicle control device upon receiving a sentry mode activation instruction sent by the user client; wherein the sentry mode activation instruction is initiated by the user client when the smart vehicle is in a stationary state; The vehicle control device is further configured to wake up the in-vehicle network upon receiving the first wake-up instruction and send a second wake-up instruction to the hazard source monitoring device; The hazard source monitoring device is further configured to obtain current environmental data within the surrounding environment area corresponding to the smart vehicle when receiving the second wake-up instruction.

4. The vehicle control system according to claim 1, characterized in that: The path planning device is used to determine the current hazard level based on the current hazard source data, and determine the current hazard source avoidance action based on the current hazard level; based on the current hazard source avoidance action, determine the current hazard source avoidance path.

5. The vehicle control system according to claim 4, characterized in that: The path planning device is used to determine the danger level corresponding to the trigger condition reached by the current danger source data as the current danger level based on the correspondence between the danger level and the trigger condition; Based on the correspondence between the danger level and the danger source avoidance action, the danger source avoidance action corresponding to the current danger level is determined as the current danger source avoidance action.

6. The vehicle control system according to claim 1, characterized in that: The vehicle control device is also used to detect an abnormal situation during the vehicle control process according to the abnormality detection method corresponding to various abnormal situations, and determine the detected abnormal situation as an abnormal situation to be processed; based on the correspondence between the abnormal situation and the processing strategy, determine the processing strategy corresponding to the abnormal situation to be processed as the target processing strategy; and process the abnormal situation to be processed based on the target processing strategy.

7. A vehicle control method, characterized in that: The hazard source monitoring device used in a vehicle control system according to any one of claims 1 to 6, the method comprising: Obtain current environmental data within the surrounding area corresponding to the smart vehicle; Extracting M consecutive frames of environmental images of the same size from the environmental video stream contained in the current environmental data; detecting the hazard source data in each frame of the environmental image using a hazard source detection model; determining, based on the hazard source data in each frame of the environmental image, when an overlap rate of hazard source regions in N consecutive frames of environmental images is greater than or equal to a first threshold and / or a confidence fluctuation degree is less than or equal to a second threshold, determining the current hazard source data based on the hazard source data in each frame of the environmental image; wherein M is a positive integer greater than 1, and N is less than or equal to M; Based on the current hazard source data, a hazard source avoidance trigger instruction is sent to trigger the path planning device to determine the current hazard source avoidance strategy corresponding to the smart vehicle based on the current hazard source data in the hazard source avoidance trigger instruction, and then trigger the vehicle control device to control the smart vehicle to perform the hazard source avoidance action based on the current hazard source avoidance strategy in the hazard source avoidance execution instruction.

8. A vehicle control method, characterized in that: A path planning device applied to a vehicle control system according to any one of claims 1 to 6, wherein the method comprises: Receive a hazard source avoidance trigger instruction sent by a hazard source monitoring device; wherein, the hazard source avoidance trigger instruction is that the hazard source monitoring device extracts M frames of continuous environmental images of the same size from the environmental video stream contained in the current environmental data based on the current environmental data in the surrounding environment area corresponding to the intelligent vehicle; uses a hazard source detection model to detect the hazard source data in each frame of the environmental image; based on the hazard source data in each frame of the environmental image, when it is determined that the overlap rate of the hazard source area in N consecutive frames of environmental images is greater than or equal to a first threshold and / or the confidence fluctuation degree is less than or equal to a second threshold, the current hazard source data is determined based on the hazard source data in each frame of the environmental image; wherein, M is a positive integer greater than 1, and N is less than or equal to M; based on the current hazard source data, send a hazard source avoidance trigger instruction; Determining a current hazard avoidance strategy corresponding to the smart vehicle based on the current hazard source data in the hazard source avoidance trigger instruction; Based on the current hazard avoidance strategy, a hazard avoidance execution instruction is sent to trigger the vehicle control device to control the smart vehicle to perform a hazard avoidance action based on the current hazard avoidance strategy in the hazard avoidance instruction.

9. An intelligent vehicle, characterized in that: The vehicle control system comprises the vehicle control system according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed on a processor, the vehicle control method according to claim 8 is implemented.

11. A computer program product comprising instructions, characterized in that The computer program product includes program code, and when the program code is run on a processor, the vehicle control method according to claim 8 is implemented.

12. A chip, characterized in that: It includes at least one interface circuit and at least one processor; the interface circuit is used to receive a signal from the memory of the intelligent vehicle and send the signal to the processor, the signal including a computer instruction stored in the memory; when the processor executes the computer instruction, the vehicle control method as claimed in claim 8 is implemented.

Images