Obstacle identification early warning method, system and device and storage medium

Abstract

The embodiment of the invention provides an obstacle recognition early warning method, system and device and a storage medium, and the method comprises the steps: recognizing an obstacle in a preset range of a vehicle based on environment perception data; determining a relative relationship between the obstacle and the vehicle based on the detection information of the obstacle; identifying a dangerous attribute obstacle based on the relative relationship; and determining the danger level of the danger attribute obstacle based on the obstacle information of the danger attribute obstacle and the vehicle information of the vehicle.

Description

Technical Field

[0001] This specification relates to the field of obstacle warning technology, and in particular to an obstacle identification and warning method, system, device, and storage medium. Background Art

[0002] In recent years, autonomous driving technology has rapidly developed, with an increasing number of new energy vehicles equipped with advanced assisted autonomous driving systems entering the market. Ports, with their inherent semi-enclosed nature, fixed routes, and highly structured tasks, are ideal locations for the implementation of autonomous driving technology. However, ports are characterized by heavy logistics and transportation, numerous large-scale machinery and equipment, irregular operation of equipment and personnel, and the tall size of transport vehicles. This creates numerous blind spots, preventing the onboard autonomous driving sensors from fully perceiving their surroundings. This increases the likelihood of "ghosting" and poses a safety threat to autonomous vehicle operations.

[0003] Therefore, an obstacle identification and warning method, system, device and storage medium are provided to help improve the ability of autonomous driving vehicles to identify dangerous obstacles and ensure the safe operation of autonomous driving vehicles in ports. Summary of the Invention

[0004] One of the embodiments of this specification provides an obstacle identification and warning method, which includes: identifying obstacles within a preset range of a vehicle based on environmental perception data; determining the relative relationship between the obstacle and the vehicle based on detection information of the obstacle; identifying dangerous obstacles based on the relative relationship; and determining the danger level of the dangerous obstacles based on the detection information of the dangerous obstacles and vehicle information of the vehicle.

[0005] One of the embodiments of this specification provides an obstacle identification and warning system, including a first identification module, configured to: identify obstacles within a preset range of a vehicle based on environmental perception data; a relative relationship determination module, configured to: determine the relative relationship between the obstacle and the vehicle based on detection information of the obstacle; a second identification module, configured to: identify dangerous obstacles based on the relative relationship; and a danger level determination module, configured to: determine the danger level of the dangerous obstacle based on the detection information of the dangerous obstacle and vehicle information of the vehicle.

[0006] One of the embodiments of this specification provides an obstacle identification and warning device, which includes at least one processor and at least one memory; the at least one memory is used to store computer instructions; and the at least one processor is used to execute at least part of the computer instructions to implement the above-mentioned obstacle identification and warning method.

[0007] One embodiment of this specification provides a computer-readable storage medium, which stores computer instructions. When a computer reads the computer instructions, the computer executes the above-mentioned obstacle identification and warning method. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] This specification will be further described in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, like numbers represent like structures, wherein:

[0009] Figure 1 is a schematic diagram of an application scenario of an obstacle recognition and warning system according to some embodiments of this specification;

[0010] Figure 2 is an exemplary module diagram of an obstacle identification and warning system according to some embodiments of this specification;

[0011] Figure 3 is an exemplary flow chart of an obstacle identification and warning method according to some embodiments of this specification;

[0012] Figure 4 is an exemplary schematic diagram of determining the danger level of a dangerous obstacle according to some embodiments of this specification;

[0013] Figure 5 This is an exemplary schematic diagram of determining whether there is an obstruction caused by a dangerous obstacle according to some embodiments of this specification. DETAILED DESCRIPTION

[0014] To more clearly illustrate the technical solutions of the embodiments of this specification, the following briefly describes the drawings required for describing the embodiments. Obviously, the drawings described below are merely examples or embodiments of this specification. Those skilled in the art can apply this specification to other similar scenarios based on these drawings without inventive effort. Unless otherwise apparent from the context or otherwise noted, the same reference numerals in the figures represent the same structure or operation.

[0015] It should be understood that the terms "system," "device," "unit," and / or "module" used herein are a method for distinguishing different components, elements, parts, portions, or assemblies at different levels. However, if other terms can achieve the same purpose, the terms may be replaced by other expressions.

[0016] As used in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not refer to the singular but also include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0017] Flowcharts are used throughout this specification to illustrate the operations performed by systems according to embodiments of this specification. It should be understood that preceding or following operations do not necessarily need to be performed in exact order. Instead, the steps may be processed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0018] Predicting the intrusion and approach of dangerous obstacles is an important measure to improve the safety of autonomous vehicles. For example, when a vehicle is carrying heavy objects, it has great inertia during driving. Sudden braking may cause the vehicle to generate a large inertial thrust, causing serious accidents such as vehicle rollover or container slippage, posing a threat to the safety of people and goods.

[0019] By building a vehicle-to-vehicle communication or vehicle-to-infrastructure communication system, more comprehensive information about the environment and surrounding vehicles can be obtained based on real-time communication technology, and potential obstacles can be warned in advance. This can improve the vehicle's reaction time and avoidance ability to obstacles. However, the construction of this system relies on a powerful communication network and other infrastructure, which is costly and requires a large investment. In actual use, network failures or delays will affect system performance. More importantly, "unfamiliar" obstacles that are not included in the system cannot be effectively monitored for safety.

[0020] By combining multiple sensors into a perception system to supplement visual blind spots, such as installing ultrasonic, lidar, and vision-based multi-sensor fusion systems, good environmental perception can be provided under different weather and lighting conditions, enhancing the ability to perceive dangerous obstacles approaching vehicles. However, for ports with multiple obstructions, simply having no blind spots near vehicles cannot avoid the danger of sudden intrusion from obstacles behind large equipment or other vehicles.

[0021] In some embodiments of this specification, obstacles within a preset range of the vehicle are identified based on environmental perception data; the relative relationship between the obstacle and the vehicle is determined based on obstacle detection information; hazardous obstacles are identified based on this relative relationship; and the hazard level of these hazardous obstacles is determined based on the detected information and vehicle information. By grading the hazard level of obstacles, the vehicle can proactively avoid them based on their hazard level, without increasing hardware costs. Accurately assessing the behavior and status of all obstacles improves the vehicle's ability to proactively perceive and respond to risks.

[0022] Figure 1 This is a schematic diagram of an application scenario of an obstacle recognition and warning system according to some embodiments of this specification. Figure 1 As shown, the application scenario 100 of the obstacle recognition and warning system may include a vehicle 110 , a processor 120 , a network 130 , and a storage device 140 .

[0023] Vehicle 110 refers to a related vehicle that requires obstacle recognition and warning. For example, vehicle 110 may include an unmanned truck, an autonomous flatbed truck, etc.

[0024] In some embodiments, the vehicle 110 may include a laser radar perception system. The laser radar perception system refers to a laser radar system used to identify obstacles. For example, the laser radar perception system may include a mechanical laser radar and / or a semi-solid laser radar. For more information about the laser radar perception system, see Figure 3 and its related descriptions.

[0025] The processor 120 can process data and / or information obtained from other components of the application scenario 100 of the obstacle identification and warning system. The processor can execute program instructions based on this data, information and / or processing results to perform one or more functions described in this application. In some embodiments, the processor 120 may include one or more sub-processing devices (e.g., a single-core processing device or a multi-core multi-core processing device). By way of example only, the processor 120 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), an application-specific instruction processor (ASIP), etc., or any combination thereof.

[0026] In some embodiments, the processor can identify obstacles within a preset range of the vehicle based on environmental perception data; determine the relative relationship between the obstacle and the vehicle based on the detection information of the obstacle; identify dangerous obstacles based on the relative relationship; and determine the danger level of the dangerous obstacles based on the detection information of the dangerous obstacles and the vehicle information of the vehicle.

[0027] In some embodiments, the processor may determine whether the angle between the heading of the obstacle and the direction of movement of the vehicle satisfies a preset angle condition; in response to the angle satisfying the preset angle condition, the obstacle is determined to be a dangerous obstacle.

[0028] In some embodiments, the processor can determine whether there is an obstruction to the dangerous obstacle based on the detection information and vehicle information; determine whether the dangerous obstacle is moving toward the vehicle based on the detection information and vehicle information; and determine the danger level of the dangerous obstacle based on the judgment result.

[0029] In some embodiments, the processor may determine multiple reference lines between the vehicle and the dangerous obstacle based on multiple vehicle positions and obstacle positions; and determine whether the dangerous obstacle is blocked based on the multiple reference lines.

[0030] In some embodiments, the processor may determine a distance sequence between the dangerous obstacle and the vehicle at multiple moments based on multiple obstacle positions and vehicle positions; and determine whether the dangerous obstacle is moving toward the vehicle based on the distance sequence.

[0031] In some embodiments, in response to the presence of an obstruction by a dangerous obstacle and not moving toward the vehicle, the processor may determine that the danger level of the dangerous obstacle is the first level; in response to the presence of an obstruction by the dangerous obstacle and moving toward the vehicle, the processor may determine that the danger level of the dangerous obstacle is the second level; in response to the presence of an obstruction by the dangerous obstacle and moving toward the vehicle, the processor may determine that the danger level of the dangerous obstacle is the third level.

[0032] The network 130 includes any suitable network capable of facilitating information and / or data exchange for the application scenario 100 of the obstacle identification and warning system. In some embodiments, one or more components of the application scenario 100 of the obstacle identification and warning system (e.g., the vehicle 110, the processor 120, the storage device 140, etc.) can exchange information and / or data with one or more components of the application scenario 100 of the obstacle identification and warning system via the network 130.

[0033] The storage device 140 is used to store data, instructions, and / or any other information. The storage device 140 may include one or more storage components, each of which may be a standalone device or part of another device. In some embodiments, the storage device 140 may include random access memory (RAM), read-only memory (ROM), removable memory, or any combination thereof. In some embodiments, the storage device 140 may be connected to the network 130 to enable communication with one or more components in the application scenario 100 of the obstacle identification and warning system.

[0034] It should be noted that the application scenarios are provided for illustrative purposes only and are not intended to limit the scope of this specification. A person skilled in the art can make various modifications or variations based on the description of this specification. For example, the application scenarios may also include a database. For another example, the application scenarios may be implemented on other devices to achieve similar or different functions. However, such changes and modifications do not deviate from the scope of this specification.

[0035] Figure 2 is an exemplary module diagram of an obstacle recognition and warning system according to some embodiments of this specification. Figure 2As shown, the obstacle recognition and warning system 200 may include a first recognition module 210 , a relative relationship determination module 220 , a second recognition module 230 and a danger level determination module 240 .

[0036] In some embodiments, the first recognition module 210 may be configured to recognize obstacles within a preset range of the vehicle based on environmental perception data.

[0037] In some embodiments, the relative relationship determination module 220 may be configured to determine the relative relationship between the obstacle and the vehicle based on the detection information of the obstacle.

[0038] In some embodiments, the second recognition module 230 may be configured to recognize dangerous obstacles based on relative relationships.

[0039] In some embodiments, the second recognition module 230 can also be configured to determine whether the angle between the heading of the obstacle and the direction of movement of the vehicle meets a preset angle condition; in response to the angle meeting the preset angle condition, the obstacle is determined to be a dangerous obstacle.

[0040] In some embodiments, the danger level determination module 240 may be configured to determine the danger level of the dangerous obstacle based on detection information of the dangerous obstacle and vehicle information of the vehicle.

[0041] In some embodiments, the danger level determination module 240 can also be configured to determine whether there is an obstruction to the dangerous attribute obstacle based on detection information and vehicle information; determine whether the dangerous attribute obstacle is moving toward the vehicle based on detection information and vehicle information; and determine the danger level of the dangerous attribute obstacle based on the judgment result.

[0042] In some embodiments, the danger level determination module 240 can also be configured to determine multiple reference lines between the vehicle and the dangerous obstacle based on multiple vehicle positions and obstacle positions; and based on the multiple reference lines, determine whether the dangerous obstacle is blocked.

[0043] In some embodiments, the danger level determination module 240 can also be configured to determine the distance sequence between the dangerous obstacle and the vehicle at multiple times based on multiple obstacle positions and vehicle positions; based on the distance sequence, determine whether the dangerous obstacle is moving toward the vehicle.

[0044] In some embodiments, the danger level determination module 240 can also be configured to: in response to the existence of occlusion and no movement of the dangerous attribute obstacle toward the vehicle, determine the danger level of the dangerous attribute obstacle as the first level; in response to the existence of occlusion and movement of the dangerous attribute obstacle toward the vehicle, determine the danger level of the dangerous attribute obstacle as the second level; in response to the existence of occlusion and movement of the dangerous attribute obstacle toward the vehicle, determine the danger level of the dangerous attribute obstacle as the third level.

[0045] In some embodiments, all or part of the modules of the obstacle identification and warning system 200 can be integrated into a processor. For more information about the first identification module 210, the relative relationship determination module 220, the second identification module 230, and the danger level determination module 240, please refer to the relevant description below.

[0046] It should be noted that the above description of the obstacle recognition and warning system and its modules is for convenience only and does not limit this specification to the embodiments described. It is understood that those skilled in the art, after understanding the principles of the system, may arbitrarily combine the modules or form subsystems connected to other modules without departing from these principles.

[0047] Figure 3 is an exemplary flow chart of the obstacle identification and warning method according to some embodiments of this specification. In some embodiments, process 300 can be executed by a processor of the obstacle identification and warning system. Figure 3 As shown, process 300 includes the following steps:

[0048] Step 310: Identify obstacles within a preset range of the vehicle based on the environmental perception data.

[0049] Environmental perception data refers to data related to the environment that the vehicle is in. For example, environmental perception data may include image data of the vehicle's environment, radar point cloud data, etc.

[0050] In some embodiments, the processor may obtain environmental perception data in a variety of ways. For example, the processor may obtain environmental perception data based on devices such as cameras and lidar.

[0051] In some embodiments, the environmental perception data may include point cloud data.

[0052] Point cloud data refers to the collection of point cloud points in the environment where the vehicle is located.

[0053] In some embodiments, the processor can acquire point cloud data based on a LiDAR perception system. The LiDAR system can include a vehicle perception hardware system and a software system. The hardware system can be used to acquire point cloud data and can include a mechanical LiDAR and / or a semi-solid-state LiDAR. The software system can be used to identify obstacles and can include a deep learning perception system and / or a clustering perception system. For more information, see Figure 3 Related descriptions are given below.

[0054] In some embodiments of this specification, point cloud data is obtained through a lidar system, which can rely on the vehicle's existing perception hardware system. There is no need to install additional sensors to reduce the vehicle's visual blind spots. While reducing costs, obstacles can be accurately identified, which helps in the subsequent identification of dangerous obstacles.

[0055] The preset range refers to a pre-set range for identifying obstacles. The preset range can be determined in a variety of ways. For example, the preset range can be manually set, or the processor can determine the acquisition range of the lidar perception system as the preset range.

[0056] In some embodiments, the processor can identify obstacles based on environmental perception data using various methods. In some embodiments, the processor can identify obstacles based on environmental perception data using a perception software system within the lidar system. The software system can include a deep learning perception system and a clustering perception system. A deep learning perception system refers to a software system centered around 3D object detection technology, while a clustering perception system refers to a software system centered around ground removal and point cloud clustering. For example, the processor can identify obstacles based on environmental perception data using a deep learning perception system centered around 3D object detection technology and / or a clustering perception system centered around ground removal and point cloud clustering. Ground removal refers to filtering out ground point cloud points from environmental perception data. The clustering perception system can perform ground removal using various feasible methods, such as ground plane fitting. Point cloud clustering refers to clustering point cloud points of the same type within the environmental perception data. The clustering perception system can perform point cloud clustering using various methods, such as Euclidean distance clustering and region growing segmentation clustering. For another example, the processor can identify obstacles based on image data within the environmental perception data using various feasible image recognition technologies.

[0057] Step 320: Determine the relative relationship between the obstacle and the vehicle based on the obstacle detection information.

[0058] Detection information refers to information related to detected obstacles. For example, detection information can include obstacle type, location, size, and direction.

[0059] Obstacle types can include vehicles, people, and large equipment. Obstacle location refers to the location of the obstacle in the environment. Obstacle size refers to information related to the obstacle's dimensions, such as its height and length. Obstacle heading refers to information related to the obstacle's direction, such as its orientation and direction of movement.

[0060] The relative relationship between the obstacle and the vehicle refers to the relative position relationship between the obstacle and the vehicle. For example, the relative relationship may include the direction of the obstacle relative to the vehicle.

[0061] In some embodiments, the processor may determine the relative relationship between the obstacle and the vehicle based on the obstacle detection information using various methods. For example, the processor may obtain the current vehicle position using the vehicle's positioning device and determine the relative relationship between the obstacle and the vehicle based on the vehicle position and the obstacle position in the obstacle detection information. The vehicle position may include the three-dimensional coordinates of the vehicle's center point, and the obstacle position may include the three-dimensional coordinates of the obstacle's center point.

[0062] In some embodiments, the detection information of the obstacle includes at least the heading of the obstacle, and the relative relationship includes at least the angle between the heading of the obstacle and the moving direction of the vehicle.

[0063] In some embodiments, the processor may determine the obstacle detection information in a variety of ways.

[0064] In some embodiments, the processor can extract detection information of a first type of obstacles in the environment through deep learning technology based on environmental perception data, and determine detection information of a second type of obstacles by clustering and bounding boxing the environmental perception data not detected by the deep learning technology. The processor can then monitor the aforementioned two types of obstacles in time and space using obstacle tracking technology to obtain detection information of obstacles in the environment. Specifically, the processor can cluster the environmental perception data not detected by the deep learning technology using a distance clustering algorithm, a density clustering algorithm, etc. based on a clustering perception system, and then determine the detection information of the second type of obstacles using a bounding box fitting method. Exemplary obstacle tracking technologies may include multi-target data association algorithms, filter prediction, etc. For example, the processor can monitor obstacles within a preset time range and spatial range based on obstacle tracking technology to obtain detection information of obstacles in the environment.

[0065] Among them, the first type of obstacles refers to obstacles directly identified by the deep learning perception system, and the second type of obstacles refers to obstacles in the environment that are not identified by the deep learning perception system. The second type of obstacles can be obtained through clustering based on the clustering perception system. For relevant descriptions of the deep learning perception system and the clustering perception system, please refer to Figure 3 Related description in the previous article.

[0066] In some embodiments, the processor can interact with the vehicle based on a network to obtain the vehicle's direction of movement.

[0067] In some embodiments, the processor may obtain the angle between the heading of the obstacle and the direction of motion of the vehicle based on the heading of the obstacle and the direction of motion of the vehicle through a vector angle calculation formula or other feasible methods.

[0068] In some embodiments of this specification, by obtaining the obstacle heading and the angle between the obstacle heading and the vehicle's direction of motion, it is possible to determine whether the obstacle is likely to intersect with the vehicle's trajectory based on the heading and the angle, which helps in the subsequent identification of dangerous obstacles.

[0069] Step 330: Identify dangerous obstacles based on the relative relationship.

[0070] Dangerous obstacles are obstacles that may pose a safety threat to vehicle driving. For example, dangerous obstacles may include obstacles that may intersect with the vehicle's trajectory.

[0071] In some embodiments, the processor may identify dangerous obstacles based on relative relationships. For example, the processor may identify an obstacle directly in front of the vehicle as a dangerous obstacle. For another example, the processor may identify an obstacle that is laterally approaching the intersection relative to the vehicle as a dangerous obstacle.

[0072] In some embodiments, the processor may determine whether the angle between the heading of the obstacle and the direction of movement of the vehicle satisfies a preset angle condition; in response to the angle satisfying the preset angle condition, the obstacle is determined to be a dangerous obstacle.

[0073] The preset angle condition refers to a pre-set angle condition that must be met between the heading of the dangerous obstacle and the direction of movement of the vehicle. In some embodiments, the preset angle condition can be set by the system or manually. For example, the preset angle condition may be that the angle between the heading of the obstacle and the direction of movement of the vehicle is between 90°±30°.

[0074] In some embodiments of the present specification, by identifying obstacles as dangerous obstacles if the angle between the obstacle's heading and the vehicle's movement direction meets a preset angle condition, obstacles that may intersect with the vehicle's trajectory can be quickly and accurately identified as dangerous obstacles, thereby improving the comprehensiveness of obstacle identification.

[0075] Step 340: Determine the danger level of the dangerous obstacle based on the detection information of the dangerous obstacle and the vehicle information. Figure 3 Related description in the previous article.

[0076] Vehicle information refers to vehicle-related information data, such as vehicle location, vehicle movement direction, etc. For more information about vehicle location, please refer to Figure 3 In some embodiments, the processor can interact with the vehicle through a network to obtain vehicle information.

[0077] The danger level is an indicator used to characterize the degree of danger of a dangerous obstacle. The higher the danger level, the higher the degree of danger of the corresponding dangerous obstacle.

[0078] In some embodiments, the processor may determine the danger level of the dangerous obstacle based on the obstacle information of the dangerous obstacle and the vehicle information of the vehicle in various ways. For example, the processor may determine the danger level of the dangerous obstacle based on the obstacle information of the dangerous obstacle and the vehicle information using a first preset rule.

[0079] The first preset rule refers to a pre-set rule for determining the danger level. An exemplary first preset rule may be: the closer the distance between the obstacle position in the obstacle information of the dangerous obstacle and the vehicle position in the vehicle information is, the higher the danger level of the dangerous obstacle.

[0080] In some embodiments, the processor can also determine whether the dangerous obstacle is blocked and whether it is moving toward the vehicle based on the obstacle information and vehicle information, and determine the danger level of the dangerous obstacle based on the judgment result. For more information, please refer to Figure 4 and its related descriptions.

[0081] In some embodiments of this specification, obstacles are identified through a deep learning perception system and a clustering perception system based on environmental perception data, which can cover all possible moving obstacles around the vehicle and improve the comprehensiveness of obstacle identification. At the same time, by identifying dangerous obstacles and determining the corresponding danger level, it helps to provide early warning, provide response time for more intelligent decision-making and planning, and avoid vehicle collision risks in advance, thereby improving vehicle driving safety.

[0082] It should be noted that the above description of the relevant processes is for illustration and purpose only and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to the processes under the guidance of this specification. However, such modifications and changes are still within the scope of this specification.

[0083] Figure 4 This is an exemplary schematic diagram of determining the danger level of a dangerous obstacle according to some embodiments of this specification.

[0084] In some embodiments, the processor may determine whether the dangerous obstacle is blocked based on the obstacle information 411 and the vehicle information 412; determine whether the dangerous obstacle is moving toward the vehicle based on the obstacle information and the vehicle information; and determine the danger level 430 of the dangerous obstacle based on the judgment result 420. For more information about obstacle information, vehicle information, and danger level, please refer to Figure 3 and its related descriptions.

[0085] Hazardous obstacle occlusion means that there are other obstacles between the vehicle and the hazardous obstacle within a preset time period, making the hazardous obstacle invisible to the vehicle. Other obstacles refer to obstacles other than the hazardous obstacle that needs to be identified, such as other vehicles and large equipment in the vehicle's driving environment.

[0086] In some embodiments, the processor can determine whether there is an obstruction caused by a dangerous obstacle in a variety of ways based on the obstacle information and vehicle information.

[0087] For example, the processor can determine whether there is occlusion of the dangerous obstacle based on the relative relationship between the obstacle position of the dangerous obstacle, the positions of other obstacles and the vehicle position at the historical moment. Exemplarily, the processor can determine that the dangerous obstacle is blocked when the obstacle position of the dangerous obstacle, the positions of other obstacles and the vehicle position at the historical moment are in the same straight line. The processor can obtain the obstacle position of the dangerous obstacle, the positions of other obstacles and the vehicle position at the historical moment in the historical data stored in the storage device through the network, wherein the determination of the positions of other obstacles is the same as that of the dangerous obstacle, which can be seen in Figure 3 Related description in the previous article.

[0088] In some embodiments, the obstacle information may include the obstacle position, and the vehicle information may include multiple vehicle positions at multiple times. The processor may determine multiple reference lines between the vehicle and the dangerous obstacle based on the multiple vehicle positions and obstacle positions; and based on the multiple reference lines, determine whether the dangerous obstacle is obstructed.

[0089] In some embodiments, the multiple moments can be set by the system or manually, and the time intervals between adjacent moments can be the same or different. Exemplarily, the multiple moments include the current moment, the number of the multiple moments can be a preset value, and the time intervals between adjacent moments are the same. For example, the multiple moments can include seven moments including the current moment, and the time interval between adjacent moments is 0.5 seconds. The vehicle position at historical moments in the multiple moments can be obtained based on historical data.

[0090] The reference line can be a line (e.g., a straight line, a curve, etc.) connecting the vehicle position and the obstacle position of the dangerous obstacle. In some embodiments, the processor can determine multiple reference lines between the vehicle and the dangerous obstacle based on multiple vehicle positions and obstacle positions. For example, the processor can connect multiple vehicle positions at multiple times with the obstacle position of the dangerous obstacle at the current time to obtain multiple reference lines.

[0091] In some embodiments, the processor may determine whether a dangerous obstacle is obstructing the image based on the plurality of reference lines using a second preset rule. The second preset rule refers to a pre-set rule for determining whether a dangerous obstacle is obstructing the image. The second preset rule may be system- or manually preset.

[0092] An exemplary second preset rule may be: the number of reference lines that intersect with other obstacle positions exceeds a number threshold and / or the proportion of reference lines that intersect with other obstacle positions in all reference lines exceeds a proportion threshold, etc.

[0093] Figure 5 This is an exemplary schematic diagram of determining whether there is a occlusion caused by a dangerous obstacle according to some embodiments of this specification. Figure 5 As shown, the aforementioned multiple moments are seven moments including the current moment, and the time interval between adjacent moments is 0.5 seconds. s1, s2, s3, s4, s5, s6, and s7 are the vehicle positions at t, t-0.5s, t-1s, t-1.5s, t-2s, t-2.5s, and t-3s, respectively. t is the current moment, and obstacle 2 is the dangerous obstacle for which obstruction is to be determined. The second preset rule is: the number of reference lines that intersect with other obstacle positions exceeds two. Figure 5 Among the 7 reference lines, only the reference line corresponding to s1 does not have an intersection with obstacle 1, so the processor can determine that obstacle 2 is blocked.

[0094] In some embodiments of this specification, based on multiple reference lines, it is determined whether there is an obstruction caused by a dangerous obstacle. The relative position relationship between the dangerous obstacle and the vehicle when it is obstructed can be restored based on the vehicle position at a historical moment, which helps to improve the accuracy of determining whether there is an obstruction caused by a dangerous obstacle.

[0095] In some embodiments, the processor may determine whether a dangerous obstacle is moving toward the vehicle based on the obstacle information and vehicle information in various ways. For example, the processor may calculate the current distance between the vehicle's current position and the dangerous obstacle's position, obtain historical distances between the vehicle's position and the dangerous obstacle's position, compare the current distance with the historical distances, and determine that the dangerous obstacle is moving toward the vehicle if the current distance is less than the historical distance.

[0096] In some embodiments, the obstacle information includes multiple obstacle positions of the dangerous obstacle at multiple times, and the vehicle information includes the vehicle position. The processor can determine the distance sequence between the dangerous obstacle and the vehicle at multiple times based on the multiple obstacle positions and the vehicle position; based on the distance sequence, determine whether the dangerous obstacle is moving toward the vehicle.

[0097] In some embodiments, the multiple moments corresponding to the multiple obstacle locations include the current moment, and the time intervals between adjacent moments may be the same or different. In some embodiments, the number of moments and the time intervals between the multiple moments may be set by the system or manually. For example, the time interval between the multiple moments may be 0.1 seconds, and the number of moments may be an integer between 3 and 10.

[0098] The distance sequence refers to a sequence consisting of the distances between multiple obstacle positions and the vehicle position at multiple moments.

[0099] In some embodiments, the processor may determine the distance sequence in various ways based on the multiple obstacle positions and the vehicle position. For example, the processor may determine the distance sequence as a sequence consisting of the distances between the center coordinates of the multiple obstacle positions and the vehicle position.

[0100] In some embodiments, the processor may determine whether the dangerous obstacle is moving toward the vehicle based on the distance sequence. For example, the processor may determine that a dangerous obstacle with a gradually decreasing distance in the distance sequence is moving toward the vehicle.

[0101] In some embodiments, the processor can obtain the obstacle contour of the dangerous obstacle at each target moment in the target moment set; based on the obstacle contour information at each target moment, determine the lateral distance of the obstacle contour at each target moment from the movement direction of the vehicle at the current moment to obtain a distance sequence; determine whether the change in the lateral distance in the distance sequence meets a preset change condition; in response to meeting the preset change condition, determine that the dangerous obstacle is moving toward the vehicle.

[0102] A target moment set refers to a collection of multiple moments corresponding to multiple obstacle positions. In some embodiments, the target moment set includes multiple target moments. The target moments are moments corresponding to multiple obstacle positions of a dangerous obstacle. In some embodiments, the target moments in the target moment set meet preset moment conditions. Preset moment conditions refer to pre-set conditions that must be met for the target moments. Exemplary preset moment conditions may include: the target moment interval is 0.1 seconds, the number of target moments is an integer between 3 and 10, and the target moments include the current moment.

[0103] The obstacle outline refers to the outer contour of the dangerous attribute obstacle. In some embodiments, the processor can determine the set of point cloud points corresponding to the outermost edge of the dangerous attribute obstacle as the obstacle outline. For more information about point cloud points, please refer to Figure 3 and its related descriptions.

[0104] Obstacle profile information refers to information related to the outline of a hazardous obstacle. For example, it can include the coordinates of each point cloud point within the obstacle profile. For example, the obstacle profile at a specific target time can be represented as {p1(x1,y1),p2(x2,y2),...,pm(xm,ym)}, where (xm,ym) represents the coordinates of point pm within the obstacle profile.

[0105] The lateral distance refers to the lateral distance between the obstacle outline and the vehicle's current direction of motion.

[0106] In some embodiments, the processor can determine the lateral distance between each point cloud point in the obstacle contour at each target moment and the direction of motion based on the vehicle position and motion direction at the current moment and the obstacle contour information at each target moment; based on the target moment concentration, the lateral distance between each point cloud point in the obstacle contour at each target moment and the direction of motion is obtained to obtain a distance sequence.

[0107] In some embodiments, the processor can determine the lateral distance between each point cloud point in the obstacle contour at each target moment and the movement direction based on the current vehicle position, movement direction, and obstacle contour information at each target moment in a variety of ways.

[0108] For example, after the processor interacts with the vehicle through the network, it obtains the vehicle position at the current moment as l0(x0, y0), the movement direction as direction(cos(yaw), sin(yaw)), and the obstacle contour information at a certain target moment as {p1(x1, y1), p2(x2, y2), ..., pm(xm, ym)}. The processor can calculate the lateral distance between the point cloud point and the movement direction using the following formula: Among them, d l0pm is the lateral distance between a certain point cloud point pm and the direction of motion. The calculation method of the lateral distance between other point cloud points and the direction of motion is the same as that of the pm point.

[0109] In some embodiments, the processor can determine the minimum value (or average value, weighted average value, etc.) of the lateral distances between multiple point cloud points and the direction of motion at the same target moment as the lateral distance of the dangerous obstacle from the direction of motion of the vehicle at the target moment, and determine the sequence consisting of the lateral distances of the dangerous obstacles from the direction of motion of the vehicle at multiple target moments as a distance sequence.

[0110] In some embodiments of this specification, by focusing on target moments, the lateral distance between each point cloud point in the obstacle outline and the direction of motion at each target moment is obtained to obtain a distance sequence, which improves the accuracy of the distance sequence determination and helps to further determine whether the dangerous obstacle is moving toward the vehicle.

[0111] The change of horizontal distance refers to the change of horizontal distance in the distance series over time.

[0112] The preset change condition refers to the condition that must be met for the lateral distance in the distance sequence to change when a dangerous obstacle moves toward the vehicle. The preset change condition can be set by the system or manually. An example of a preset change condition is that the lateral distance in the distance sequence gradually decreases over time.

[0113] Some embodiments of this specification, by determining whether the change in lateral distance in a distance sequence meets a preset change condition, combine the lateral distances corresponding to multiple target moments, which helps to make a more accurate judgment on whether a dangerous obstacle is moving toward the vehicle.

[0114] In some embodiments of this specification, based on a distance sequence, it is determined whether a dangerous obstacle is moving toward a vehicle. By determining the distance between the dangerous obstacle and the vehicle position, the dangerous obstacle moving toward the vehicle is accurately determined, which helps to subsequently more accurately classify the dangerous level of the dangerous obstacle.

[0115] The judgment result refers to whether the dangerous obstacle is blocking the vehicle and whether it is moving towards the vehicle.

[0116] In some embodiments, the processor may determine the danger level of the dangerous obstacle based on the judgment results in various ways. For example, if the judgment results are all negative, the corresponding danger level is lower; if the judgment results are all positive, the corresponding danger level is higher.

[0117] In some embodiments, in response to the presence of an obstruction by a dangerous obstacle and not moving toward the vehicle, the processor may determine that the danger level of the dangerous obstacle is the first level; in response to the presence of an obstruction by the dangerous obstacle and moving toward the vehicle, the processor may determine that the danger level of the dangerous obstacle is the second level; in response to the presence of an obstruction by the dangerous obstacle and moving toward the vehicle, the processor may determine that the danger level of the dangerous obstacle is the third level.

[0118] The first level refers to the danger level when there is a dangerous obstacle blocking the vehicle and not moving towards it.

[0119] The second level refers to the danger level when the dangerous obstacle does not block the vehicle but moves towards it.

[0120] When the danger level of a dangerous obstacle is level one or level two, it has a certain degree of danger and poses a certain threat to the safety of the vehicle.

[0121] Level 3 refers to the danger level when a dangerous obstacle is blocking the vehicle and moving toward it. A level 3 dangerous obstacle is considered highly dangerous and poses a significant threat to vehicle safety.

[0122] In some embodiments, in response to the dangerous obstacle not being blocked and not moving toward the vehicle, the processor may determine that the danger level of the dangerous obstacle is a normal level.

[0123] Normal refers to the danger level when a dangerous obstacle does not obstruct the vehicle and is not moving toward it. When a dangerous obstacle is rated Normal, the danger level is low and poses a minimal threat to vehicle safety.

[0124] In some embodiments of this specification, by determining that the danger level of an obstacle is a common danger level, the danger level of the obstacle can be more clearly classified, which helps the vehicle to make an early response to safety threats based on the clearer danger level.

[0125] In some embodiments of this specification, dangerous obstacles are divided into the first, second, and third levels based on the judgment results, which helps to provide more accurate warnings for obstacle identification, enables the vehicle to make driving decisions with early response and improved safety performance, and provides a layer of safety protection for the safe driving of the vehicle.

[0126] In some embodiments, the processor may determine vehicle control decisions based on the risk level.

[0127] Vehicle control decisions refer to driving decisions used to avoid obstacles. For example, vehicle control decisions may include deceleration, parking, and other decisions.

[0128] In some embodiments, the processor may determine vehicle control decisions based on the danger level using a third preset rule. The third preset rule refers to a pre-set rule for determining vehicle control decisions. An exemplary third preset rule may be: when the danger level is normal, the vehicle performs normal operations such as collision detection and path planning; when the danger level is level 1 or 2, the vehicle decelerates; and when the danger level is level 3, the vehicle stops.

[0129] In some embodiments of this specification, the danger level of a dangerous obstacle is determined based on whether the dangerous obstacle is blocked and whether it is moving toward the vehicle. This helps to make appropriate vehicle control decisions for the vehicle, prevent the vehicle from colliding due to excessive speed or close distance to the dangerous obstacle, leave sufficient response time for subsequent more flexible decision-making and planning, and improve the safety of vehicle driving.

[0130] While the basic concepts have been described above, it will be apparent to those skilled in the art that the detailed disclosure is merely illustrative and does not limit this specification. Although not explicitly stated herein, various modifications, improvements, and revisions to this specification may be made by those skilled in the art. Such modifications, improvements, and revisions are suggested in this specification and remain within the spirit and scope of the exemplary embodiments of this specification.

[0131] This specification also uses specific terms to describe the embodiments of this specification. For example, "one embodiment," "an embodiment," and / or "some embodiments" refer to a feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "one embodiment," "an embodiment," or "an alternative embodiment" two or more times in different locations in this specification do not necessarily refer to the same embodiment. Furthermore, certain features, structures, or characteristics of one or more embodiments of this specification may be appropriately combined.

[0132] In addition, unless expressly stated in the claims, the order of the processing elements and sequences described in this specification, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and laminar flow hoods in this specification. Although the above disclosure discusses some of the invention embodiments currently considered useful through various examples, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the spirit and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0133] Similarly, it should be noted that, in order to simplify the presentation of this specification and thus facilitate understanding of one or more embodiments of the invention, the foregoing descriptions of the embodiments of this specification sometimes combine multiple features into a single embodiment, figure, or description thereof. However, this disclosure method does not imply that the subject matter of this specification requires more features than those recited in the claims. In fact, an embodiment may have fewer features than all of the features of a single disclosed embodiment.

[0134] In some embodiments, numbers are used to describe the quantity of components and attributes. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise stated, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the description and claims are approximate values, which may change according to the required characteristics of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of this specification are approximate values, in specific embodiments, the settings of such numerical values ​​are as accurate as possible within the feasible range.

[0135] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, and documents, cited in this specification is hereby incorporated by reference in its entirety. This includes application history documents that are inconsistent with or conflict with the content of this specification, as well as documents (currently or subsequently attached to this specification) that limit the broadest scope of the claims of this specification. It should be noted that if the descriptions, definitions, and / or terminology used in the accompanying materials are inconsistent or conflicting with the content of this specification, the descriptions, definitions, and / or terminology used in this specification will control.

[0136] Finally, it should be understood that the embodiments described in this specification are intended only to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, by way of example and not limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly described and illustrated in this specification.

Claims

1. An obstacle identification and warning method, characterized in that: include: Identify obstacles within the vehicle's preset range based on environmental perception data; determining a relative relationship between the obstacle and the vehicle based on the detection information of the obstacle; Based on the relative relationship, identifying dangerous obstacles; Based on the detection information of the dangerous obstacle and the vehicle information of the vehicle, a danger level of the dangerous obstacle is determined.

2. The method according to claim 1, characterized in that The detection information of the obstacle includes at least the heading of the obstacle; The relative relationship includes at least an angle between the heading of the obstacle and the moving direction of the vehicle.

3. The method according to claim 2, characterized in that The step of identifying a dangerous obstacle based on the relative relationship includes: Determining whether the angle between the heading of the obstacle and the moving direction of the vehicle meets a preset angle condition; In response to the included angle satisfying the preset angle condition, the obstacle is determined to be the dangerous obstacle.

4. The method according to claim 1, wherein The determining the danger level of the dangerous obstacle based on the detection information of the dangerous obstacle and the vehicle information of the vehicle includes: Based on the detection information and the vehicle information, determining whether there is an obstruction caused by the dangerous obstacle; Based on the detection information and the vehicle information, determining whether the dangerous obstacle is moving toward the vehicle; Based on the judgment result, the danger level of the dangerous obstacle is determined.

5. The method according to claim 4, characterized in that The detection information includes an obstacle position, the vehicle information includes multiple vehicle positions of the vehicle at multiple moments, and the determining, based on the detection information and the vehicle information, whether there is an obstruction caused by the dangerous obstacle includes: Determining a plurality of reference lines between the vehicle and the dangerous obstacle based on the plurality of vehicle positions and the obstacle position; Based on the multiple reference lines, it is determined whether there is any obstruction caused by the dangerous obstacle.

6. The method according to claim 4, characterized in that The detection information includes a plurality of obstacle positions of the dangerous obstacle at a plurality of moments, the vehicle information includes a vehicle position, and determining whether the dangerous obstacle is moving toward the vehicle based on the detection information and the vehicle information includes: Determining a sequence of distances between the dangerous obstacle and the vehicle at the multiple moments based on the multiple obstacle positions and the vehicle position; Based on the distance sequence, it is determined whether the dangerous obstacle is moving toward the vehicle.

7. The method according to claim 4, characterized in that Determining the danger level of the dangerous obstacle based on the judgment result includes: In response to the dangerous obstacle being blocked and not moving toward the vehicle, determining that the dangerous level of the dangerous obstacle is level 1; In response to the dangerous obstacle not being blocked and moving toward the vehicle, determining that the danger level of the dangerous obstacle is a second level; In response to the dangerous obstacle being blocked and moving toward the vehicle, the dangerous level of the dangerous obstacle is determined to be level three.

8. An obstacle recognition and warning system, characterized in that: include: The first recognition module is configured to: identify obstacles within a preset range of the vehicle based on the environmental perception data; a relative relationship determination module, configured to: determine the relative relationship between the obstacle and the vehicle based on the detection information of the obstacle; A second recognition module is configured to: identify dangerous obstacles based on the relative relationship; The danger level determination module is configured to determine the danger level of the dangerous obstacle based on the detection information of the dangerous obstacle and the vehicle information of the vehicle.

9. An obstacle recognition and warning device, characterized in that: The apparatus comprises at least one processor and at least one memory; The at least one memory is for storing computer instructions; The at least one processor is configured to execute at least part of the computer instructions to implement the obstacle identification and warning method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The storage medium stores computer instructions. When a computer reads the computer instructions, the computer executes the obstacle identification and warning method according to any one of claims 1 to 7.

Images