Road surface collapse processing method and device and computer readable storage medium

Abstract

The embodiment of the invention relates to the technical field of artificial intelligence and the technical field of maps, and discloses a road surface collapse processing method and device and a computer readable storage medium, and the method comprises the steps: obtaining sensing data in front of driving through a vehicle-mounted sensing assembly; determining area information, depth information and position information of the abnormal region according to the sensing data; determining a collapse confidence value of the abnormal region based on the area information and the depth information; and under the condition of determining that an early warning condition is satisfied based on the collapse confidence value and the position information, sending out collapse early warning. By applying the technical scheme of the invention, the collapse condition in front of driving can be detected in real time, the condition of missing detection of pavement collapse is avoided, and the driving safety is effectively guaranteed by sending out collapse early warning in time.

Description

Technical Field

[0001] The embodiments of the present invention relate to the fields of artificial intelligence technology and map technology, and specifically to a method and device for processing road collapse and a computer-readable storage medium. Background Art

[0002] Road collapse can cause pits or holes in the road. If there is a collapse in front of a vehicle, it may cause personal injury and property damage.

[0003] In the related art, a collapse detection device or a camera installed on the road surface is used to detect whether there is a collapse in the road surface. However, the coverage rate is low, and there are many missed detections, which cannot guarantee driving safety. Summary of the invention

[0004] In view of the above problems, the embodiments of the present invention provide a method, device and computer-readable storage medium for processing road subsidence, which are used to solve the problem of missing road subsidence areas and failing to ensure driving safety in the prior art.

[0005] According to one aspect of an embodiment of the present invention, a method for processing road collapse is provided, the method comprising: acquiring sensor data ahead of the vehicle through a vehicle-mounted sensor component; determining area information, depth information and position information of an abnormal area based on the sensor data; determining a collapse confidence value of the abnormal area based on the area information and depth information; and issuing a collapse warning when it is determined based on the collapse confidence value and position information that warning conditions are met.

[0006] According to another aspect of an embodiment of the present invention, there is provided a device for processing road collapse, comprising:

[0007] A data acquisition module, used to acquire sensor data ahead of the vehicle through an on-board sensor component;

[0008] An information determination module, used to determine the area information, depth information and location information of the abnormal area based on the sensor data;

[0009] A collapse confidence value determination module, used to determine the collapse confidence value of the abnormal area based on the area information and the depth information;

[0010] The early warning module is used to issue a collapse early warning when it is determined that the early warning conditions are met based on the collapse confidence value and the position information.

[0011] According to another aspect of an embodiment of the present invention, an electronic device is provided, comprising: acquiring sensor data in front of the vehicle through a vehicle-mounted sensor component; determining area information, depth information and position information of an abnormal area based on the sensor data; determining a collapse confidence value of the abnormal area based on the area information and depth information; and issuing a collapse warning when it is determined that a warning condition is met based on the collapse confidence value and the position information.

[0012] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables an electronic device to perform the following operations:

[0013] According to another aspect of an embodiment of the present invention, an electronic device is provided, comprising: acquiring sensor data in front of the vehicle through a vehicle-mounted sensor component; determining area information, depth information and position information of an abnormal area based on the sensor data; determining a collapse confidence value of the abnormal area based on the area information and depth information; and issuing a collapse warning when it is determined that a warning condition is met based on the collapse confidence value and the position information.

[0014] According to the embodiment of the present invention, during the driving process of a vehicle, sensor data in front of the vehicle is acquired in real time, and whether there is an abnormal area in front of the vehicle is detected through the sensor data. A collapse confidence value is determined according to the area information and depth information of the abnormal area. The collapse confidence value represents the credibility of the abnormal area being a collapse area. When the warning conditions are met based on the collapse confidence value and the position information, it indicates that the credibility of the abnormal area being a collapse area is high, and the vehicle is close to the abnormal area, and a collapse warning is issued. The collapse warning prompts the driver of the abnormal condition of the road surface, and the collapse condition in front of the vehicle can be detected in real time, thereby avoiding the situation where the road collapse is missed. By issuing a collapse warning in time, driving safety is effectively guaranteed.

[0015] The above description is only an overview of the technical solution of the embodiment of the present invention. In order to more clearly understand the technical means of the embodiment of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0017] Figure 1 A schematic flow chart of a first embodiment of a method for processing road collapse provided by the present invention is shown;

[0018] Figure 2A schematic diagram showing an application scenario of the method for processing road collapse provided by the present invention;

[0019] Figure 3 A schematic flow chart of a second embodiment of a method for processing road collapse provided by the present invention is shown;

[0020] Figure 4 A schematic structural diagram of a first embodiment of a device for processing road collapse provided by the present invention is shown;

[0021] Figure 5 A schematic structural diagram of an embodiment of an electronic device provided by the present invention is shown. DETAILED DESCRIPTION

[0022] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0023] Figure 1 The flowchart of the first embodiment of the method for processing road collapse of the present invention is shown. The method is executed by an electronic device, which may be a vehicle-mounted terminal. Figure 1 As shown, the method comprises the following steps:

[0024] Step 110: Acquire sensor data ahead of the vehicle through the vehicle-mounted sensor component.

[0025] Among them, the vehicle-mounted sensing component can be at least one of a vehicle-mounted radar and a vehicle-mounted camera. The vehicle-mounted radar can specifically be a forward-looking vehicle-mounted laser radar, and the vehicle-mounted camera can specifically be a forward-looking vehicle-mounted camera; accordingly, the sensing data can be at least one of the point cloud data collected by the vehicle-mounted radar and the image data collected by the vehicle-mounted camera; the driving front is the road surface in front of the vehicle.

[0026] Specifically, when the vehicle is in motion, the sensor data of the vehicle ahead is acquired in real time through the vehicle-mounted sensor component.

[0027] Optionally, when the vehicle's driving environment is a high-brightness environment, the sensor data includes point cloud data and image data, and the point cloud data in front of the vehicle is obtained through the vehicle-mounted radar, and the image data in front of the vehicle is obtained through the vehicle-mounted camera; when the vehicle's driving environment is a low-brightness environment, the sensor data includes point cloud data, and the point cloud data in front of the vehicle is obtained through the vehicle-mounted radar.

[0028] It should be noted that the ambient brightness in a high-brightness environment is higher than the ambient brightness in a low-brightness environment. In a low-brightness environment, the image data obtained by the vehicle-mounted camera may not be clear. In order to improve efficiency, the image data may not be obtained.

[0029] In actual applications, the vehicle's existing on-board sensor components can be reused. When the vehicle's existing functions need to obtain real-time sensor data in front of the vehicle, the sensor data can also be reused directly without adding additional costs, reducing redundant data and improving the efficiency of vehicle data processing.

[0030] Step 120: Determine the area information, depth information and location information of the abnormal area based on the sensing data.

[0031] Among them, the abnormal area is the area in front of the vehicle that may collapse; the area information is used to reflect the area size of the abnormal area, the depth information is used to reflect the depth of the abnormal area, and the position information is used to reflect the position of the abnormal area compared to the vehicle.

[0032] Optionally, the sensing data includes point cloud data, and the electronic device performs target recognition on the point cloud data, and when an abnormal area is identified, determines the area information, depth information and position information of the abnormal area; in a possible implementation method, it can be identified whether there is an abnormal area ahead of the vehicle based on the height value in the point cloud data, and in the point cloud data, the height value of the data point corresponding to the abnormal area is lower than the height value of the data point corresponding to the non-abnormal area.

[0033] Optionally, the sensing data includes point cloud data and image data; the electronic device performs target recognition on the point cloud data and the image data, and performs target fusion on the abnormal targets respectively recognized to obtain area information, depth information and position information of the abnormal area.

[0034] It should be noted that, for target recognition on point cloud data, target recognition may be performed on point cloud data collected at multiple moments in sequence to obtain multiple first recognition results, and the multiple first recognition results are processed by a perception tracking algorithm to determine a first abnormal target; for target recognition on image data to obtain a second abnormal target, target recognition may be performed on image data collected at multiple moments in sequence to obtain multiple second recognition results, and the multiple second recognition results are processed by a perception tracking algorithm to determine the second abnormal target; the point cloud data and image data are acquired in a time-synchronous manner.

[0035] Step 130: Determine the collapse confidence value of the abnormal area based on the area information and the depth information.

[0036] The collapse confidence value is used to reflect the credibility of whether the abnormal area is a collapse area.

[0037] Specifically, the electronic device can determine the corresponding area coefficient according to the area information, determine the corresponding depth coefficient according to the depth information, and calculate the collapse confidence value through the area coefficient and the depth coefficient; when the area coefficient and the depth coefficient are larger, the collapse confidence value is larger.

[0038] Optionally, when the sensor data includes point cloud data, the area information and the depth information are determined based on the point cloud data, and the collapse confidence value is calculated by the area coefficient corresponding to the area information and the depth coefficient corresponding to the depth information; when the sensor data includes point cloud data and image data, the area information includes first area information determined based on the point cloud data, and second area information determined based on the image data, and the collapse confidence value can be calculated based on the area coefficient corresponding to the first area information, the area coefficient corresponding to the second area information, and the depth coefficient corresponding to the depth information.

[0039] Step 140: When it is determined based on the collapse confidence value and the position information that the warning condition is met, a collapse warning is issued.

[0040] Among them, the collapse warning is used to prompt that there may be a collapse ahead of the vehicle. The collapse warning can be issued through at least one of audio, text and light; the driver can slow down or stop the car based on the collapse warning.

[0041] Specifically, the electronic device determines the distance between the vehicle and the abnormal area based on the location information, and issues a collapse warning when the distance and the collapse confidence value meet the warning conditions; in actual applications, the distance and the collapse confidence value meet the warning conditions, which may be that the distance belongs to the distance warning interval, and the collapse confidence value belongs to the confidence warning interval; in this embodiment, the distance that does not belong to the distance warning interval is smaller than the distance that belongs to the distance warning interval, and the collapse confidence value that does not belong to the collapse confidence interval is smaller than the collapse confidence value that belongs to the collapse confidence interval.

[0042] Optionally, the electronic device determines the distance between the vehicle and the abnormal area based on the location information, and determines the time it takes for the vehicle to reach the abnormal area based on the vehicle speed and distance, and issues a collapse warning ahead when the time belongs to the time warning interval and the collapse confidence value belongs to the confidence warning interval; in this embodiment, the time that belongs to the time warning interval is less than the time that does not belong to the time warning interval.

[0043] Optionally, when it is determined that the warning condition is met based on the collapse confidence value, the position information, the area information and the depth information, a collapse warning is issued.

[0044] The method for handling road collapse proposed in the embodiment of the present application obtains sensor data ahead of the vehicle in real time during vehicle driving, detects whether there is an abnormal area ahead of the vehicle through the sensor data, determines a collapse confidence value based on area information and depth information of the abnormal area, and characterizes the credibility of the abnormal area being a collapse area. When the warning conditions are met based on the collapse confidence value and position information, indicating that the credibility of the abnormal area being a collapse area is high, and the vehicle is close to the abnormal area, a collapse warning is issued. The collapse warning prompts the driver of the abnormal condition of the road surface, and the collapse condition ahead of the vehicle can be detected in real time, thereby avoiding the situation where road collapse is missed. By issuing a collapse warning in a timely manner, driving safety is effectively guaranteed.

[0045] In some embodiments, the sensor data includes point cloud data acquired by a vehicle-mounted radar and image data acquired by a vehicle-mounted camera; the area information, depth information, and position information of the abnormal area are determined based on the sensor data, including: when the vehicle is on a preset road and the driving environment is a high-brightness environment, first candidate area information, candidate depth information, and first candidate position information of the first abnormal target are determined based on the point cloud data; second candidate area information and second candidate position information of the second abnormal target are determined based on the image data; based on the first candidate area information, the second candidate area information, the first candidate position information, and the second candidate position information, target fusion is performed on the first abnormal target and the second abnormal target to obtain the abnormal area, as well as the area information and position information of the abnormal area; and the candidate depth information corresponding to the abnormal area is used as the depth information of the abnormal area.

[0046] Among them, the preset road can be a highway, a city road and a national highway; the high-brightness environment can be an environment where the brightness is greater than a preset brightness threshold, such as a daytime environment or a brightly lit environment.

[0047] Specifically, the electronic device obtains the current road of the vehicle and the ambient brightness, and when the road is a preset road and the ambient brightness is greater than a preset brightness threshold, performs target recognition on the point cloud data to obtain a first abnormal target, and first candidate area information, candidate depth information and first candidate position information of the first abnormal target; the electronic device performs target recognition on the image data to obtain a second abnormal target and second candidate area information and second candidate position information of the second abnormal target; based on the first candidate area information, the second candidate area information, the first candidate position information and the second candidate position information, the first abnormal target and the second abnormal target are target-level fused to obtain an abnormal area.

[0048] The first candidate area information corresponding to the abnormal area is used as the first area information of the abnormal area, that is, the first area information is determined based on the point cloud data; the second candidate area information corresponding to the abnormal area is used as the second area information of the abnormal area, that is, the second area information is determined based on the image data; the area information of the abnormal area includes the first area information and the second area information.

[0049] The position information of the abnormal area is determined according to the first candidate position information and the second candidate position information corresponding to the abnormal area; and the candidate depth information of the first abnormal target corresponding to the abnormal area is used as the depth information of the abnormal area.

[0050] In a possible implementation, the network model can be used to perform target recognition on the image data to obtain a second abnormal target. It should be noted that target-level fusion of the first abnormal target and the second abnormal target refers to determining the same target in the first abnormal target and the second abnormal target and treating the same target as the abnormal area.

[0051] Optionally, when the vehicle is not on a preset road, determining the area information, depth information and location information of the abnormal area based on the sensor data will not be performed; when the vehicle is not on a preset road, for example, the vehicle is on a rural road or a winding mountain road, there may be many potholes on the road surface, which may easily lead to false warnings and affect the normal driving of the vehicle.

[0052] Optionally, when the vehicle is on a preset road and the driving environment is a low-brightness environment, the area information, depth information and location information of the abnormal area are determined based on the point cloud data. In a low-brightness environment, the image data obtained by the vehicle camera may not be clear, and in order to improve efficiency, the image data may not be obtained.

[0053] In the above embodiment, the point cloud data and image data are integrated to detect the area information, depth information and position information of the abnormal area ahead of the vehicle, which can improve the accuracy of the abnormal area, as well as the area information, depth information and position information, and thus also improve the accuracy of road collapse processing.

[0054] In some embodiments, based on the area information and the depth information, the collapse confidence value of the abnormal area is determined, including: determining the area coefficient according to the area interval to which the area information belongs; determining the depth coefficient according to the depth interval to which the depth information belongs; determining the area confidence value according to the area information and the area coefficient; determining the collapse confidence value of the abnormal area according to the depth coefficient and the area confidence value.

[0055] Among them, the area interval and the corresponding relationship between the area interval and the area coefficient are preset; the depth interval and the corresponding relationship between the depth interval and the depth coefficient are preset.

[0056] The area confidence value is used to reflect the confidence that the abnormal area is a collapsed area in terms of area.

[0057] Specifically, the electronic device compares the area information with the preset area intervals, determines the area interval to which the area information belongs, and obtains the area coefficient corresponding to the area interval to which the area information belongs; compares the depth information with the preset depth intervals, determines the depth interval to which the depth information belongs, and obtains the depth coefficient corresponding to the depth interval to which the depth information belongs; calculates the product of the depth area coefficient and the area information to obtain an area confidence value, and then calculates the product between the area confidence value and the depth coefficient to obtain a collapse confidence value.

[0058] In the above embodiment, the area and depth data are combined to calculate the collapse confidence value through the area coefficient, area information, depth coefficient and depth information, which improves the accuracy of the collapse confidence value and further improves the accuracy of road collapse processing.

[0059] In some embodiments, the sensing data includes point cloud data and image data; the area information includes first area information determined based on the point cloud data and second area information determined based on the image data;

[0060] Determining an area confidence value according to area information and an area coefficient includes: determining a first area confidence value according to a first area coefficient and the first area information corresponding to the first area information; determining a second area confidence value according to a second area coefficient and the second area information corresponding to the second area information; and determining an area confidence value according to the first area confidence value and the second area confidence value.

[0061] The first area coefficient is determined according to the area interval to which the first area information belongs, and the second area coefficient is determined according to the area interval to which the second area information belongs.

[0062] Optionally, the area coefficient is determined based on the area interval to which the area information belongs, including: comparing the first area information with preset radar area intervals to determine the radar area interval to which the first area information belongs, and using the radar area coefficient corresponding to the radar area interval to which the first area information belongs as the first area coefficient; comparing the second area information with preset image area intervals to determine the image area interval to which the second area information belongs, and using the image area coefficient corresponding to the image area interval to which the second area information belongs as the second area coefficient.

[0063] Exemplarily, the corresponding relationship between the depth interval and the depth coefficient, the corresponding relationship between the radar area interval and the radar area coefficient, and the corresponding relationship between the image area interval and the image area coefficient are shown in Table 1.

[0064] Table 1

[0065]

[0066] The first area confidence value can be calculated by formula (1), and the second area confidence value can be calculated by formula (2).

[0067] Formula (1):

[0068] Formula (2): ;

[0069] in, is the first area confidence value, is the first area information, is the first area coefficient, is the second area confidence value, is the second area information, is the second area coefficient.

[0070] The collapse confidence value is calculated using formula (3).

[0071] Formula (3): ;

[0072] in, is the collapsed confidence value, is the depth coefficient, is the first area confidence value, is the first area information, is the first area coefficient, is the second area confidence value, is the second area information, is the second area coefficient.

[0073] Optionally, the sensor data includes point cloud data; the area information includes area information determined based on the point cloud data, and the collapse confidence value of the abnormal area is determined based on the area information and the depth information, including: comparing the area information with preset radar area intervals, determining the radar area interval to which the area information belongs, and using the radar area coefficient corresponding to the radar area interval to which the area information belongs as the area coefficient; calculating the product between the area coefficient and the area information to obtain the area confidence value; obtaining the depth coefficient corresponding to the depth interval to which the depth information belongs, and using the product between the depth coefficient and the area confidence value as the collapse confidence value.

[0074] In the above embodiment, the area confidence value is determined based on the first surface information of the abnormal area determined by the point cloud data and the second area information of the abnormal area determined by the image data, which improves the accuracy of the area confidence value and further improves the accuracy of the collapse confidence value.

[0075] In some embodiments, when it is determined based on the collapse confidence value and the location information that the warning conditions are met, a collapse warning is issued, including: when it is determined based on the location information that the abnormal area is in the map path, based on the location information and the vehicle speed, determining the time it takes for the vehicle to reach the abnormal area; when the collapse confidence value belongs to the confidence warning interval and the duration belongs to the duration warning interval, a collapse warning is issued.

[0076] Among them, the map path refers to the passable road in the map; in actual application, since the sensor data ahead is obtained, when there is a curve ahead, there may be an abnormal area outside the passable road. In this case, there is no need for collapse warning.

[0077] Meeting the warning conditions means that the collapse confidence value belongs to the confidence warning interval and the duration belongs to the duration warning interval.

[0078] The confidence warning interval and the duration warning interval can be set according to actual needs. The embodiment of the present application does not limit the values ​​of the confidence warning interval and the duration warning interval.

[0079] Specifically, the electronic device matches the location information with the map path. If the location information belongs to the map path, indicating that the abnormal area is on the road ahead of the vehicle, the distance between the vehicle and the abnormal area is determined based on the location information and the vehicle's position, and the time it takes for the vehicle to reach the abnormal area is determined based on the distance and the vehicle speed. If the collapse confidence value belongs to the confidence warning interval and the duration belongs to the duration warning interval, a collapse warning is issued.

[0080] Optionally, when it is determined based on the collapse confidence value and the location information that the warning conditions are met, a collapse warning is issued, including: when it is determined based on the location information that the abnormal area is in the map path, based on the location information and the vehicle speed, determining the time it takes for the vehicle to reach the abnormal area; when the collapse confidence value belongs to the confidence warning interval, the area information belongs to the area warning interval, the depth information belongs to the depth warning interval, and the duration belongs to the duration warning interval, a collapse warning is issued.

[0081] Among them, the area warning interval and the depth warning interval can be set according to actual needs. The embodiment of the present application does not limit the values ​​of the area warning interval and the depth warning interval; that is, when the collapse confidence value, duration, area information and depth information all belong to their respective corresponding warning intervals, a collapse warning is issued, thereby improving the reliability of the collapse warning.

[0082] In the above embodiment, by binding the abnormal area with the map path, it is determined whether the abnormal area is on the map path, which limits the scenarios in which the collapse warning takes effect and avoids false prompts; in addition, when it is determined that the warning conditions are met based on the collapse confidence value and the length of time the vehicle has reached the abnormal area, a collapse warning is issued, so that the driver can receive the collapse warning prompt in time before reaching the abnormal area, effectively ensuring driving safety.

[0083] In some embodiments, the method for processing road collapse further includes: performing braking processing on the vehicle when it is determined that the braking condition is met based on the collapse confidence value and the position information.

[0084] Optionally, when it is determined based on the location information that the abnormal area is in the map path, the time it takes for the vehicle to reach the abnormal area is determined based on the location information and the vehicle speed; when the collapse confidence value belongs to the confidence braking interval and the duration belongs to the duration braking interval, the vehicle is braked.

[0085] It should be noted that the collapse confidence value belonging to the confidence braking interval is greater than the collapse confidence value belonging to the confidence warning interval; the duration belonging to the duration braking interval is less than the duration belonging to the duration warning interval; the collapse confidence value belongs to the confidence braking interval and the duration belongs to the duration braking interval, which means that there is a high possibility that the abnormal area is a collapse area. In order to ensure driving safety, the vehicle can be braked directly.

[0086] Optionally, when the collapse confidence value belongs to the confidence braking interval, the area information belongs to the area braking interval, the depth information belongs to the depth braking interval, and the duration belongs to the duration braking interval, braking processing is performed on the vehicle.

[0087] Optionally, the collapse confidence value and the position information satisfy the braking condition, which may be that after the collapse warning is issued, the duration of the collapse confidence value belonging to the confidence braking interval reaches a preset duration threshold, and the duration of the vehicle reaching the abnormal area determined according to the position information belongs to the duration braking interval. That is, after the collapse warning is issued, the driver does not brake, and as the vehicle gets closer and closer to the abnormal area, the determined collapse confidence value continues to belong to the confidence braking interval, indicating that the abnormal area is very likely to be a collapse area. In order to ensure driving safety, the vehicle can be directly braked.

[0088] It should be noted that the area of ​​the area braking interval is greater than the area of ​​the area warning interval; the depth of the depth braking interval is greater than the depth of the depth warning interval; the collapse confidence value, duration, area information and depth information are comprehensively used to determine whether the braking conditions are met, thereby avoiding false triggering of the brakes and improving the reliability of vehicle braking against road collapse.

[0089] In the above embodiment, when it is determined that the braking conditions are met, the vehicle is braked. When there is a high possibility that the abnormal area is a collapsed area, the vehicle is prevented from entering the collapsed area, thereby effectively ensuring driving safety.

[0090] In some embodiments, the method for handling road collapse also includes: displaying collapse detail information when a collapse warning is issued; in response to a confirmation operation on the collapse detail information, sending the collapse detail information to a cloud platform, and the collapse detail information is used to instruct the cloud platform to add the collapse detail information to the map.

[0091] The collapse details information may include: image data, location information, area information and depth information corresponding to the abnormal area.

[0092] Specifically, after the electronic device issues a collapse warning, the collapse details information can be displayed on the display screen of the electronic device. The driver can confirm the collapse details information after confirming that there is a collapse on the road ahead. The electronic device responds to the confirmation operation and sends the collapse details information to the cloud platform. The cloud platform adds the collapse details information on the map based on the location information, area information and depth information, so that users who use the map application can view the collapse details information. For example, the user can view the corresponding location information, area information and depth information of the abnormal area, as well as the image data of the abnormal area in the map application, and the on-site situation of the abnormal area can be more intuitively understood through the image data.

[0093] Optionally, after the cloud platform adds the collapse details information on the map, for the currently provided navigation path, it obtains the target navigation path including the path where the abnormal area is located in the navigation path, and pushes the collapse details information to users viewing the target navigation path, so that the users can understand the collapse details information and choose to change the route in time.

[0094] Optionally, during the process of braking the vehicle, if it is detected that the throttle opening is greater than a first threshold, or the throttle pedal change rate is greater than a second threshold, the braking of the vehicle is stopped. For example, the first threshold may be 80%, and the second threshold may be 200%. The first threshold and the second threshold may be set according to actual needs, and the embodiment of the present application does not limit this.

[0095] In the above embodiment, the collapse details are sent to the cloud platform to promptly notify other road participants of the collapse time, effectively reducing the risk of subsequent vehicles encountering collapse and further ensuring driving safety.

[0096] In a specific example, the method for handling road collapse can be applied to Figure 2The processing scenario of road collapse shown in the figure. The processing scenario of road collapse includes vehicles, cloud platforms and other vehicles; the vehicle is equipped with a forward-looking vehicle camera and a forward-looking vehicle radar, and the electronic equipment of the vehicle includes an intelligent driving domain controller, which includes a perception fusion unit and a planning control unit. The intelligent driving domain controller can issue a collapse warning and brake the vehicle; the perception fusion unit is used to determine the area information, depth information and location information of the abnormal area based on the sensor data; based on the area information and depth information, the collapse confidence value of the abnormal area is determined; the planning control unit is used to determine whether the warning conditions are met according to the collapse confidence value and location information, and if so, a collapse warning is issued, and whether the braking conditions are met according to the collapse confidence value and location information, and if so, the vehicle is braked. The cloud platform is used to receive the collapse details information and add the collapse details information to the map. Other vehicles can view the collapse details information on the map through the map application.

[0097] In a specific example, Figure 3 As shown, the method for processing road collapse includes: obtaining sensor data of the driving front through a vehicle-mounted sensor component; when the vehicle is on a preset road and the driving environment is a high-brightness environment, the sensor data includes point cloud data and image data, and the area information, depth information and position information of the abnormal area are identified through the point cloud data and the image data; specifically, determining first candidate area information, candidate depth information and first candidate position information of the first abnormal target according to the point cloud data; determining second candidate area information and second candidate position information of the second abnormal target according to the image data; performing target fusion on the first abnormal target and the second abnormal target according to the first candidate area information, the second candidate area information, the first candidate position information and the second candidate position information to obtain the abnormal area, and The area information and position information of the abnormal area are determined based on the sensor data. The candidate depth information corresponding to the abnormal area is used as the depth information of the abnormal area. It is determined whether the abnormal area is on the map path. If it is not on the map path, the step of obtaining the sensor data of the driving ahead through the vehicle-mounted sensor component is returned to the process. If it is on the map path, the collapse confidence value of the abnormal area is determined based on the area information and the depth information. The time it takes for the vehicle to reach the collapse area is determined according to the position information and the vehicle speed. It is determined whether the collapse confidence value belongs to the collapse warning interval and whether the duration belongs to the duration warning interval. If so, a collapse warning is issued. If not, the step of obtaining the sensor data of the driving ahead through the vehicle-mounted sensor component is returned to the process.

[0098] When a collapse warning is issued, determine whether the collapse confidence value belongs to the confidence braking interval and whether the duration belongs to the duration braking interval. If so, brake the vehicle; if not, return to the step of acquiring sensor data ahead through the vehicle-mounted sensor component.

[0099] After a collapse warning is issued or the vehicle is braked, the collapse details information is displayed. In response to a confirmation operation on the collapse details information, the collapse details information is sent to the cloud platform. The cloud platform adds the collapse details information to the map so that other users can view the collapse details information through the map application.

[0100] The method for handling road collapse proposed in the embodiment of the present application obtains sensor data ahead of the vehicle in real time during vehicle driving, detects whether there is an abnormal area ahead of the vehicle through the sensor data, determines a collapse confidence value based on area information and depth information of the abnormal area, and the collapse confidence value represents the credibility of the abnormal area being a collapse area. When the warning conditions are met based on the collapse confidence value and position information, it indicates that the credibility of the abnormal area being a collapse area is high, and the vehicle is close to the abnormal area, and a collapse warning is issued. The collapse warning reminds the driver of the abnormal condition of the road surface, and the collapse condition ahead of the vehicle can be detected in real time, avoiding the situation of missed detection of road collapse. By issuing a collapse warning in time, driving safety is effectively guaranteed.

[0101] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0102] Figure 4 FIG. 2 is a schematic diagram showing the structure of an embodiment of a device for processing road collapse according to the present invention. Figure 4 As shown, the device 400 includes: a data acquisition module 410 , an information determination module 420 , a collapse confidence value determination module 430 and an early warning module 440 .

[0103] The data acquisition module 410 is used to acquire the sensor data ahead of the vehicle through the vehicle-mounted sensor component;

[0104] An information determination module 420, for determining area information, depth information and location information of the abnormal area based on the sensing data;

[0105] A collapse confidence value determination module 430, for determining a collapse confidence value of an abnormal area based on area information and depth information;

[0106] The warning module 440 is used to issue a collapse warning when it is determined based on the collapse confidence value and the position information that the warning condition is met.

[0107] In an optional manner, the sensor data includes point cloud data acquired by the vehicle-mounted radar and image data acquired by the vehicle-mounted camera; the information determination module 420 is also used to determine the first candidate area information, candidate depth information and first candidate position information of the first abnormal target according to the point cloud data when the vehicle is on a preset road and the driving environment is a high-brightness environment; determine the second candidate area information and second candidate position information of the second abnormal target according to the image data; perform target fusion on the first abnormal target and the second abnormal target based on the first candidate area information, the second candidate area information, the first candidate position information and the second candidate position information to obtain the abnormal area, as well as the area information and position information of the abnormal area; and use the candidate depth information corresponding to the abnormal area as the depth information of the abnormal area.

[0108] In an optional manner, the collapse confidence value determination module 430 is also used to determine the area coefficient based on the area interval to which the area information belongs; determine the depth coefficient based on the depth interval to which the depth information belongs; determine the area confidence value based on the area information and the area coefficient; and determine the collapse confidence value of the abnormal area based on the depth coefficient and the area confidence value.

[0109] In an optional manner, the sensing data includes point cloud data and image data; the area information includes first area information determined based on the point cloud data and second area information determined based on the image data;

[0110] The collapse confidence value determination module 430 is also used to determine the first area confidence value based on the first area coefficient and the first area information corresponding to the first area information; determine the second area confidence value based on the second area coefficient and the second area information corresponding to the second area information; and determine the area confidence value based on the first area confidence value and the second area confidence value.

[0111] In an optional manner, the warning module 440 is also used to determine the time it takes for the vehicle to reach the abnormal area based on the position information and the vehicle speed when the abnormal area is determined to be in the map path based on the position information; and to issue a collapse warning when the collapse confidence value belongs to the confidence warning interval and the duration belongs to the duration warning interval.

[0112] In an optional manner, the device for processing road collapse further includes: a braking module, which is used to perform braking processing on the vehicle when it is determined that the braking condition is met based on the collapse confidence value and the position information.

[0113] In an optional manner, the road collapse processing device also includes: an event upload module, which is used to display collapse detail information when a collapse warning is issued; in response to a confirmation operation on the collapse detail information, the collapse detail information is sent to a cloud platform, and the collapse detail information is used to instruct the cloud platform to add the collapse detail information to the map.

[0114] The device for processing road collapse proposed in the embodiment of the present application obtains sensor data ahead of the vehicle in real time during vehicle driving, detects whether there is an abnormal area ahead of the vehicle through the sensor data, determines a collapse confidence value based on area information and depth information of the abnormal area, and the collapse confidence value represents the credibility of the abnormal area being a collapse area. When the warning conditions are met based on the collapse confidence value and position information, it indicates that the credibility of the abnormal area being a collapse area is high, and the vehicle is close to the abnormal area, and a collapse warning is issued. The collapse warning prompts the driver of the abnormal condition of the road surface, and the collapse condition ahead of the vehicle can be detected in real time, thereby avoiding the situation of missed detection of road collapse. By issuing a collapse warning in time, driving safety is effectively guaranteed.

[0115] Figure 5 The schematic diagram of the structure of an embodiment of the electronic device of the present invention is shown, and the specific embodiment of the present invention does not limit the specific implementation of the electronic device.

[0116] like Figure 5 As shown, the electronic device may include: a processor (processor) 502 , a communication interface (Communications Interface) 504 , a memory (memory) 506 , and a communication bus 508 .

[0117] The processor 502, the communication interface 504, and the memory 506 communicate with each other via the communication bus 508. The communication interface 504 is used to communicate with other devices such as a client or other server network elements. The processor 502 is used to execute the program 510, which can specifically execute the relevant steps in the above-mentioned embodiment of the method for processing road collapse.

[0118] Specifically, the program 510 may include program code including computer-executable instructions.

[0119] The processor 502 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the electronic device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0120] The memory 506 is used to store the program 510. The memory 506 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0121] The program 510 may be specifically called by the processor 502 to enable the electronic device to perform the following operations:

[0122] The sensor data in front of the vehicle is obtained through the vehicle-mounted sensor component; the area information, depth information and position information of the abnormal area are determined based on the sensor data; the collapse confidence value of the abnormal area is determined based on the area information and depth information; and a collapse warning is issued when it is determined that the warning conditions are met based on the collapse confidence value and position information.

[0123] An embodiment of the present invention provides a computer-readable storage medium, which stores at least one executable instruction. When the executable instruction is executed on an electronic device, the electronic device executes the method for processing road collapse in any of the above method embodiments.

[0124] The executable instructions can be specifically used to enable the electronic device to perform the following operations: obtain sensor data in front of the vehicle through the on-board sensor component; determine the area information, depth information and position information of the abnormal area based on the sensor data; determine the collapse confidence value of the abnormal area based on the area information and depth information; and issue a collapse warning when it is determined that the warning conditions are met based on the collapse confidence value and position information.

[0125] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system or other device. In addition, the embodiments of the present invention are not directed to any particular programming language.

[0126] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. Similarly, in order to simplify the present invention and help understand one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, the various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. Wherein, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself is a separate embodiment of the present invention.

[0127] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and further may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive.

[0128] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

Claims

1. A method for treating road collapse, characterized in that: The method comprises: Acquire sensor data ahead of the vehicle through the vehicle-mounted sensor components; Determining area information, depth information and location information of the abnormal area based on the sensing data; Determining a collapse confidence value of the abnormal area based on the area information and the depth information; When it is determined based on the collapse confidence value and the position information that a warning condition is met, a collapse warning is issued.

2. The method according to claim 1, characterized in that The sensor data includes point cloud data acquired by the vehicle-mounted radar and image data acquired by the vehicle-mounted camera; The determining of the area information, depth information and location information of the abnormal area based on the sensing data includes: When the vehicle is on a preset road and the driving environment is a high-brightness environment, determining first candidate area information, candidate depth information, and first candidate position information of the first abnormal target according to the point cloud data; Determine second candidate area information and second candidate position information of a second abnormal target according to the image data; According to the first candidate area information, the second candidate area information, the first candidate position information and the second candidate position information, target fusion is performed on the first abnormal target and the second abnormal target to obtain an abnormal area, and area information and position information of the abnormal area; The candidate depth information corresponding to the abnormal area is used as the depth information of the abnormal area.

3. The method according to claim 1, characterized in that The determining, based on the area information and the depth information, a collapse confidence value of the abnormal area comprises: Determining an area coefficient according to the area interval to which the area information belongs; Determining a depth coefficient according to the depth interval to which the depth information belongs; determining an area confidence value according to the area information and the area coefficient; A collapse confidence value of the abnormal area is determined according to the depth coefficient and the area confidence value.

4. The method according to claim 3, characterized in that The sensing data includes point cloud data and image data; the area information includes first area information determined based on the point cloud data and second area information determined based on the image data; The determining the area confidence value according to the area information and the area coefficient includes: determining a first area confidence value according to a first area coefficient corresponding to the first area information and the first area information; determining a second area confidence value according to a second area coefficient corresponding to the second area information and the second area information; An area confidence value is determined based on the first area confidence value and the second area confidence value.

5. The method according to any one of claims 1 to 4, characterized in that The issuing of a collapse warning when it is determined based on the collapse confidence value and the position information that a warning condition is met includes: In the case where it is determined according to the position information that the abnormal area is in the map path, determining the time taken for the vehicle to reach the abnormal area based on the position information and the vehicle speed; When the collapse confidence value belongs to the confidence warning interval and the duration belongs to the duration warning interval, a collapse warning is issued.

6. The method according to any one of claims 1 to 4, characterized in that The method further comprises: When it is determined based on the collapse confidence value and the position information that a braking condition is satisfied, braking processing is performed on the vehicle.

7. The method according to any one of claims 1 to 4, characterized in that The method further comprises: In case of a collapse warning, detailed collapse information is displayed; In response to a confirmation operation on the collapse detail information, the collapse detail information is sent to a cloud platform, and the collapse detail information is used to instruct the cloud platform to add the collapse detail information to a map.

8. A device for treating road collapse, characterized in that: The device comprises: A data acquisition module, used to acquire sensor data ahead of the vehicle through an on-board sensor component; An information determination module, used to determine the area information, depth information and location information of the abnormal area based on the sensing data; A collapse confidence value determination module, used to determine the collapse confidence value of the abnormal area based on the area information and the depth information; The warning module is used to issue a collapse warning when it is determined based on the collapse confidence value and the position information that a warning condition is met.

9. An electronic device, characterized in that: include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the operation of the road collapse processing method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one executable instruction. When the executable instruction is executed on the electronic device, the road collapse processing device executes the operation of the road collapse processing method according to any one of claims 1 to 7.

Images