Abnormal road surface object identification method and device, vehicle and storage medium

Abstract

The invention discloses an abnormal road surface object identification method and device, a vehicle and a storage medium. The method comprises the following steps: acquiring attribute information of a candidate object set contained in a target image; based on the attribute information, a first abnormal road surface object set is recognized from the candidate object set, and abnormal road surface objects in the first abnormal road surface object set are located in a drivable area of the vehicle in the driving scene; based on the position information of the abnormal road surface objects in the first abnormal road surface object set in the drivable area and the driving direction of the vehicle in the driving scene, a second abnormal road surface object set is recognized from the first abnormal road surface object set, and the abnormal road surface objects in the second abnormal road surface object set are located in the driving direction; and outputting a second abnormal road surface object set in response to the fact that the road type of the road in the drivable area is the target road type. The technical problem of low recognition accuracy of the abnormal road surface object is solved.

Description

Technical Field

[0001] The present invention relates to the field of vehicles, and in particular to a method, a device, a vehicle and a storage medium for identifying abnormal road objects. Background Art

[0002] At present, in intelligent vehicle systems, the vehicle's ability to perceive the surrounding environment is the basis for realizing autonomous driving and driving assistance functions. Among them, accurate identification of abnormal road objects (such as speed bumps) on the road is crucial to ensure driving safety and improve driving comfort and efficiency. As a common traffic facility, accurate identification of speed bumps can help vehicles slow down in advance and avoid unnecessary bumps, especially in situations with poor visibility such as at night, rainy and snowy weather, and low light conditions.

[0003] In the related art, the intelligent vehicle system uses a vision-based camera as the main perception means to capture images of the vehicle's surrounding environment to obtain abnormal road objects. However, camera perception technology faces many challenges in practical applications. Specifically, the recognition range of the camera is limited by physical design and environmental factors. For example, at a long distance or at a specific angle, abnormal road objects may be difficult to capture clearly, affecting the camera's recognition results. Changes in light intensity and direction can affect the image quality of the camera. In low-light or strong backlight environments, the features of abnormal road objects may not be easily recognized, resulting in a decrease in the camera's recognition rate. Severe weather conditions, such as rain, snow, fog, etc., may also reduce the visibility of the camera, block or blur the outline of the speed bump, and increase the difficulty of camera recognition. When the camera encounters a pattern or object similar to a speed bump, such as a curb, a shadow of a telephone pole, a parking space barrier, etc., it may misidentify and misreport the above-mentioned non-speed bump object as a speed bump, causing the driver or the autonomous driving system to make an incorrect response. Therefore, there is still a technical problem of low recognition accuracy of abnormal road objects.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] Embodiments of the present invention provide a method, device, vehicle and storage medium for identifying abnormal road objects, so as to at least solve the technical problem of low accuracy in identifying abnormal road objects.

[0006] According to one aspect of an embodiment of the present invention, a method for identifying abnormal road surface objects is provided, the method comprising: acquiring attribute information of a candidate object set contained in a target image, wherein the target image is an image collected during vehicle driving, and the attribute information is at least used to represent the attributes of the candidate objects in the candidate object set in a driving scenario; based on the attribute information, identifying a first abnormal road surface object set from the candidate object set, wherein the abnormal road surface objects in the first abnormal road surface object set are located in a drivable area of ​​the vehicle in a driving scenario; based on the position information of the abnormal road surface objects in the first abnormal road surface object set in the drivable area, and the driving direction of the vehicle in the driving scenario, identifying a second abnormal road surface object set from the first abnormal road surface object set, wherein the abnormal road surface objects in the second abnormal road surface object set are located in the driving direction; in response to the road type of the road in the drivable area being the target road type, outputting a second abnormal road surface object set, wherein the probability of the abnormal road surface objects in the second abnormal road surface object set appearing on a road of the target road type is higher than a probability threshold.

[0007] Optionally, the attribute information includes identification information, and the identification information is used to identify candidate objects. Based on the attribute information, a first abnormal road surface object set is identified from the candidate object set, including: in response to the number of frames of the target image being multiple frames, identifying the identification information corresponding to the same candidate object in multiple consecutive target image frames to obtain a first identification result; in response to the first identification result being that the identification information of the same candidate object in multiple consecutive target image frames is different, adjusting the identification information to obtain a first adjustment result; in response to the first adjustment result being that the identification information of the same candidate object in multiple consecutive target image frames is the same, or in response to the number of frames of the target image being a single frame, identifying candidate objects corresponding to different identification information in the target image to obtain a second identification result, wherein the second identification result is at least used to indicate whether there is an overlapping relationship between candidate objects corresponding to different identification information; based on the second identification result, screening the candidate objects in the candidate object set to obtain a screened candidate object set; and identifying the first abnormal road surface object set from the screened candidate object set.

[0008] Optionally, the second recognition result includes a first sub-recognition result, a second sub-recognition result and a third sub-recognition result. In response to the first adjustment result that the identification information of the same candidate object in multiple consecutive frames of target images is the same, or the number of frames of the target image is a single frame, the candidate objects corresponding to different identification information in the target image are identified to obtain the second recognition result, including: in response to the first adjustment result that the identification information of the same candidate object in multiple consecutive frames of target images is the same, in the target image of the current frame in the multiple consecutive frames, candidate objects with different identification information are identified to obtain the first sub-recognition result; in response to the first sub-recognition result that there is a first candidate object subset in the current frame, candidate objects with different identification information in target images of a target number of frames consecutively after the current frame are identified to obtain the second sub-recognition result, wherein at least two candidate objects in the first candidate object subset have an overlapping relationship, and the overlapping rate between at least two candidate objects is greater than the overlapping rate threshold; in response to the number of frames of the target image being a single frame, candidate objects with different identification information in the target image of the current frame in the single frame are identified to obtain the third sub-recognition result.

[0009] Optionally, based on the second recognition result, the candidate objects in the candidate object set are screened to obtain a screened candidate object set, including: in response to the second sub-recognition result being that a first candidate object subset exists in the target image of a target number of frames, or the third sub-recognition result being that a first candidate object subset exists in the target image, obtaining first distances between at least two candidate objects in the first candidate object subset and the vehicle respectively; and deleting, from the candidate object set, candidate objects in the first candidate object subset whose first distances are greater than a first distance threshold, to obtain a screened candidate object set.

[0010] Optionally, identifying a first abnormal road object set from the screened candidate object set includes: based on the drivable area and attribute information, identifying the candidate objects in the screened candidate object set in the target image of the current frame to obtain a third recognition result; in response to the third recognition result being that there is a second candidate object subset in the screened candidate object set of the current frame, and there are multiple consecutive frames of target images after the current frame, identifying the screened candidate object set in the target images of a target number of consecutive frames after the current frame to obtain a fourth recognition result, wherein the candidate objects in the second candidate object subset are located outside the drivable area, and the proportion of the total area of ​​the candidate objects is greater than or equal to a proportion threshold; in response to the fourth recognition result being that there is a second candidate object subset in the screened candidate object set of a target number of frames, or in response to the third recognition result being that there is a second candidate object subset in the current frame, and there is no target image after the current frame, deleting the second candidate object subset from the screened candidate object set to obtain the first abnormal road object set.

[0011] Optionally, the attribute information of the candidate object set contained in the target image is obtained, including: using a target recognition model to identify the target image to obtain an initial candidate object set in the target image, and initial attribute information corresponding to the initial candidate object set, wherein the target recognition model is obtained by training a deep learning model using image samples and attribute information samples, the image content of the image samples includes candidate object samples, and the attribute information samples are at least used to represent the attributes of the candidate object samples under the driving scene samples; based on the initial attribute information, the positional relationship between the candidate objects in the initial candidate object set is identified to obtain a fifth recognition result, wherein the fifth recognition result is used to indicate whether there is a target position relationship between at least two candidate objects; based on the fifth recognition result, the candidate object set and the attribute information are determined.

[0012] Optionally, based on the fifth recognition result, determining the candidate object set and attribute information includes: in response to the fifth recognition result that there is a third candidate object subset in the initial candidate object set, aggregating the candidate objects in the third candidate object subset to obtain a candidate object aggregation result, and aggregating the initial attribute information of the third candidate object subset to obtain an attribute information aggregation result, wherein the third candidate object subset includes at least two candidate objects, at least two candidate objects have a target position relationship, and a second distance between at least two candidate objects is less than a second distance threshold; determining the candidate object aggregation result and the candidate objects in the initial candidate object set except the third candidate object subset as the candidate object set, and determining the attribute information aggregation result and the initial attribute information corresponding to the candidate objects in the initial candidate object set except the third candidate object subset as the attribute information.

[0013] Optionally, based on the fifth recognition result, determining the candidate object set and attribute information includes: in response to the fifth recognition result that the third candidate object subset does not exist in the initial candidate object set, determining the initial candidate object set as the candidate object set, and determining the initial attribute information as the attribute information, wherein the third candidate object subset includes at least two candidate objects, at least two candidate objects have a target position relationship, and a second distance between at least two candidate objects is less than a second distance threshold.

[0014] Optionally, based on the position information of the abnormal road surface objects in the first abnormal road surface object set in the drivable area and the driving direction of the vehicle in the driving scenario, a second abnormal road surface object set is identified from the first abnormal road surface object set, including: based on the position information and the driving direction, the first abnormal road surface object set is identified in the target image of the current frame to obtain a sixth identification result; in response to the sixth identification result that the first abnormal road surface object exists in the target image of the current frame and there are multiple consecutive target images after the current frame, the first abnormal road surface object set is identified in a target number of consecutive target images after the current frame to obtain a seventh identification result, wherein the first abnormal road surface object is an abnormal road surface object in the first abnormal road surface object set that is not in the driving direction; in response to the seventh identification result that the first abnormal road surface object exists in the target image of the target number of frames, or in response to the sixth identification result that the first abnormal road surface object exists in the target image of the current frame and there is no target image after the current frame, the first abnormal road surface object is deleted from the first abnormal road surface object set to obtain a second abnormal road surface object set.

[0015] Optionally, the method also includes: in response to the road type being the target road type, obtaining length information and width information of abnormal road surface objects in the second abnormal road surface object set; in response to the length information not satisfying the length information threshold, adjusting the length information according to the length information threshold; and / or, in response to the width information not satisfying the width information threshold, adjusting the width information according to the width information threshold; based on the adjusted length information and / or the adjusted width information, formulating an abnormal road surface object model corresponding to the abnormal road surface objects in the second abnormal road surface object set according to a target scaling ratio, wherein the abnormal road surface object model is used to prompt the existence of abnormal road surface objects in the driving direction at future moments that will affect the normal driving of the vehicle; and rendering and displaying the abnormal road surface object model on the vehicle's graphical user interface.

[0016] Optionally, the abnormal road surface object model is rendered onto a graphical user interface of the vehicle, including: determining the relative position between the abnormal road surface object and the vehicle based on the position information of the abnormal road surface object; rendering the abnormal road surface object model and a vehicle model corresponding to the vehicle on the graphical user interface according to the relative position; and controlling the abnormal road surface object model to move toward the vehicle model according to the speed of the vehicle.

[0017] According to another aspect of an embodiment of the present invention, there is also provided an apparatus for identifying abnormal road surface objects, comprising: an acquisition unit, for acquiring attribute information of a candidate object set contained in a target image, wherein the target image is an image collected during vehicle driving, and the attribute information is at least used to represent the attributes of the candidate objects in the candidate object set in a driving scenario; a first identification unit, for identifying a first abnormal road surface object set from the candidate object set based on the attribute information, wherein the abnormal road surface objects in the first abnormal road surface object set are located in a drivable area of ​​the vehicle in a driving scenario; a second identification unit, for identifying a second abnormal road surface object set from the first abnormal road surface object set based on position information of the abnormal road surface objects in the first abnormal road surface object set in the drivable area and the driving direction of the vehicle in the driving scenario, wherein the abnormal road surface objects in the second abnormal road surface object set are located in the driving direction; and an output unit, for outputting a second abnormal road surface object set in response to the road type of the road in the drivable area being the target road type, wherein the probability of the abnormal road surface objects in the second abnormal road surface object set appearing on a road of the target road type is higher than a probability threshold.

[0018] According to another aspect of an embodiment of the present invention, there is further provided an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein the method in each embodiment of the present invention is executed when the program is running.

[0019] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present invention.

[0020] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0021] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0022] According to another aspect of the embodiments of the present invention, a computer program is further provided. When the computer program is executed by a processor, the methods in the embodiments of the present invention are implemented.

[0023] According to another aspect of an embodiment of the present invention, a vehicle is provided, including a memory and a processor, wherein the memory stores an executable program, and the processor is used to run the program, wherein the method in each embodiment of the present invention is executed when the program is run.

[0024] In an embodiment of the present invention, during the driving process of the vehicle, if it is necessary to identify the abnormal road surface objects around the vehicle, the target image can be collected in real time, and the attribute information of the candidate object set contained in the target image can be identified. The abnormal road surface objects are first screened by judging whether the candidate objects in the candidate object set are in the drivable area of ​​the vehicle through the attribute information, and the first abnormal road surface object set is screened. The abnormal road surface objects can be screened for the second time through the position information of the abnormal road surface objects in the first abnormal road surface object set and the driving direction of the vehicle, and the second abnormal road surface object set is screened. It can be judged whether the road type of the road currently traveled by the vehicle is the target road type. If the road type is the target road type, it can be explained that the probability of abnormal road surface objects appearing on the road is high, and the abnormal road surface objects in the second abnormal road surface object set can be retained. On the contrary, if the road type is not the target road type, it can be explained that the probability of abnormal road surface objects appearing on the road is low, and the abnormal road surface objects in the second abnormal road surface object set can be deleted. In an embodiment of the present invention, through the above method, accurate identification and effective screening of abnormal road surface objects are achieved to ensure that the focus can be placed on abnormal road surface objects that have a direct impact on the future driving of the vehicle. The purpose of reducing misidentification and missed identification is achieved, providing a more reliable and safer environmental perception for the intelligent vehicle system, ensuring the safety and comfort of vehicle driving. This achieves the technical effect of improving the accuracy of identifying abnormal road objects and solves the technical problem of low accuracy in identifying abnormal road objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0026] Figure 1 is a flow chart of a method for identifying abnormal road surface objects according to an embodiment of the present invention;

[0027] Figure 2 is a schematic diagram of a vehicle computer display and misidentification correction and filtering system for speed bumps according to an embodiment of the present invention;

[0028] Figure 3 is a structural block diagram of a device for identifying abnormal road surface objects according to an embodiment of the present invention;

[0029] Figure 4 is a structural block diagram of an autonomous driving vehicle according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only embodiments of a part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] According to an embodiment of the present invention, an embodiment of a method for identifying abnormal road surface objects is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0033] The embodiment of the present application provides a method for identifying abnormal road objects. The method can be used to provide abnormal road object identification and prompts for vehicles in preset application scenarios to guide vehicles to travel in future periods. The above-mentioned preset application scenarios may include the following scenarios in the vehicle field: commuting automatic driving scenarios, artificial intelligence (AI) driving scenarios for family cars, automatic parking assistance (APA) scenarios (such as memory parking for self-owned parking spaces in garages, smart parking for designated parking spaces in parking lots, etc.), and intelligent navigation assistance (Navigation Guided Pilot, NGP) scenarios in urban areas or high-speed areas. In addition, the above-mentioned preset application scenarios may also include but are not limited to: automatic driving scenarios of smart driving trucks or unmanned trucks in the field of logistics and transportation that require the use of augmented reality navigation functions, automatic driving scenarios of automatic driving agricultural vehicles in the field of agricultural machinery that require the use of augmented reality navigation functions, automatic driving scenarios of drones that require the use of augmented reality navigation functions, and automatic driving scenarios of intelligent robots (such as cleaning robots, service robots, delivery robots, etc.) that require the use of augmented reality navigation functions.

[0034] When the above-mentioned preset application scenarios are scenarios in fields other than the vehicle field, those skilled in the art should be able to understand that the vehicle in the above-mentioned vehicle information interaction method can be replaced with other objects (such as agricultural machinery, drones, robots, etc.), and accordingly, the use of augmented reality navigation function to guide the above-mentioned vehicle driving is replaced with navigation for other objects, and guiding other objects to move, fly or drive. On this basis, in the embodiments of the present application, the specific implementation method of the above-mentioned vehicle information interaction method is exemplified by taking the vehicle field as an example.

[0035] Figure 1 FIG. 1 is a flow chart of a method for identifying abnormal road surface objects according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0036] Step S102: Acquire attribute information of a set of candidate objects contained in the target image.

[0037] In the technical solution provided in the above step S102 of the present invention, the target image may be an image collected during the driving process of the vehicle, for example, it may be an image around the vehicle. The attribute information may be used at least to represent the attributes of the candidate objects in the candidate object set in the driving scene, and may be the state information of the candidate objects in the candidate object set, that is, the state information of each object in the image around the vehicle, for example, it may be information such as the length and position of the speed bump. The candidate objects may be various objects in the environment around the vehicle in the target image, such as pedestrians, objects (other vehicles, traffic signs, speed bumps, etc.).

[0038] In this embodiment, when the vehicle is traveling on a road in the driving scene, a target image of the vehicle's surrounding environment can be acquired in real time. The target image can be recognized to obtain attribute information of the candidate object set contained therein.

[0039] Optionally, the above method is a key link in the process of an intelligent vehicle system (eg, an intelligent driving system) identifying abnormal road objects around a vehicle, and is used to extract attribute information of a set of candidate objects contained in a target image acquired in real time.

[0040] Optionally, the target image can be an image of the vehicle's surroundings captured in real time by a vehicle-mounted camera (referred to as camera). The above-mentioned vehicle-mounted camera can be deployed at the front, rear or both sides of the vehicle to provide all-round field of view coverage to ensure that abnormal road objects in the direction of vehicle travel can be captured. The image quality captured by the camera directly affects the accuracy and efficiency of subsequent processing. It should be noted that the above-mentioned method for obtaining the target image, the device for collecting the target image and the deployment location are only for illustration and are not specifically limited here.

[0041] Optionally, identifying the candidate object set contained in the target image is achieved through image processing technologies such as deep learning and computer vision. The candidate object set includes all possible abnormal road objects in the image, such as speed bumps, potholes, obstacles, etc., as well as other objects related to vehicle driving safety, such as pedestrians, other vehicles, traffic signs, etc. The process of identifying the candidate object set includes but is not limited to the steps of target detection, feature extraction and classification.

[0042] Optionally, if the target is a speed bump that affects the vehicle's driving process in the future period, the candidate objects in the candidate object set may be the speed bump identified from the target image, and other objects whose similarity to the speed bump is greater than a similarity threshold. If the target is various objects that affect the vehicle's driving process in the future period, the candidate objects in the candidate object set may be various items identified from the target image. It should be noted that the above candidate objects are only examples and are not specifically limited here.

[0043] Optionally, the attribute information obtained from the target image is used in a subsequent screening step to determine which candidate objects are truly abnormal road surface objects that will affect the vehicle driving process in the future period, and which ones need to be filtered out.

[0044] Optionally, in the process of extracting attribute information from the target image, there may be various technical challenges, such as image blur, illumination changes, complex background, etc. To overcome the above challenges, enhanced image processing algorithms, illumination compensation techniques, background suppression methods, etc. may be used to improve the accuracy and reliability of the acquired attribute information.

[0045] In the embodiment of the present invention, by identifying a set of candidate objects from the target image and extracting their attribute information, a key data basis is provided for the subsequent screening and identification of abnormal road objects. The accuracy and efficiency of the above embodiment are the key to ensuring that the intelligent driving system can effectively detect and respond to abnormal road objects.

[0046] Step S104: identifying a first abnormal road surface object set from the candidate object set based on the attribute information.

[0047] In the technical solution provided in the above step S104 of the present invention, the abnormal road surface objects in the first abnormal road surface object set may be located in the drivable area of ​​the vehicle in the driving scenario. The first abnormal road surface object set may be obtained by performing a first-level screening of the candidate objects in the candidate object set through the relationship between the candidate objects and the drivable area. The drivable area refers to the area where the vehicle can or should travel in the current driving scenario, which is usually closely related to the vehicle's driving path, lane information and road type. The drivable area may be a place where the vehicle can travel in a broad sense, for example, it may be the road on which the vehicle is currently traveling, that is, it may be the area after excluding the areas on the curbs on both sides of the road.

[0048] In this embodiment, after acquiring the attribute information of the candidate object set contained in the target image, a first abnormal road surface object set consisting of abnormal road surface objects located in the drivable area of ​​the vehicle can be identified from the candidate object set based on the attribute information.

[0049] Optionally, the above embodiment is a core link of the abnormal road object recognition process in the intelligent driving system. Based on the attribute information extracted from the target image, the abnormal road objects located in the vehicle drivable area can be screened out from the candidate object set to form a first abnormal road object set. The above process is crucial to ensure the safety and comfort of vehicle driving.

[0050] Optionally, the identification of the drivable area is based on the comprehensive judgment of the vehicle's real-time position, driving direction, lane information provided by the lane keeping system, and high-precision map data. The above definition of the drivable area helps to focus on abnormal road objects that may have an impact on the vehicle's driving in the future period, and avoids processing information that is not related to driving safety.

[0051] Optionally, based on the extracted attribute information, in particular the location of the candidate object, it can be determined whether each candidate object is located within the drivable area. For example, this can be achieved by comparing the location of the candidate object with the boundary of the drivable area. For candidate objects whose location is shown to be outside the drivable area, no further processing will be performed, thereby reducing the processing burden of false positives and invalid information. For candidate objects whose location is shown to be within the drivable area, they can be used as abnormal road surface objects in the first abnormal road surface object set for subsequent further processing.

[0052] It should be noted that the above process and method for determining whether a candidate object in the candidate object set is in a drivable area is for illustrative purposes only and is not specifically limited here. As long as the candidate objects in the candidate object set can be screened according to the attribute information, the candidate objects in the drivable area can be screened out and the process and method for forming the first abnormal road surface object set are within the protection scope of the embodiments of the present invention.

[0053] Optionally, in addition to the position, if the attribute information is information such as length, width, height, etc., it is also a key basis for determining whether the candidate object belongs to the first abnormal road surface object set. For example, if the length of the speed bump exceeds a certain threshold and its position indicates that it crosses the driving path of the vehicle, then the speed bump can be identified as an abnormal road surface object in the first abnormal road surface object set that has a potential impact on the vehicle's driving. For candidate objects whose attribute information indicates that they do not pose a threat to driving safety, they will be excluded from the first abnormal road surface object set.

[0054] In an embodiment of the present invention, by analyzing and screening attribute information, the intelligent driving system can accurately identify abnormal road objects located in the drivable area from a large number of candidate objects, forming a first abnormal road object set, providing accurate environmental perception information for subsequent intelligent driving decisions, and effectively improving the safety and comfort of vehicle driving.

[0055] Step S106: Based on the position information of the abnormal road surface objects in the first abnormal road surface object set in the drivable area and the driving direction of the vehicle in the driving scenario, a second abnormal road surface object set is identified from the first abnormal road surface object set.

[0056] In the technical solution provided in the above step S106 of the present invention, the abnormal road surface objects in the second abnormal road surface object set are located in the driving direction, that is, the abnormal road surface objects in the second abnormal road surface object set can be abnormal road surface objects that affect the current travel of the vehicle. The position information can be the coordinate information of the abnormal road surface object. The driving direction can also be referred to as the current direction of travel of the vehicle (current travel path). Contrary to the driving direction, the scenes in directions other than the driving direction can be referred to as the perception disadvantage scenes of the vehicle. For example, if the current driving direction of the vehicle is forward, the sides and rear of the vehicle can be perception disadvantage scenes. At this time, the abnormal road surface objects in the second abnormal road surface object set are in front of the vehicle, and the abnormal road surface objects removed from the first abnormal road surface object set are on both sides and rear of the vehicle.

[0057] In this embodiment, after the first abnormal road surface object is identified from the candidate object set based on the attribute information, the abnormal road surface objects can be screened at a second level based on the location information of the abnormal road surface objects in the first abnormal road surface object set and the driving direction of the vehicle to obtain a second abnormal road surface object set.

[0058] Optionally, the above embodiment not only considers the position information of the abnormal road surface object in the drivable area, but also combines the actual driving direction of the vehicle to ensure that only the abnormal road surface objects that have a direct impact on the current driving direction of the vehicle are focused on.

[0059] Optionally, the vehicle's driving direction, provided by the vehicle's navigation system or autonomous driving module, is an important reference for determining the driving path ahead and potential obstacles. Position information, i.e., the coordinate information of each object in the first abnormal road surface object set, is the basis for determining whether the object is located in the driving direction. By combining the above two, it is possible to accurately identify which abnormal road surface objects are in the vehicle's driving path and which are not in the vehicle's driving path.

[0060] Optionally, each abnormal road surface object in the first abnormal road surface object set is evaluated, and its position information relative to the vehicle's driving direction is used to determine whether it is on the vehicle's path. Only abnormal road surface objects whose position information indicates that they are in the driving direction will be included in the second abnormal road surface object set. The above method ensures that the focus is on the driving direction, that is, the area where the vehicle is about to drive to, rather than the perception of vulnerable scenes. Therefore, the perception of abnormal road surface objects in vulnerable scenes has less impact on the current driving safety and comfort of the vehicle.

[0061] For example, abnormal road objects that are not in the driving direction will be removed in the process of forming the second abnormal road object set. For example, if the vehicle is currently driving forward, the abnormal road objects on both sides and behind the vehicle will be regarded as abnormal road objects in the perception-weak scene and will not be included in the second abnormal road object set. The above method helps to reduce the complexity of processing and the consumption of computing resources, while avoiding unnecessary alarms, improving the driver's driving experience and the efficiency of the autonomous driving system.

[0062] Optionally, the recognition of abnormal road objects is limited to the direction of travel for both safety and driving experience considerations. From a safety perspective, abnormal road objects in front of the vehicle are critical to immediate driving decisions and can be recognized and responded to in a timely manner. From a driving experience perspective, avoiding false alarms of abnormal road objects in perception-weak scenarios can reduce unnecessary distractions and allow the driver or autonomous driving system to focus more on the driving task ahead.

[0063] In an embodiment of the present invention, through the above method, the intelligent driving system can more accurately identify and screen out abnormal road objects that have a direct impact on vehicle driving safety and comfort, forming a second set of abnormal road objects, providing key information for subsequent decision-making and response, helping to improve driving safety, reduce misidentification and invalid alarms, and enhance the overall performance of the intelligent driving system.

[0064] Step S108: In response to the road type of the road in the drivable area being the target road type, outputting a second abnormal road surface object set.

[0065] In the technical solution provided in the above step S108 of the present invention, the probability of abnormal road surface objects in the second abnormal road surface object set appearing on the road of the target road type is higher than the probability threshold. The probability of abnormal road surface objects appearing on the road of the target road type is higher than the probability threshold. For example, if the identification of speed bumps is targeted, the road of the target road type may have a probability of speed bumps appearing on the road of the park, a small road, and an underground parking lot, etc., which are higher than the probability threshold. It should be noted that the above target road types are for illustrative purposes only and are not specifically limited here.

[0066] In this embodiment, after determining the second abnormal road surface object set based on the position information of the abnormal road surface objects in the drivable area and the driving direction of the vehicle in the first abnormal road surface object set, the road type of the current driving road in the drivable area of ​​the vehicle can be obtained. It can be determined whether the road type of the road is the target road type. If so, the second abnormal road surface object can be output. Otherwise, the abnormal road surface object in the second abnormal road surface object set can be deleted.

[0067] Optionally, based on the analysis of the road type and the probability of occurrence of abnormal road surface objects, the second abnormal road surface object set is dynamically managed for output or deletion. The above embodiments are intended to improve driving safety and driving experience while reducing false alarms and resource waste of intelligent vehicle systems.

[0068] Optionally, the probability of occurrence of abnormal road surface objects (such as speed bumps) on different road types is significantly different. Speed ​​bumps are more commonly set up on sections such as urban internal roads, park roads, paths or underground parking lots due to the particularity of traffic management and usage scenarios; while on highways, urban expressways, etc., the probability of speed bumps is relatively low due to the requirements of design speed and road surface continuity. The system analyzes the road type and matches it with the pre-set speed bump occurrence probability threshold to determine whether the current section belongs to the target road type.

[0069] For example, the vehicle's global positioning system (GPS), high-precision map data, and real-time traffic information are used to obtain and identify the type of road the vehicle is currently on in real time. Through the above road type identification, it is ensured that the recognition and response strategy for abnormal road objects can be intelligently adjusted according to the specific environment in which the vehicle is located.

[0070] Optionally, based on the acquired road type information, it is determined whether the current road section belongs to the target road type. If the current road type matches the target road type, that is, the probability of the speed bump appearing is higher than a set threshold, a second abnormal road surface object set can be output to remind the driver or the automatic driving system to pay attention to the speed bump ahead. Conversely, if the current road type does not belong to the target road type, that is, the probability of the speed bump appearing is low, the speed bump information in the second abnormal road surface object set will not be output, avoiding unnecessary warnings and reducing the processing load.

[0071] Optionally, on non-target road types, by deleting the second abnormal road surface object set, interference to the driver or misguidance to the automatic driving system is avoided. The above method reduces unnecessary calculation and resource consumption, and improves the overall efficiency and response speed of the intelligent driving system.

[0072] For example, on the target road type, the output second abnormal road surface object set will be transmitted to the driver or the automatic driving system through the vehicle's driving assistance system or automatic driving control module. For example, the location, type, size and other information of the abnormal road surface objects in the second abnormal road surface object set can be displayed on the vehicle's dashboard, central control screen or head display system, as well as possible recommended driving speed or path adjustment suggestions. Through the above-mentioned early warning information, the driver can react in advance and improve driving safety.

[0073] It should be noted that the above processing and function for the output second abnormal road surface object set are only for illustration and no specific limitation is imposed here.

[0074] Optionally, the definition of target road types is flexible and can be adjusted based on the specific conditions of different regions and driving environments. For example, the probability threshold of speed bumps on different road types can be dynamically updated based on the latest traffic data and user feedback to adapt to the ever-changing driving environment and road conditions. The above dynamic adaptability and scalability ensure that its recognition and decision-making strategies can be continuously optimized to adapt to various complex driving scenarios.

[0075] In the embodiment of the present invention, by accurately identifying and managing abnormal road object information and combining road type and driving environment, driving safety is enhanced and driving experience is optimized, reflecting the intelligent management and decision-making capabilities of the intelligent driving system for environmental perception.

[0076] In the above steps S102 to S108 of the present invention, during the vehicle driving process, if it is necessary to identify the abnormal road surface objects around the vehicle, the target image can be collected in real time, and the attribute information of the candidate object set contained in the target image can be identified. The abnormal road surface objects are first screened by judging whether the candidate objects in the candidate object set are in the drivable area of ​​the vehicle through the attribute information, and the first abnormal road surface object set is obtained by screening. The abnormal road surface objects can be secondly screened by the position information of the abnormal road surface objects in the first abnormal road surface object set and the driving direction of the vehicle, and the second abnormal road surface object set is obtained by screening. It can be judged whether the road type of the road currently traveled by the vehicle is the target road type. If the road type is the target road type, it can be explained that the probability of abnormal road surface objects appearing on the road is high, and the abnormal road surface objects in the second abnormal road surface object set can be retained. On the contrary, if the road type is not the target road type, it can be explained that the probability of abnormal road surface objects appearing on the road is low, and the abnormal road surface objects in the second abnormal road surface object set can be deleted. In this embodiment, the above method is used to achieve accurate identification and effective screening of abnormal road objects, ensuring that the focus can be placed on abnormal road objects that have a direct impact on the vehicle's future driving. The purpose of reducing misidentification and missed identification is achieved, providing a more reliable and safe environmental perception for the intelligent vehicle system, and ensuring the safety and comfort of vehicle driving. Thereby achieving the technical effect of improving the accuracy of identifying abnormal road objects and solving the technical problem of low accuracy of identifying abnormal road objects.

[0077] The following further describes how to perform the first-level screening of abnormal road surface objects based on attribute information in this embodiment.

[0078] As an optional implementation, the attribute information includes identification information, and the identification information is used to identify the candidate object. Step S104, based on the attribute information, identifies a first abnormal road surface object set from the candidate object set, including: in response to the number of frames of the target image being multiple frames, identifying the identification information corresponding to the same candidate object in multiple consecutive target image frames to obtain a first identification result; in response to the first identification result being that the identification information of the same candidate object in multiple consecutive target image frames is different, adjusting the identification information to obtain a first adjustment result; in response to the first adjustment result being that the identification information of the same candidate object in multiple consecutive target image frames is the same, or in response to the number of frames of the target image being a single frame, identifying the candidate objects corresponding to different identification information in the target image to obtain a second identification result, wherein the second identification result is at least used to indicate whether there is an overlapping relationship between the candidate objects corresponding to different identification information; based on the second identification result, screening the candidate objects in the candidate object set to obtain a screened candidate object set; and identifying the first abnormal road surface object set from the screened candidate object set.

[0079] In this embodiment, the attribute information includes identification information, and the identification information may be an identifier (ID) of the candidate object, for example, a speed bump ID. The first recognition result may be used to indicate whether the identification information of the same candidate object in multiple consecutive target image frames is the same, that is, whether the identification information of the candidate object is stable. The second recognition result may be used to indicate whether the candidate objects corresponding to different identification information have an overlapping relationship.

[0080] Optionally, based on multiple frames of images and attribute information (especially identification information), candidate objects are screened and identified to form a complex process of the first abnormal road object set. This process is intended to ensure the accuracy and stability of the abnormal road objects identified by the system and avoid misidentification caused by jumps in identification information or overlapping relationships between candidate objects.

[0081] Optionally, when processing multiple consecutive target images, the identification information of the same candidate object in multiple consecutive frames is identified and compared. The above process can detect whether the identification information remains consistent in multiple frames, that is, the stability of the ID. For example, the same speed bump should have the same ID in several consecutive frames to ensure that the same candidate object can be continuously tracked, rather than being mistaken for a different candidate object.

[0082] Optionally, if the identification information of the same candidate object is found to be inconsistent in the first recognition results of multiple consecutive frames, that is, there is an ID jump, the ID can be adjusted to ensure that the ID of the same candidate object remains stable in subsequent processing. The above adjustment can be based on the characteristics of the candidate object (such as position, shape) to reallocate the ID, or analyze multiple frames of data through a fusion model to determine a stable ID. The adjusted result is the first adjustment result. It should be noted that the above process and method for adjusting the ID are only for illustration and are not specifically limited here.

[0083] Optionally, whether processing multiple frames or a single frame, the second recognition result can be obtained based on the adjusted identification information or by directly identifying the candidate object in the single frame. The above second recognition result not only includes the state information of the candidate object, but also may include the overlapping relationship between the candidate objects corresponding to different identification information, that is, whether multiple IDs point to the same or overlapping areas in the image, which is helpful for the subsequent screening process.

[0084] Optionally, the candidate object set is screened according to the overlapping relationship reflected in the second recognition result. If the candidate objects corresponding to multiple identification information overlap or almost overlap in space, a candidate object that is most likely to be an abnormal road surface object may be retained, and the remaining overlapping objects may be deleted to avoid repeated recognition and false positives. The screened candidate object set will more accurately reflect the possible abnormal road surface objects.

[0085] In the embodiment of the present invention, the step of identifying the first abnormal road surface object set from the candidate object set ensures accurate, stable and efficient identification of abnormal road surface objects through identification information identification, adjustment and screening based on overlapping relationships, which is an important part of environmental perception and decision-making in the intelligent driving system. The implementation of the above method helps to improve driving safety and comfort, and at the same time reflects the adaptability of the intelligent driving system to complex environments and the intelligent level of data processing.

[0086] In this embodiment, when the identification information of the same candidate object in multiple consecutive frames of target images is the same, or when the number of frames of the target image is a single frame, the process of performing the first-level screening of abnormal road surface objects based on attribute information is further explained below.

[0087] As an optional implementation, the second recognition result includes a first sub-recognition result, a second sub-recognition result, and a third sub-recognition result. In response to the first adjustment result that the identification information of the same candidate object in multiple consecutive frames of target images is the same, or the number of frames of the target image is a single frame, the candidate objects corresponding to different identification information in the target image are identified to obtain the second recognition result, including: in response to the first adjustment result that the identification information of the same candidate object in multiple consecutive frames of target images is the same, the candidate objects with different identification information in the target image of the current frame in the multiple consecutive frames are identified to obtain the first sub-recognition result; in response to the first sub-recognition result that there is a first candidate object subset in the current frame, the candidate objects with different identification information in the target images of the target number of frames consecutively after the current frame are identified to obtain the second sub-recognition result, wherein at least two candidate objects in the first candidate object subset have an overlapping relationship, and the overlapping rate between at least two candidate objects is greater than the overlapping rate threshold; in response to the number of frames of the target image being a single frame, the candidate objects with different identification information in the target image of the current frame in the single frame are identified to obtain the third sub-recognition result.

[0088] In this embodiment, the second recognition result may include a first sub-recognition result, a second sub-recognition result, and a third sub-recognition result. There is an overlapping relationship between at least two candidate objects in the first candidate object subset, and the overlap rate between at least two candidate objects is greater than the overlap rate threshold. The target number of frames can be a preset number of frames, for example, it can be 4 consecutive frames after the current frame. The overlap rate threshold can be a preset percentage, for example, it can be preset to 40%. It should be noted that the specific numerical values ​​of the above-mentioned target number of frames and the overlap rate threshold are only for example purposes and are not specifically limited here.

[0089] Optionally, the above embodiment is a process of multi-level recognition and screening of candidate objects based on the stability and overlapping relationship of identification information to improve the accuracy and stability of abnormal road surface object (e.g., speed bump) recognition. The above process is divided into the generation of three sub-recognition results, each of which has its specific conditions and purpose.

[0090] Optionally, when the identification information remains stable in multiple consecutive frames of target images, that is, the ID of the same candidate object is the same in multiple frames, the system will identify it in the target image of the current frame to obtain the first sub-identification result. The above recognition process not only covers all candidate objects with different identification information, but also provides a basis for the subsequent identification of candidate object subsets with overlapping relationships in the current frame. Through the above process, the consistency of object identification can be ensured, avoiding tracking difficulties caused by inconsistent identification information.

[0091] Optionally, if the first sub-recognition result indicates that there are at least two candidate objects in the current frame that have an overlapping relationship, and the overlapping rate is greater than a preset overlapping rate threshold (e.g., 40%), the system will further identify the target image of a target number of consecutive frames (e.g., 4 consecutive frames) after the current frame to obtain a second sub-recognition result. The purpose of the above process is to verify whether the candidate objects in the overlapping relationship still exist in subsequent frames and whether the overlapping relationship between them continues. In this way, the stability of the overlapping relationship can be judged based on time series data, providing a more reliable basis for the final abnormal road object recognition.

[0092] Optionally, for the processing of a single-frame target image, candidate objects with different identification information can be directly identified in the frame image to obtain a third sub-recognition result. The third sub-recognition result is mainly for special cases other than image sequence processing. For example, when processing a single-frame image, it is also necessary to be able to identify possible abnormal road objects to ensure that accurate environmental perception information can be provided in relevant situations.

[0093] In the embodiment of the present invention, by generating the first sub-recognition result, the second sub-recognition result and the third sub-recognition result, combined with the stability of the identification information and the overlapping relationship between the candidate objects, the accurate, stable and efficient recognition of abnormal road objects is achieved. The implementation of the above recognition process helps to improve the overall performance of the intelligent driving system, ensure driving safety and comfort, and at the same time reflects the system's adaptability to complex environments and the intelligent level of data processing.

[0094] The following describes in detail how to screen the candidate objects in the candidate object set according to the second recognition result in this embodiment.

[0095] As an optional implementation, based on the second recognition result, the candidate objects in the candidate object set are screened to obtain a screened candidate object set, including: in response to the second sub-recognition result being that a first candidate object subset exists in the target image of a target number of frames, or the third sub-recognition result being that a first candidate object subset exists in the target image, obtaining the first distances between at least two candidate objects in the first candidate object subset and the vehicle respectively; and deleting from the candidate object set the candidate objects whose first distances in the first candidate object subset are greater than a first distance threshold, to obtain a screened candidate object set.

[0096] In this embodiment, the first distance threshold can be a preset value, which can be used to screen the candidate objects in the first candidate object subset. For example, the first distance threshold is used to filter out the candidate objects that are far away from the vehicle and retain the candidate objects that are close to the vehicle. In the intelligent driving system, how to perform distance screening on the candidate objects based on the second recognition result to further improve the practicality and effectiveness of identifying abnormal road objects (such as speed bumps). The above process mainly focuses on distance information to ensure that those candidate objects that have a direct impact on the current vehicle driving are retained, thereby reducing unnecessary data processing and warnings, and improving the system's response speed and driving experience.

[0097] Optionally, based on the second recognition result, in particular, the first candidate object subset in the second sub-recognition result and the third sub-recognition result, a first distance between at least two candidate objects in the subset and the vehicle is obtained. The first distance is usually obtained based on the vehicle's position information (e.g., GPS data) and the relative position of the candidate object in the image, and is calculated using a built-in geometric model of the system.

[0098] Optionally, after obtaining the first distance, a pre-set first distance threshold can be applied to screen which candidate objects have an actual impact on the current driving state of the vehicle. For example, setting a first distance threshold of 50 meters means that the system will only retain those candidate objects within 50 meters of the vehicle. The setting of the above thresholds takes into account the vehicle's driving speed, reaction time, and warning requirements for abnormal road objects, ensuring the timeliness and effectiveness of the warning information.

[0099] Optionally, candidate objects whose first distance in the first candidate object subset is greater than the first distance threshold are deleted from the candidate object set. The above process excludes those candidate objects that are far away from the vehicle and will not affect driving in a short period of time, thereby reducing the system processing load, avoiding unnecessary interference to the driver, and also improving the system's response speed and processing efficiency.

[0100] In the embodiment of the present invention, by obtaining the first distance and applying the first distance threshold to screen the candidate objects, the intelligent driving system can effectively identify and process those abnormal road objects that have a practical impact on vehicle driving. The implementation of the above screening steps reflects the intelligence and efficiency of the system in environmental perception and decision-making, and is a key link in ensuring driving safety and improving driving experience.

[0101] The following further explains the process of how to identify the first abnormal road surface object set from the screened candidate object set in this embodiment.

[0102] As an optional implementation, step S104, identifying a first abnormal road object set from the screened candidate object set, includes: based on the drivable area and attribute information, identifying the candidate objects in the screened candidate object set in the target image of the current frame to obtain a third recognition result; in response to the third recognition result being that there is a second candidate object subset in the screened candidate object set of the current frame, and there are multiple consecutive frames of target images after the current frame, identifying the screened candidate object set in the target images of a target number of consecutive frames after the current frame to obtain a fourth recognition result, wherein the candidate objects in the second candidate object subset are located outside the drivable area, and the proportion of the total area of ​​the candidate objects is greater than or equal to a proportion threshold; in response to the fourth recognition result being that there is a second candidate object subset in the screened candidate object set of a target number of frames, or in response to the third recognition result being that there is a second candidate object subset in the current frame, and there is no target image after the current frame, deleting the second candidate object subset from the screened candidate object set to obtain the first abnormal road object set.

[0103] In this embodiment, the ratio threshold may be a preset value, for example, the ratio threshold may be preset to 80%. It should be noted that the size of the above ratio threshold is only for illustration and is not specifically limited here. The above embodiment illustrates how the intelligent driving system further screens and identifies abnormal road surface objects based on the drivable area and attribute information, ensuring that the first abnormal road surface object set outputted in the end not only meets the conditions in terms of identification information and distance, but is also within the range of the vehicle's drivable path, thereby improving the recognition accuracy of the system and driving safety.

[0104] Optionally, when processing the filtered candidate object set, the candidate object in the current frame target image is identified based on the current drivable area of ​​the vehicle and the attribute information of the candidate object to obtain a third identification result. The key to the above identification process is to determine whether the candidate object is located on the drivable path of the vehicle, that is, whether it is in the drivable area. Only candidate objects in the drivable area may have an impact on the vehicle's driving. Therefore, the implementation of the above method can effectively exclude those insignificant objects and reduce false alarms.

[0105] Optionally, if the third recognition result indicates that there is a second subset of candidate objects in the current frame, and the proportion of the area outside the drivable area in the above candidate objects to the total area is greater than or equal to a preset ratio threshold (e.g., 80%), it can be further determined whether there are multiple consecutive frames of target images after the current frame. If so, the screened candidate object set in the consecutive frames is identified to obtain a fourth recognition result. The purpose of the above process is to verify the stability of the second subset of candidate objects, that is, whether the candidate objects in the second subset of candidate objects continue to appear in subsequent consecutive frames, and whether their positions and attributes continue to meet the conditions, to ensure the reliability and continuity of the recognition results.

[0106] Optionally, the generation of the fourth recognition result is a further confirmation of the stability and positional relationship of the second candidate object subset in the multi-frame image. If the fourth recognition result also indicates that the second candidate object subset exists in the candidate object set after screening of the target number of frames, or the third recognition result indicates that the second candidate object subset exists in the current frame and there is no target image afterwards (which means that its stability cannot be further verified), the second candidate object subset will be deleted from the candidate object set after screening to obtain the final first abnormal road surface object set.

[0107] Optionally, a ratio threshold (e.g., 80%) is set to ensure that only those candidate objects that are mostly within the drivable area are focused on. The rationality of the above ratio threshold ensures that objects that have little impact on driving can be effectively excluded, such as curbstones, parking space barriers, etc. Even if the above candidate objects are identified as speed bumps, it may be because they have similar visual features to speed bumps, but they do not actually require the vehicle to slow down. Reasonable setting of the ratio threshold helps reduce misidentification and improve the accuracy and effectiveness of the system.

[0108] In the embodiment of the present invention, the intelligent level of environmental perception and decision-making is demonstrated through the identification of drivable areas and screening based on ratio thresholds. The above decision logic not only ensures that abnormal road objects that have a practical impact on driving can be accurately identified, but also improves processing efficiency, reduces the processing of irrelevant data, and reduces the burden on computing resources. The identification and screening process based on drivable areas and attribute information is part of the multi-layer recognition strategy of the intelligent driving system. It not only optimizes the identification of abnormal road objects, but also provides a basis for the expansion of the system (such as adapting to different road types and conditions). By reasonably adjusting the definition of the ratio threshold and the drivable area, the system can adapt to a wider range of driving environments and improve its universality and robustness.

[0109] In summary, the identification and screening steps based on the drivable area and attribute information can ensure that the first abnormal road object set output by the intelligent driving system not only meets the conditions in terms of identification information and distance, but is also within the reasonable range of the vehicle's driving path, thereby improving the system's recognition accuracy and the accuracy of early warning. The implementation of the above process reflects the intelligence and efficiency of the system in environmental perception and decision-making logic, and is an important technical means to ensure driving safety and improve driving experience.

[0110] The following further describes the process of how to obtain the attribute information of the candidate object set contained in the target image in this embodiment.

[0111] As an optional implementation, step S102, obtaining attribute information of a candidate object set contained in a target image, includes: using a target recognition model to identify the target image to obtain an initial candidate object set in the target image, and initial attribute information corresponding to the initial candidate object set, wherein the target recognition model is obtained by training a deep learning model using image samples and attribute information samples, the image content of the image samples includes candidate object samples, and the attribute information samples are at least used to represent the attributes of the candidate object samples under the driving scene samples; based on the initial attribute information, the positional relationship between the candidate objects in the initial candidate object set is identified to obtain a fifth recognition result, wherein the fifth recognition result is used to indicate whether there is a target position relationship between at least two candidate objects; based on the fifth recognition result, the candidate object set and the attribute information are determined.

[0112] In this embodiment, the target recognition model can be obtained by training a deep learning model using initial image samples and attribute information samples. The image content of the image sample includes a candidate object sample. The attribute information sample is at least used to represent the attributes of the candidate object sample under the driving scene sample. The fifth recognition result can be used to indicate whether there is a target position relationship between at least two candidate objects. The above embodiment describes how the intelligent driving system uses a deep learning model to obtain the attribute information of a set of candidate objects in a target image, and identifies the positional relationship between candidate objects based on the above information, and finally determines the set of candidate objects that need to be paid attention to and their attributes. The above process is an important part of the environmental perception of the intelligent driving system, which can ensure that the system's recognition of abnormal road objects (such as speed bumps) is both accurate and context-aware.

[0113] Optionally, after obtaining the initial candidate object set and its attribute information, the relative position relationship between the candidate objects can be further analyzed to obtain a fifth recognition result. The fifth recognition result reflects whether there is a target position relationship between at least two candidate objects, that is, whether there is an obvious correlation or overlap between the candidate objects in the direction and path of the vehicle. The above process is very critical for excluding insignificant objects, such as those located at the edge of the vehicle's path or completely off the path.

[0114] Optionally, based on the fifth recognition result, it is possible to determine which candidate objects have a direct relationship with the vehicle's driving path, thereby determining the final set of candidate objects that need to be paid attention to and their attribute information. The above confirmation process further filters out those objects that are initially recognized by the model but actually have little impact on driving decisions, thereby improving the efficiency and accuracy of system processing.

[0115] Optionally, the recognition of positional relationships is a key step to ensure that the abnormal road objects identified by the system do pose a potential impact on vehicle driving. Through the above recognition, objects that require the driver or the autonomous driving system to take immediate action can be further accurately located, reducing the risk of misidentification and improving driving safety. At the same time, the above recognition process also reflects the depth of the system's understanding of the environment, that is, it not only recognizes the objects themselves, but also understands the spatial relationship between objects and the significance of this relationship for vehicle driving.

[0116] In the embodiment of the present invention, the entire process of acquiring the attribute information of the candidate object set in the target image reflects the intelligence and dynamism of the intelligent driving system in environmental perception and information processing. Through the recognition of the deep learning model and further screening based on the position relationship, it is possible to quickly and accurately identify abnormal road objects that affect the current driving state, and at the same time dynamically adjust the recognition strategy according to the actual driving path and environmental conditions of the vehicle, thereby improving the adaptability and flexibility of the system. In summary, by training and utilizing the target recognition model, identifying the candidate object set and its attribute information in the target image, and then analyzing and screening the position relationship based on the above information, and finally determining the candidate object set and attributes that the system needs to pay attention to, it is a key technology in the environmental perception and decision-making of the intelligent driving system. The implementation of the above process ensures that abnormal road objects that pose a potential risk to vehicle driving can be accurately and efficiently identified, providing an important guarantee for driving safety.

[0117] The following further describes the process of determining the attribute information of the candidate object set based on the fifth recognition result in this embodiment.

[0118] As an optional implementation, based on the fifth recognition result, a candidate object set and attribute information are determined, including: in response to the fifth recognition result that a third candidate object subset exists in the initial candidate object set, the candidate objects in the third candidate object subset are aggregated to obtain a candidate object aggregation result, and the initial attribute information of the third candidate object subset is aggregated to obtain an attribute information aggregation result, wherein the third candidate object subset includes at least two candidate objects, at least two candidate objects have a target position relationship, and a second distance between at least two candidate objects is less than a second distance threshold; the candidate object aggregation result and the candidate objects in the initial candidate object set except the third candidate object subset are determined as the candidate object set, and the attribute information aggregation result and the initial attribute information corresponding to the candidate objects in the initial candidate object set except the third candidate object subset are determined as the attribute information.

[0119] In this embodiment, the third candidate object subset may include at least two candidate objects that have a target position relationship with each other, and the second distance between them is less than the second distance threshold. The above embodiment describes how the intelligent driving system determines the final candidate object set and attribute information through aggregation processing based on the fifth recognition result, and the above process is intended to improve the accuracy of recognition and the efficiency of system processing.

[0120] Optionally, the fifth recognition result reveals whether there is a target position relationship between the above-mentioned initial candidate objects, that is, whether there is a subset of candidate objects that are positionally associated with each other. The recognition of the above-mentioned position relationship, based on the relative distance and spatial layout between the candidate objects, ensures that those objects that may constitute continuous or repeated obstacles on the driving path can be identified.

[0121] Optionally, if the fifth recognition result indicates that there is a third candidate object subset, that is, at least two candidate objects in the initial candidate object set have a target position relationship, and the second distance between these objects is less than a second distance threshold (for example, the threshold is set to ensure that the system can identify objects such as speed bumps that may appear continuously or very close). The system will perform aggregation processing on the candidate objects in the third candidate object subset to reduce the risk of repeated recognition and false alarms and improve processing efficiency. At the same time, the initial attribute information of the third candidate object subset is aggregated to reflect a more accurate attribute state of the aggregated object, such as size, position, etc.

[0122] Optionally, after the aggregation process, the generated candidate object aggregation results are combined with other candidate objects in the initial candidate object set except the third candidate object subset to determine the final candidate object set. At the same time, the aggregated attribute information is combined with the initial attribute information of the candidate objects that have not been aggregated to determine the final attribute information. This process ensures that the system only retains those candidate objects that have a real impact on the driving status, as well as the precise attribute information of these objects, providing a more reliable data basis for subsequent decision-making and early warning.

[0123] Optionally, the second distance threshold is set (for example, to 0.1 meters) to ensure that candidate objects that are extremely close or appear continuously in space, that is, continuous speed bumps or very close obstacles, can be identified. For example, the speed bumps corresponding to the lane where the vehicle is currently traveling and the object lane are connected to each other. The rationality of the above second distance threshold avoids identifying objects that are actually the same obstacle or continuous obstacles as multiple independent objects, reducing the complexity of misidentification and data processing.

[0124] In the embodiment of the present invention, the aggregation of candidate objects and the updating of attribute information based on the fifth recognition result is a key step in the environment recognition and processing of the intelligent driving system. Through the above process, those abnormal road objects that have a practical impact on the driving path can be identified and processed, avoiding the processing of irrelevant data, improving the accuracy and efficiency of recognition, and providing important guarantees for driving safety and comfort.

[0125] The following further describes the process of determining the attribute information of the candidate object set based on the fifth recognition result in this embodiment.

[0126] As an optional implementation, based on the fifth recognition result, determining a candidate object set and attribute information, including: in response to the fifth recognition result that the third candidate object subset does not exist in the initial candidate object set, determining the initial candidate object set as the candidate object set, and determining the initial attribute information as the attribute information, wherein the third candidate object subset includes at least two candidate objects, at least two candidate objects have a target position relationship, and a second distance between at least two candidate objects is less than a second distance threshold.

[0127] In this embodiment, the above embodiment describes how to directly determine the candidate object set and attribute information in the intelligent driving system when the fifth recognition result indicates that there is no third candidate object subset that needs to be aggregated. The above implementation ensures that the system can quickly and accurately process image data when no continuous or very close abnormal road objects are identified, and directly use the identified objects and their attribute information for subsequent decision-making and warning.

[0128] Optionally, the target image is identified using a target recognition model to obtain an initial candidate object set and its corresponding initial attribute information. Subsequently, the fifth recognition result is used to determine whether there is a target position relationship between the objects in the initial candidate object set, that is, whether they are continuous or very close in space and meet the second distance threshold. The above process is the basis for the system to identify and process abnormal road objects, ensuring the accuracy and effectiveness of subsequent steps.

[0129] Optionally, if the fifth recognition result indicates that the third candidate object subset does not exist in the initial candidate object set, that is, no object subset that needs to be aggregated is identified, it usually means that the currently identified objects are scattered in spatial layout, and there are no continuous obstacles or very close obstacles that need special processing. In the above case, the system will directly determine the initial candidate object set as the candidate object set and the initial attribute information as the attribute information for subsequent decision making and early warning.

[0130] In an embodiment of the present invention, when the fifth recognition result indicates that there is no third candidate object subset that requires special processing, the intelligent driving system can directly determine the initial candidate object set as the final candidate object set, and the initial attribute information as the final attribute information. The implementation of the above process ensures efficiency and accuracy in processing abnormal road objects, and reflects the system's adaptability to different driving scenarios and flexibility in decision-making.

[0131] The following further describes the process of identifying the second abnormal road surface object set from the first abnormal road surface object set based on the location information of the abnormal road surface objects in the first abnormal road surface object set in the drivable area and the driving direction of the vehicle in the driving scenario in this embodiment.

[0132] As an optional implementation, step S106, based on the position information of the abnormal road surface objects in the first abnormal road surface object set in the drivable area and the driving direction of the vehicle in the driving scene, identifies the second abnormal road surface object set from the first abnormal road surface object set, including: based on the position information and the driving direction, identifying the first abnormal road surface object set in the target image of the current frame to obtain a sixth recognition result; in response to the sixth recognition result that the first abnormal road surface object exists in the target image of the current frame, and there are multiple consecutive frames of target images after the current frame, identifying the first abnormal road surface object set in target images of a target number of frames consecutively after the current frame to obtain a seventh recognition result, wherein the first abnormal road surface object is an abnormal road surface object in the first abnormal road surface object set that is not in the driving direction; in response to the seventh recognition result that the first abnormal road surface object exists in the target images of the target number of frames, or in response to the sixth recognition result that the first abnormal road surface object exists in the target image of the current frame and there is no target image after the current frame, deleting the first abnormal road surface object from the first abnormal road surface object set to obtain a second abnormal road surface object set.

[0133] In this embodiment, the first abnormal road surface object may be an abnormal road surface object in the first abnormal road surface object set that is not in the driving direction. The above embodiment describes how the intelligent driving system further screens and identifies the second abnormal road surface object set based on the location information of the abnormal road surface object in the drivable area and the driving direction of the vehicle. The above process ensures that the abnormal road surface object information output by the system truly affects the vehicle's driving path and direction, thereby improving driving safety and the pertinence of warning information.

[0134] Optionally, based on the position information and the driving direction, the first abnormal road surface object set in the current frame target image is identified to obtain a sixth identification result. The above identification is based on the current driving direction and drivable area of ​​the vehicle to determine which abnormal road surface objects are actually on the driving path of the vehicle, that is, the first abnormal road surface objects (first abnormal road surface objects) that are not in the driving direction are identified to ensure that the system can focus on objects that really pose a threat to driving.

[0135] Optionally, if the sixth recognition result indicates that there is a first abnormal road surface object in the target image of the current frame, and the system determines that there are multiple consecutive target image frames after the current frame, then the system will recognize the first abnormal road surface object set in the consecutive frames to obtain the seventh recognition result. The generation of the seventh recognition result verifies the stability of the first abnormal road surface object in subsequent consecutive frames, that is, whether the abnormal road surface object continues to appear on the vehicle's driving path, thereby further confirming their impact on driving.

[0136] Optionally, according to the seventh recognition result, if the first abnormal road surface object exists in the target image of the target number of frames, or the sixth recognition result indicates that the first abnormal road surface object exists in the target image of the current frame, but there is no target image later (which means that its stability cannot be verified), the system will delete the first abnormal road surface object from the first abnormal road surface object set to obtain the second abnormal road surface object set. The above process ensures that the system only retains those abnormal road surface objects that have a real impact on the driving path and have been verified by continuous frames, thereby improving the accuracy and practicality of the warning information.

[0137] In the embodiment of the present invention, the intelligent driving system can accurately identify abnormal road objects that have a practical impact on vehicle driving and determine the second abnormal road object set through identification based on location information and driving direction, and further screening based on continuous frame stability verification. The implementation of the above process reflects the intelligence and efficiency of the system in environmental perception and decision-making, and is an important part of ensuring driving safety and improving driving experience.

[0138] In this embodiment, the process of how to render on the graphical user interface after the abnormal road surface object that affects the vehicle driving process is identified is further described below.

[0139] As an optional implementation, the method also includes: in response to the road type being the target road type, obtaining length information and width information of abnormal road surface objects in the second abnormal road surface object set; in response to the length information not satisfying the length information threshold, adjusting the length information according to the length information threshold; and / or, in response to the width information not satisfying the width information threshold, adjusting the width information according to the width information threshold; based on the adjusted length information and / or the adjusted width information, formulating an abnormal road surface object model corresponding to the abnormal road surface objects in the second abnormal road surface object set according to a target scaling ratio, wherein the abnormal road surface object model is used to prompt the existence of abnormal road surface objects in the driving direction at future moments that will affect the normal driving of the vehicle; and rendering and displaying the abnormal road surface object model on the vehicle's graphical user interface.

[0140] In this embodiment, the graphical user interface (GUI) can be a vehicle screen or vehicle navigation, which is not specifically limited here. The length information threshold can be preset to 3m, and the width information threshold can be preset to 0.2m. No specific limitation is made here. The above embodiment describes how the intelligent driving system adjusts and displays the model of the abnormal road surface object in the second abnormal road surface object set according to the road type to ensure the accuracy and visualization of the driving warning information.

[0141] Optionally, in response to the condition that the road type is a target road type (such as an urban road, a highway, etc.), the length information and width information of the abnormal road surface objects in the second abnormal road surface object set are obtained. The above process shows that different road types may require different abnormal road surface object information processing strategies, because the size and layout of abnormal road surface objects (such as speed bumps) on different road types may be significantly different, and the impact on vehicle driving is also different.

[0142] Optionally, check whether the length information and width information of the abnormal road surface object meet the preset length information threshold and width information threshold. If the length or width of an abnormal road surface object does not meet the threshold requirement, the system will adjust it according to the set threshold. The above adjustment process is intended to ensure that the size information of the abnormal road surface object is sufficiently accurate and consistent to correctly reflect the impact of road conditions on vehicle driving, while avoiding a deterioration in driving experience caused by misleading or mismatched information.

[0143] Optionally, based on the adjusted length information and / or width information, an abnormal road surface object model can be formulated according to a target scaling ratio. The setting of the target scaling ratio ensures that the display of the abnormal road surface object model on the vehicle graphical user interface is both intuitive and accurate, and can be appropriately scaled according to the relative distance between the vehicle and the object and the viewing angle, so that the driver can intuitively understand the size and position of the abnormal road surface object, and thus make driving preparations in advance.

[0144] Optionally, the prepared abnormal road surface object model is rendered and displayed on the vehicle's graphical user interface. The above display is usually combined with the vehicle's real-time position, driving direction and speed information, and the abnormal road surface object model is placed in front of the vehicle model to intuitively remind the driver of the potential risks in the future driving direction. The above visual warning improves driving safety and also enhances the driving experience, allowing the driver to more clearly understand the conditions of the road ahead and take appropriate driving strategies in a timely manner.

[0145] Optionally, the setting of the length information threshold and the width information threshold, as well as the information adjustment process, is intended to ensure that the system's recognition and display of abnormal road objects are both accurate and reasonable. The size information of abnormal road objects directly affects the accuracy and effectiveness of driving warnings. Reasonable threshold adjustment can ensure that the system can provide consistent and reliable warning information under different road types and driving conditions.

[0146] Optionally, by adjusting the size information of abnormal road objects and formulating abnormal road object models according to the target scaling ratio, the system demonstrates its intelligence level in information processing and visualization. The above process ensures that the driving warning information is not only based on the real-time vehicle driving status, but also takes into account the road type and the actual size of abnormal road objects, improving the accuracy of the warning information and the intuitive feeling of the driver.

[0147] In the embodiment of the present invention, through the above embodiment, not only the accuracy of the system in identifying abnormal road objects is improved, but also the timeliness and intuitiveness of driving warning information are ensured, which is a key technical means to improve driving safety and comfort. The intelligent driving system ensures the accuracy and visualization of driving warning information by adjusting the size information of abnormal road objects according to the road type and formulating and displaying the abnormal road object model according to a specific scaling ratio.

[0148] The following further describes the process of how to render the abnormal road surface object model on the graphical user interface of the vehicle in this embodiment.

[0149] As an optional implementation, rendering the abnormal road surface object model onto the vehicle's graphical user interface includes: determining the relative position between the abnormal road surface object and the vehicle based on the position information of the abnormal road surface object; rendering the abnormal road surface object model and the vehicle model corresponding to the vehicle on the graphical user interface according to the relative position; and controlling the abnormal road surface object model to move toward the vehicle model according to the speed of the vehicle.

[0150] In this embodiment, the above embodiment renders the abnormal road surface object model onto the vehicle's graphical user interface GUI, which is a key step for the intelligent driving system to provide intuitive driving warning information to the driver.

[0151] Optionally, the relative position between the abnormal road surface object and the vehicle is determined using the vehicle's positioning system and the coordinate information of the abnormal road surface object. The above process usually involves coordinate conversion and distance calculation to ensure that the abnormal road surface object model can be accurately placed in front of the vehicle model and is consistent with the actual driving path and direction of the vehicle. The determination of the relative position is crucial for subsequent rendering and mobile control, which directly affects the accuracy and intuitiveness of the driving warning information. After determining the relative position of the abnormal road surface object and the vehicle, the model of the abnormal road surface object and the model of the vehicle can be rendered on the GUI of the vehicle. The above rendering usually adopts 3D rendering technology in computer graphics, and the abnormal road surface object model is placed in front of the vehicle model in the form of a rectangular frame, while taking into account factors such as viewing angle, scaling and ambient lighting, so that the driver can intuitively see the abnormal situation on the road ahead. The use of a rectangular frame not only simplifies the calculation and rendering of the model, but also can clearly indicate the scope of the abnormal road surface object, which is convenient for the driver to judge and deal with.

[0152] Optionally, in order to simulate the actual driving state of the vehicle on the road, the abnormal road surface object model is dynamically controlled to move toward the vehicle model according to the speed of the vehicle. The above dynamic rendering effect makes the abnormal road surface object model seem to be "oncoming" on the road, enhancing the urgency and real-time nature of the warning information. The real-time update of vehicle speed information and the synchronous control of model movement not only improve the accuracy of driving warnings, but also make the driving experience more realistic and immersive.

[0153] In an embodiment of the present invention, by determining the relative position of the abnormal road surface object and the vehicle, dynamically rendering the abnormal road surface object model to the vehicle's GUI, and controlling the movement of the model according to the vehicle speed, an intuitive, real-time and dynamic driving warning is provided to the driver.

[0154] The technical solution of the embodiment of the present invention is illustrated below in conjunction with preferred implementation modes.

[0155] At present, drivers cannot see speed bumps in advance in conditions with poor vision, such as at night, in backlight, in rain, snow, or in congestion, which causes unexpected bumps when the vehicle passes over the speed bump, causing panic. Vehicles' perception of speed bumps is relatively unstable, and they may misidentify speed bumps due to various patterns / objects similar to speed bumps (curb, shadows of electric poles, parking space barriers, etc.). When the driver sees the misidentified speed bump on the large screen of the vehicle, he will think that there is a speed bump in front of him or under his vehicle, resulting in sudden deceleration and other behaviors that affect driving safety.

[0156] However, the embodiment of the present invention proposes a method for correcting and filtering the in-vehicle display and misidentification of speed bumps, which displays the speed bumps perceived by the vehicle in the environmental reconstruction on the large screen of the in-vehicle and the instrument, helping the driver to recognize the speed bumps ahead in advance when the field of vision is poor. In order to solve the problem of unstable recognition of speed bumps, the speed bumps are deleted and filtered according to the relative position of the speed bumps and the vehicle and the road grade of the vehicle, avoiding the driver's sudden deceleration due to misidentification of speed bumps on highways, urban expressways and main roads. Thereby achieving the technical effect of improving the recognition accuracy of abnormal road surface objects and solving the technical problem of low accuracy in the recognition of abnormal road surface objects.

[0157] The method of the embodiment of the present invention is further illustrated below.

[0158] Figure 2 is a schematic diagram of a vehicle computer display and misidentification correction and filtering system for speed bumps according to an embodiment of the present invention. Figure 2 As shown, the system may include an autonomous driving domain controller A and a cockpit domain controller B. The autonomous driving domain controller A may include a perception module 201, a fusion module 202, and a correction module 203. The cockpit domain controller B may include a rendering module 204.

[0159] Alternatively, if Figure 2 As shown, the perception module 201 can use the image of the surroundings of the vehicle collected by the camera as input, and use the state information of each object in the image, including the length and position of the speed bump, as output. Specifically, the vehicle camera collects the image of the surroundings of the vehicle and transmits it to the perception module 201. The perception module 201 determines the state of each object in the image through the deep learning model, frames the object identified as the speed bump with a box, aggregates multiple speed bumps, and outputs them to the fusion module 202.

[0160] Alternatively, if Figure 2 As shown, the fusion module 202 can take the state information of the surrounding objects of the perception module 201, including the state information of the ground speed bump, as input. The starting and ending point coordinate information of the ground speed bump is output. Specifically, the fusion module 202 receives the data of the upstream perception module, is responsible for the stable tracking of the ground speed bump, and performs deduplication and filtering of the speed bump according to the rules: 1) Track each speed bump to ensure that the ID of the same speed bump is stable and the speed bump ID does not jump due to short-term occlusion; 2) If the overlap rate of speed bumps with different IDs exceeds 40% and lasts for 5 frames, filter out the speed bumps farther from the vehicle and retain the speed bumps closer to the vehicle; 3) According to the vehicle drivable area provided by the upstream, filter out the speed bumps that are 80% or more outside the drivable area; 4) Perform multi-frame filtering on the speed bump, and output it after the upstream speed bump appears stable for 5 frames to prevent the output of the speed bump from jumping.

[0161] Alternatively, if Figure 2 As shown, the deviation correction module 203 takes the coordinate information of the starting and ending points of the ground speed bump as input to place the four corner points of the rectangular frame of the speed bump. Specifically, the deviation correction module receives the speed bump information of the fusion module, and makes fine adjustments to the size of the speed bump to ensure that the display of the speed bump is more beautiful and closer to reality (there will be no speed bumps that are too long or too wide). At the same time, the speed bump is filtered and deleted according to the relative position of the speed bump and the vehicle and the road grade where the vehicle is located. According to the coordinates of the speed bump, it is determined whether the speed bump spans across the front of the vehicle. For the speed bump that spans the front of the vehicle and is longer than 3m, the speed bump is cut to a length of 3m based on the end face closer to the vehicle, ensuring that the display of the speed bump can serve as a reminder to the driver, while not being too abrupt and not matching with other road elements. The width of the speed bump is uniformly set to 0.2m to ensure the consistency of the display width and achieve the beauty of the display. Speed ​​bumps that are in the weak perception scenario and have no impact on the current movement of the vehicle (such as speed bumps on the side of the vehicle body) are deleted and filtered to avoid display errors and jumps. The navigation positioning signal is received, and the speed bumps identified in the sections with low probability of speed bumps such as expressways, urban expressways, provincial roads, national roads, and urban trunk roads are filtered and deleted according to the road grade where the vehicle is currently located. Speed ​​bumps in sections with high probability of speed bumps such as internal roads, small roads, and underground parking lots in the park are displayed transparently.

[0162] Alternatively, if Figure 2 As shown, the rendering module 204 takes the four corner points of the rectangular frame in which the speed bump is placed as input, and renders the speed bump on the vehicle large screen as output. Specifically, the speed bump model is placed in the rectangular frame, and the speed bump is associated with the vehicle speed. According to the relative position of the speed bump and the vehicle, the speed bump is rendered around the vehicle model on the large screen and the instrument panel.

[0163] In the embodiment of the present invention, the fusion module selects and merges the speed bumps according to the drivable area and the overlap of the speed bumps. The correction module trims and adjusts the length and width of the speed bumps. The correction module removes and filters the speed bumps according to the positional relationship between the speed bumps and the vehicle. The correction module removes and filters the speed bumps according to the road grade where the vehicle is located. This achieves the technical effect of improving the recognition accuracy of abnormal road surface objects and solves the technical problem of low recognition accuracy of abnormal road surface objects.

[0164] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0165] According to another aspect of the embodiment of the present invention, corresponding to the embodiment of the above-mentioned method for identifying abnormal road objects, this specification also provides a device for identifying abnormal road objects. Figure 3 FIG. 1 is a structural block diagram of a device for identifying abnormal road surface objects according to an embodiment of the present invention. Figure 3 As shown, the abnormal road surface object recognition device 300 may include: an acquisition unit 302 , a first recognition unit 304 , a second recognition unit 306 and an output unit 308 .

[0166] The acquisition unit 302 is used to acquire the attribute information of the candidate object set contained in the target image.

[0167] The first identification unit 304 is used to identify a first abnormal road surface object set from the candidate object set based on the attribute information.

[0168] The second identification unit 306 is used to identify a second abnormal road surface object set from the first abnormal road surface object set based on the position information of the abnormal road surface objects in the drivable area and the driving direction of the vehicle in the driving scene.

[0169] The output unit 308 is configured to output a second abnormal road surface object set in response to the road type of the road in the drivable area being the target road type.

[0170] In this embodiment, the acquisition unit 302 acquires the attribute information of the candidate object set contained in the target image; the first recognition unit 304 recognizes the first abnormal road surface object set from the candidate object set based on the attribute information; the second recognition unit 306 recognizes the second abnormal road surface object set from the first abnormal road surface object set based on the position information of the abnormal road surface objects in the drivable area and the driving direction of the vehicle in the driving scene; the output unit 308 outputs the second abnormal road surface object set in response to the road type of the road in the drivable area being the target road type. Thus, the technical effect of improving the recognition accuracy of abnormal road surface objects is achieved, and the technical problem of low recognition accuracy of abnormal road surface objects is solved.

[0171] According to another aspect of an embodiment of the present invention, an embodiment of the present application further provides a vehicle, comprising: a memory storing an executable program; and a processor for running the program, wherein the method of each embodiment of the present invention is executed when the program is running.

[0172] Figure 4 is a structural block diagram of an automatic driving vehicle according to an embodiment of the present invention. Figure 4 As shown, the components of the autonomous driving vehicle 400 include but are not limited to a memory 410 and a processor 420. The processor 420 and the memory 410 are connected via a bus 430, and a database 460 is used to store data.

[0173] The autonomous vehicle 400 may also include an access device 440 that enables the autonomous vehicle 400 to communicate via one or more networks 450. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 440 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and the like.

[0174] In one embodiment of the present disclosure, the above components of the autonomous driving vehicle 400 and Figure 4 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 4 The structure block diagram of the autonomous driving vehicle shown is for illustrative purposes only and is not intended to limit the scope of the present disclosure. Those skilled in the art may add or replace other components as needed.

[0175] An embodiment of the present application further provides a computer-readable storage medium, which includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present invention.

[0176] An embodiment of the present application further provides a computer program product, including a computer program, which implements the methods in various embodiments of the present invention when executed by a processor.

[0177] An embodiment of the present application further provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0178] The embodiments of the present application further provide a computer program, which implements the methods in the above-mentioned embodiments of the present invention when executed by a processor.

[0179] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0180] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0181] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0182] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0183] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0184] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for identifying abnormal road objects, characterized in that: include: Acquire attribute information of a candidate object set contained in a target image, wherein the target image is an image collected during vehicle driving, and the attribute information is at least used to represent attributes of the candidate objects in the candidate object set in the driving scene; Based on the attribute information, a first abnormal road surface object set is identified from the candidate object set, wherein the abnormal road surface objects in the first abnormal road surface object set are located in a drivable area of ​​the vehicle in the driving scenario; Based on the position information of the abnormal road surface objects in the first abnormal road surface object set in the drivable area and the driving direction of the vehicle in the driving scenario, identifying a second abnormal road surface object set from the first abnormal road surface object set, wherein the abnormal road surface objects in the second abnormal road surface object set are located in the driving direction; In response to the road type of the road in the drivable area being a target road type, the second abnormal road surface object set is output, wherein the probability of abnormal road surface objects in the second abnormal road surface object set appearing on the road of the target road type is higher than a probability threshold.

2. The method according to claim 1, characterized in that The attribute information includes identification information, and the identification information is used to identify the candidate objects. Based on the attribute information, a first abnormal road surface object set is identified from the candidate object set, including: In response to the target image having a plurality of frames, identifying the identification information corresponding to the same candidate object in the plurality of consecutive frames of the target image, and obtaining a first recognition result; In response to the first recognition result being that the identification information of the same candidate object in a plurality of consecutive frames of the target image is different, adjusting the identification information to obtain a first adjustment result; In response to the first adjustment result that the identification information of the same candidate object in multiple consecutive frames of the target image is the same, or in response to the number of frames of the target image being a single frame, identifying the candidate objects corresponding to different identification information in the target image to obtain a second identification result, wherein the second identification result is at least used to indicate whether the candidate objects corresponding to different identification information have an overlapping relationship; Based on the second recognition result, the candidate objects in the candidate object set are screened to obtain the screened candidate object set; The first abnormal road surface object set is identified from the screened candidate object set.

3. The method according to claim 2, characterized in that The second recognition result includes a first sub-recognition result, a second sub-recognition result, and a third sub-recognition result. In response to the first adjustment result that the identification information of the same candidate object in multiple consecutive frames of the target image is the same, or in response to the number of frames of the target image being a single frame, the candidate objects corresponding to different identification information in the target image are recognized to obtain a second recognition result, including: In response to the first adjustment result that the identification information of the same candidate object in the target image of the consecutive multiple frames is the same, identifying the candidate objects with different identification information in the target image of the current frame in the consecutive multiple frames to obtain the first sub-recognition result; In response to the first sub-recognition result being that a first candidate object subset exists in the current frame, the candidate objects with different identification information in the target image of a target number of frames consecutively after the current frame are recognized to obtain a second sub-recognition result, wherein at least two of the candidate objects in the first candidate object subset have the overlapping relationship, and the overlapping rate between at least two of the candidate objects is greater than an overlapping rate threshold; In response to the target image having a single frame, the candidate objects with different identification information in the target image of the current frame in the single frame are identified to obtain the third sub-identification result.

4. The method according to claim 3, characterized in that Based on the second recognition result, the candidate objects in the candidate object set are screened to obtain the screened candidate object set, including: In response to the second sub-recognition result being that the first candidate object subset exists in the target image of the target number frames, or the third sub-recognition result being that the first candidate object subset exists in the target image, obtaining first distances between at least two of the candidate objects in the first candidate object subset and the vehicle respectively; The candidate objects in the first candidate object subset whose first distance is greater than a first distance threshold are deleted from the candidate object set to obtain the filtered candidate object set.

5. The method according to claim 2, characterized in that: Identifying the first abnormal road surface object set from the screened candidate object set includes: Based on the drivable area and the attribute information, identifying the candidate object in the filtered candidate object set in the target image of the current frame to obtain a third recognition result; In response to the third recognition result being that a second candidate object subset exists in the candidate object set after screening of the current frame, and there are multiple consecutive frames of the target image after the current frame, the candidate object set after screening is recognized in the target number of consecutive frames of the target image after the current frame to obtain a fourth recognition result, wherein the candidate objects in the second candidate object subset are located in an area outside the drivable area, and a proportion of the total area of ​​the candidate objects is greater than or equal to a proportion threshold; In response to the fourth recognition result being that the second candidate object subset exists in the candidate object set after screening of the target number of frames, or in response to the third recognition result being that the second candidate object subset exists in the current frame and the target image does not exist after the current frame, the second candidate object subset is deleted from the screened candidate object set to obtain the first abnormal road surface object set.

6. The method according to claim 1, characterized in that Get the attribute information of the candidate object set contained in the target image, including: Using a target recognition model, the target image is recognized to obtain an initial candidate object set in the target image and initial attribute information corresponding to the initial candidate object set, wherein the target recognition model is obtained by training a deep learning model using image samples and attribute information samples, the image content of the image samples includes candidate object samples, and the attribute information samples are at least used to represent the attributes of the candidate object samples under the driving scene samples; Based on the initial attribute information, identifying the positional relationship between the candidate objects in the initial candidate object set to obtain a fifth recognition result, wherein the fifth recognition result is used to indicate whether there is a target positional relationship between at least two of the candidate objects; Based on the fifth recognition result, the candidate object set and the attribute information are determined.

7. The method according to claim 6, characterized in that Determining the candidate object set and the attribute information based on the fifth recognition result includes: In response to the fifth recognition result that a third candidate object subset exists in the initial candidate object set, performing aggregation processing on the candidate objects in the third candidate object subset to obtain a candidate object aggregation result, and performing aggregation processing on the initial attribute information of the third candidate object subset to obtain an attribute information aggregation result, wherein the third candidate object subset includes at least two candidate objects, at least two of the candidate objects have a target position relationship, and a second distance between at least two of the candidate objects is less than a second distance threshold; The candidate object aggregation result and the candidate objects in the initial candidate object set except the third candidate object subset are determined as the candidate object set, and the attribute information aggregation result and the initial attribute information corresponding to the candidate objects in the initial candidate object set except the third candidate object subset are determined as the attribute information.

8. The method according to claim 6, characterized in that Determining the candidate object set and the attribute information based on the fifth recognition result includes: In response to the fifth recognition result that the third candidate object subset does not exist in the initial candidate object set, the initial candidate object set is determined as the candidate object set, and the initial attribute information is determined as the attribute information, wherein the third candidate object subset includes at least two of the candidate objects, at least two of the candidate objects have a target position relationship, and the second distance between at least two of the candidate objects is less than a second distance threshold.

9. The method according to claim 1, characterized in that: Based on the position information of the abnormal road surface objects in the first abnormal road surface object set in the drivable area and the driving direction of the vehicle in the driving scene, identifying a second abnormal road surface object set from the first abnormal road surface object set includes: Based on the position information and the driving direction, identifying the first abnormal road surface object set in the target image of the current frame to obtain a sixth identification result; In response to the sixth recognition result that there is a first abnormal road surface object in the target image of the current frame, and there are multiple consecutive frames of the target image after the current frame, the first abnormal road surface object set is recognized in a target number of consecutive frames of the target image after the current frame to obtain a seventh recognition result, wherein the first abnormal road surface object is an abnormal road surface object in the first abnormal road surface object set that is not in the driving direction; In response to the seventh recognition result being that the first abnormal road object exists in the target image of the target number of frames, or in response to the sixth recognition result being that the first abnormal road object exists in the target image of the current frame and the target image does not exist after the current frame, the first abnormal road object is deleted from the first abnormal road object set to obtain the second abnormal road object set.

10. The method according to claim 1, characterized in that The attribute information includes length information and width information, and the method further includes: In response to the road type being the target road type, acquiring the length information and the width information of the abnormal road surface object in the second abnormal road surface object set; In response to the length information not satisfying a length information threshold, adjusting the length information according to the length information threshold; and / or, in response to the width information not satisfying a width information threshold, adjusting the width information according to the width information threshold; Based on the adjusted length information and / or the adjusted width information, an abnormal road surface object model corresponding to the abnormal road surface object in the second abnormal road surface object set is formulated according to a target scaling ratio, wherein the abnormal road surface object model is used to prompt that the abnormal road surface object that affects the normal driving of the vehicle exists in the driving direction at a future moment; The abnormal road surface object model is rendered and displayed on a graphical user interface of the vehicle.

11. The method according to claim 10, characterized in that Rendering the abnormal road surface object model onto a graphical user interface of the vehicle includes: Based on the position information of the abnormal road surface object, determining a relative position between the abnormal road surface object and the vehicle; On the graphical user interface, rendering the abnormal road surface object model and the vehicle model corresponding to the vehicle according to the relative position; According to the speed of the vehicle, the abnormal road surface object model is controlled to move toward the vehicle model at the speed.

12. A device for identifying abnormal road objects, characterized in that: include: an acquisition unit, configured to acquire attribute information of a candidate object set contained in a target image, wherein the target image is an image collected during vehicle driving, and the attribute information is at least used to represent attributes of the candidate objects in the candidate object set in the driving scene; A first identification unit is used to identify a first abnormal road surface object set from the candidate object set based on the attribute information, wherein the abnormal road surface objects in the first abnormal road surface object set are located in a drivable area of ​​the vehicle in the driving scenario; a second identification unit, configured to identify a second abnormal road surface object set from the first abnormal road surface object set based on position information of abnormal road surface objects in the drivable area in the first abnormal road surface object set and a driving direction of the vehicle in the driving scenario, wherein the abnormal road surface objects in the second abnormal road surface object set are located in the driving direction; An output unit is used to output the second abnormal road surface object set in response to the road type of the road in the drivable area being the target road type, wherein the probability of abnormal road surface objects in the second abnormal road surface object set appearing on the road of the target road type is higher than a probability threshold.

13. A vehicle, characterized in that: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 11 when running.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the storage medium is located is controlled to execute the method according to any one of claims 1 to 11.

15. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 11.

Images