Automatic driving assistance method and device in intersection pavement fault scene and vehicle

Abstract

The embodiment of the invention provides an automatic driving assistance method and device in an intersection pavement fault scene, a medium and a vehicle, and the method comprises the steps: collecting a vehicle-mounted image, and extracting shape elements and / or character elements in the vehicle-mounted image; screening the obtained shape elements and / or character elements according to the degree of importance of information to determine key elements, searching a first category corresponding to the key elements in the Internet of Vehicles, screening the first category by calculating the similarity to determine a target category, and obtaining road condition information according to the target category by adopting a deep neural network; the driving data and / or the road announcement obtained from the Internet of Vehicles and the road condition information are verified to obtain a verification result, when the verification result is consistent with the road condition information, a decision command is generated according to the road condition information, and the corresponding vehicle plans a driving track according to the decision command; wherein the decision command comprises lane changing in the current direction and retrograde lane borrowing. The problem that a safe and accurate driving instruction cannot be given in an existing automatic driving assistance system is solved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to an autonomous driving assistance method, device, medium, and vehicle for road surface fault scenarios at intersections. Background Technology

[0002] Autonomous driving methods refer to the driving methods of unmanned intelligent vehicles through computer systems. These methods enable the vehicle to drive automatically, park automatically, open and close doors automatically, and generate a driving route from the current location to the input destination, and then drive according to the generated route.

[0003] Currently, autonomous driving methods can effectively ensure the safety and stability of vehicles on roads with clear lanes, large lanes, and smooth surfaces. However, when lane lines disappear at intersections, road conditions become more complex, or there are road damage or construction issues, existing autonomous driving methods struggle to provide safe and accurate driving instructions, failing to meet people's safety requirements and driving needs for autonomous vehicles.

[0004] Therefore, there is an urgent need to propose an autonomous driving assistance method that can accurately identify road conditions and replan driving routes in complex road conditions, so as to adapt to different road conditions and ensure driving safety. Summary of the Invention

[0005] To overcome the problems existing in related technologies, this disclosure provides an autonomous driving assistance method, device, medium, and vehicle for intersection road surface fault scenarios to solve the technical problems of reduced safety and reduced accuracy of driving commands of autonomous vehicles caused by deteriorating road conditions in related technologies.

[0006] This specification provides one or more embodiments of an autonomous driving assistance method for a road surface fault scenario at an intersection, including the following steps:

[0007] The vehicle-mounted images captured during driving are collected, and shape elements and / or text elements are extracted from the vehicle-mounted images. The obtained shape elements and / or text elements are filtered and key elements are determined according to the importance of the information.

[0008] In the Internet of Vehicles, the first category corresponding to the key element is searched, and the target category is determined by filtering the first category through similarity calculation. A deep neural network is used to obtain road condition information based on the target category. The road condition information includes: road segment name and road segment condition.

[0009] The driving data and / or road announcements obtained from the vehicle network are verified with the road condition information to obtain a verification result. When the verification result matches the road condition information, a decision command is generated based on the road condition information, and the corresponding vehicle plans its driving trajectory according to the decision command.

[0010] The decision commands include lane changing in the same direction and lane changing in the opposite direction.

[0011] Optionally, the step of finding the first category corresponding to the key elements in the Internet of Vehicles and filtering the first category by calculating similarity to determine the target category includes: outlining each key element in the vehicle image, and finding the category corresponding to each outline in the Internet of Vehicles based on each outline to obtain a complete category map;

[0012] The target categories in the complete category diagram are filtered using priority relationships.

[0013] Optionally, the step of using a deep neural network to obtain road condition information based on the target category includes:

[0014] The target categories are sorted according to their priority to obtain a judgment sequence list;

[0015] Extract key information from the target category, and combine the extracted key information with the judgment sequence table to obtain a road condition analysis table;

[0016] Traffic information is obtained by searching and judging based on the semantic information in the traffic analysis table.

[0017] Optionally, the verification result obtained by verifying the driving data and road notices acquired from the vehicle network includes:

[0018] The vehicle obtains driving data and road condition information fed back to the vehicle network from the vehicle network, and determines the current location of the vehicle based on the driving data and road condition information fed back by the other vehicles.

[0019] Based on the vehicle's current location, search for the corresponding road announcement in the vehicle network, and filter the road announcement to obtain the road segment name and road segment information;

[0020] Verify whether the selected road segment conditions and road condition information are consistent with the road condition information to obtain the verification results.

[0021] Optionally, the decision command further includes:

[0022] The driving trajectories of other vehicles within a preset distance are obtained based on the vehicle-mounted image. The lane condition of the current lane is determined based on the driving trajectories of the other vehicles, and a decision command is generated.

[0023] Optionally, the method further includes the step of uploading the road condition results and the driving trajectory to the vehicle network.

[0024] This specification provides one or more embodiments of an autonomous driving assistance device for a road surface fault scenario at an intersection, including:

[0025] The element extraction module is used to collect vehicle images taken during driving, extract shape elements and / or text elements from the vehicle images, and filter and determine key elements based on the importance of the acquired shape elements and / or text elements.

[0026] The traffic condition acquisition module is used to find the first category corresponding to the key element in the vehicle network, filter the first category by calculating similarity to determine the target category, and obtain traffic condition information from the target category using a deep neural network. The traffic condition information includes: road segment name and road segment condition.

[0027] The decision generation module is used to verify the driving data and / or road announcements obtained from the vehicle network with the road condition information to obtain a verification result. When the verification result matches the road condition information, a decision command is generated based on the road condition information, and the corresponding vehicle plans a driving trajectory according to the decision command.

[0028] The decision commands include lane changing in the same direction and lane changing in the opposite direction.

[0029] Optionally, the road condition acquisition module is used to search for a first category corresponding to the key elements in the vehicle network, and to determine the target category by calculating similarity to filter the first category. Specifically, it is configured as follows:

[0030] Each key element is outlined in the in-vehicle image. Based on each outline, the category corresponding to each outline is found in the Internet of Vehicles to obtain a complete category diagram.

[0031] The target categories in the complete category diagram are filtered using priority relationships.

[0032] This specification provides one or more embodiments of a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described autonomous driving assistance method.

[0033] This specification provides one or more embodiments of a vehicle that stores a set of instructions, which are executed by the vehicle to implement the above-described method for assisting autonomous driving in the event of road surface malfunctions at intersections.

[0034] This includes any of the aforementioned intersection road surface fault scenarios for autonomous driving assistance devices.

[0035] The present invention provides an autonomous driving assistance method, device, and vehicle for road surface fault scenarios at intersections. Its advantages lie in that by collecting onboard images captured during vehicle operation, extracting key elements from these images, and obtaining real-time road conditions, the system can identify target categories from the vehicle network corresponding to each key element, obtain road condition results, and index and filter the obtained road condition information to obtain real-time road condition results. This allows for accurate acquisition and analysis of road conditions, which are then compared with road announcements and driving data obtained from the vehicle network to determine the current road conditions. The autonomous driving assistance system can then judge and verify the road results, providing guidance for driving planning. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 A flowchart illustrating an autonomous driving assistance method for a road surface fault scenario at an intersection, provided for one or more embodiments of this specification;

[0038] Figure 2 A schematic diagram of an autonomous driving assistance device for a road surface fault scenario at an intersection, provided for one or more embodiments of this specification;

[0039] Figure 3 This is a schematic block diagram of the structure of a vehicle provided for one or more embodiments of this specification. Detailed Implementation

[0040] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this invention.

[0041] The present invention will now be described in detail with reference to specific embodiments and accompanying drawings.

[0042] Method Implementation Examples

[0043] Because current autonomous driving technology is relatively mature on roads with good conditions, it can quickly determine and issue driving instructions. However, when road conditions deteriorate, existing autonomous driving systems struggle to provide accurate driving instructions to ensure driving safety. Therefore, this invention provides an autonomous driving assistance method for intersection road surface fault scenarios, applicable to intersections with complex road conditions and road faults, such as... Figure 1 As shown in the flowchart of an embodiment of the present invention, the method includes:

[0044] Step 11: Collect in-vehicle images taken during driving, extract shape elements and / or text elements from the in-vehicle images, and filter and determine key elements based on the importance of the acquired shape elements and / or text elements.

[0045] Step 12: In the vehicle network, find the first category corresponding to the shape element and / or text element, filter the first category by calculating similarity to obtain the target category, and use a deep neural network to obtain road condition information based on the target category. The road condition information includes: road segment name and road segment conditions. The road segment conditions specifically include time-limited traffic restrictions, lane closures, detours, traffic control, and guidance.

[0046] In this embodiment, the first category is filtered by calculating similarity to obtain the target category. Specifically, the similarity coefficient of the categories related to road condition information in the first category is calculated by using a similarity formula. The target category is determined based on whether the similarity coefficient is close to the threshold. The specific similarity calculation formula includes cosine similarity, Euclidean distance, etc.

[0047] The method of using deep neural networks to learn road condition information can be achieved by training a corresponding relational model to realize the relationship between the target category and the road condition information. The relational model can also be validated through a validation set. The corresponding road condition information can be obtained through the relational model. This method is existing technology and will not be elaborated on here.

[0048] Step 13: Verify the driving data and / or road announcements obtained from the vehicle network with the road condition information to obtain a verification result. When the verification result matches the road condition information, generate a decision command based on the road condition information, and the corresponding vehicle plans its driving trajectory according to the decision command.

[0049] The decision commands include lane changing in the same direction and lane changing in the opposite direction.

[0050] In addition, driving data includes road segment information from road navigation, such as road segment name, whether the road segment is congested, whether the road segment is blocked, whether it is under traffic control, whether lane changes are required, etc. Road announcements are generally text announcements, such as notices that a certain road segment in a certain area needs to be closed or restricted at a certain time for a certain reason, and that lane changes or detours are required. By comparing the actual confirmed road condition information with the information in the driving data and the information in the road announcements, the corresponding road condition information can be obtained.

[0051] In this embodiment of the invention, by acquiring in-vehicle images captured during vehicle operation and extracting key elements from these images, real-time information on road conditions, lane information, and the driving status of other vehicles can be obtained. This allows for information interaction with all vehicles that have opted for autonomous driving. The system indexes categories corresponding to these key elements within the vehicle network, filters target categories within those categories, and obtains road condition results from these target categories. The obtained road condition information is then indexed and filtered to obtain real-time road condition results, enabling accurate acquisition and analysis of road conditions. Furthermore, the system obtains road notices and driving data through the vehicle network and integrates these road notices with… The driving data is verified to obtain a verification result. When the verification result matches the road condition result, the road condition result is used as a decision command to intercept or guide the road at the intersection ahead. The obtained road condition result is compared with the information in the vehicle network to determine the real situation of the current road. This automatic assisted driving system can judge and verify the road result, provide guidance for driving planning, plan the driving trajectory according to the decision command, and upload the decision command to the vehicle network. The system plans the driving trajectory according to the decision command obtained from the road condition result, so that the vehicle can accurately select the driving trajectory to improve the safety of autonomous driving, and upload the result to the vehicle network for other vehicles to refer to.

[0052] In this embodiment, the generation of decision commands based on the road condition information can be achieved using existing technologies, such as searching through a table to find the corresponding decision command once the road condition information is known.

[0053] In one embodiment, in step 11, in-vehicle images captured by the vehicle during driving are collected, and key elements in the in-vehicle images are extracted. The key elements include text elements and graphic elements, and there are categories in the Internet of Vehicles that correspond one-to-one with the key elements.

[0054] Among them, vehicle images can be obtained through vehicle-mounted cameras, monitoring of the current road section, and other devices that can capture images. There are no specific restrictions. The key elements in vehicle images mainly include various elements such as text elements and graphic elements.

[0055] Textual elements include various language symbols, and graphic elements include flowers, grass, trees, guardrails, various signs, obstacles, and vehicles, etc. In the Internet of Vehicles, there are categories that correspond one-to-one with the aforementioned key elements. The corresponding categories in the Internet of Vehicles can be indexed through textual elements and / or graphic elements. Categories also include general terms, such as various language symbols belonging to the text category, flowers, grass, and trees belonging to the vegetation category, and various vehicles belonging to the vehicle category, etc.

[0056] In one embodiment, step 12, which involves finding the first category corresponding to the shape and text elements in the vehicle network and filtering the first category by calculating similarity to obtain the target category, includes: drawing outlines of the text and shape elements in the vehicle image, finding the category corresponding to each outline in the vehicle network to obtain a complete category map; and filtering the target category in the complete category map using a priority relationship.

[0057] In the Internet of Vehicles (IoV), there are categories that correspond one-to-one with the aforementioned key elements. These categories can be indexed using textual and / or graphical elements. Specifically, firstly, through a category association step, visual automatic tracking is used to draw the corresponding category outlines in the vehicle image based on each key element, either on an object basis or on the boundary of an object. For example, if the vehicle image includes the sky, road surface, guardrails, vegetation, and obstacles, then these key elements are used as categories. That is, the area where each category is located is outlined with a frame. Then, each indexed category is mapped to the frame to obtain a complete category map. For example, category A is marked in area A, category B is marked in area B, and so on.

[0058] Secondly, through the category ignoring step, the target categories in the complete category map are obtained based on priority relationships. Specifically, the complete category map contains several categories. For example, from the analysis of road conditions, it can be seen that road surface images, obstacle images, and guardrail conditions are all analyzable objects. Considering priority, factors such as vegetation and sky can be ignored. Therefore, the interfering categories are vegetation and sky. Interfering categories are excluded by priority relationships. The interfering categories in the complete category map are ignored and filtered to obtain only the analyzable target categories. Target categories include, for example, obstacles, guide signs, lane guardrails, and driving vehicles, etc. Priority relationships include, for example, whether there are obstruction signs on the road surface as the first priority, the road surface image as the second priority, and the road fence as the third priority, etc.

[0059] In this embodiment of the invention, a complete category diagram is formed by drawing the category diagrams corresponding to each key element, thus forming complete road information. A filtering strategy is used to filter out interfering categories in the complete category diagram to obtain the target category. This can eliminate interference by removing irrelevant interfering factors and other misleading categories in the image, so as to avoid excessive processing and confusion in subsequent category analysis, and ensure the high accuracy of the road condition information provided.

[0060] In one embodiment, step 12 further includes obtaining traffic information based on the target category using a deep neural network, which includes: sorting the target categories according to priority to obtain a judgment sequence list; extracting key information from the target categories; combining the extracted key information with the judgment sequence list to obtain a traffic analysis table; and performing a search and judgment based on the semantic information of the traffic analysis table to obtain traffic information.

[0061] In this embodiment, a learning strategy is used to obtain traffic conditions from the target category. The learning strategy includes a priority ranking step, a keyword extraction step, and a traffic condition feedback step.

[0062] In the priority sorting step, the target categories are sorted according to the priority relationship to obtain the judgment sequence list. The target categories in the target category map are sorted according to the priority relationship to obtain the judgment sequence list. The judgment sequence list includes A-obstacle-1, B-sign-2, C-guardrail-3, D-road surface-5, and E-vehicle-6.

[0063] In the keyword extraction step, keywords from the target category are extracted sequentially according to the judgment sequence list. The keywords are combined with the judgment sequence list to obtain a road condition analysis table. The keywords include roadblock icons, guide arrows, and directional text. Assuming that the road condition analysis table for the first case includes Class A - Obstacles - 1 - Prohibition icon, Class B - Sign - 2 - Guide arrow and prohibition text, Class C - Guardrail - 3, Class D - Road surface - 5, and Class E - Driving vehicle - 6. Assuming that the road condition analysis table for the second case includes Class A - Obstacles - 1 - Guide arrow, Class B - Sign - 2 - Guide arrow, Class C - Guardrail - 3, Class D - Road surface - 5, and Class E - Driving vehicle - 6.

[0064] In the traffic feedback step, the traffic results are determined according to the traffic analysis table in priority order. When the traffic analysis table obtained in the traffic feedback step is the first case, the traffic result is the result of traveling in the wrong direction and using the wrong lane. When the traffic analysis table obtained in the traffic feedback step is the second case, the traffic result is the result of changing lanes in the same direction.

[0065] In this embodiment of the invention, the learning strategy in the road condition judgment module can sort the hit categories for priority analysis, thereby improving the efficiency of the analysis. The road condition analysis table is obtained by combining keywords with the judgment sequence list, which can comprehensively obtain road condition information. The road condition results are obtained by judging the road condition analysis table in priority order. The target categories are sorted for priority analysis, thereby improving the efficiency of the analysis and obtaining comprehensive and detailed road condition results. This makes it easier for the autonomous driving assistance system to guarantee safe and accurate driving instructions.

[0066] In this embodiment, step 13, which verifies the driving data and road notices obtained from the vehicle network to obtain the verification result, includes the following steps:

[0067] Step 131: Obtain driving data and road condition information fed back to the vehicle network from the vehicle network, and determine the current location of the vehicle based on the driving data and road condition information fed back by other vehicles. In this embodiment, the driving data for information verification is not only the current location of the vehicle, but also some information in the navigation. Because if vehicle A has already driven through the road segment, it will feed the information back to the network. The driving data includes the feedback information of vehicle A. The driving data and road notices are used for verification.

[0068] Step 132: Based on the vehicle's current location, search for the corresponding road announcement in the vehicle network, filter the road announcement to obtain the road segment name and road segment information; verify whether the filtered road segment information and road segment information are consistent with the road condition information, and obtain the verification result.

[0069] The Internet of Vehicles (IoV) includes real-time road announcements issued by road administration and real-time driving data recorded by navigation systems. The verification results are obtained by combining these road announcements and driving data within the IoV and applying a verification strategy. This strategy includes announcement analysis and road condition comparison steps.

[0070] In the announcement analysis step, the vehicle can obtain its current location in real time while driving. Then, based on the current location, the vehicle network indexes the corresponding road announcements released in real time by the road administration, and filters the obtained road announcements to obtain the road segment name and road segment information.

[0071] In the road condition comparison step, the road segment name and road segment conditions are compared with the road condition results. If they match, the verification is successful. Otherwise, due to delays in road announcements, navigation, or network, the verification is more likely to fail. Therefore, the road condition judgment and result verification are repeated at different times until the verification result is the same as the road condition result. Only when the verification result is exactly the same as the road condition result will the vehicle use the road condition result as the decision command.

[0072] In this embodiment, the verification strategy in the result verification module can connect vehicle networks to each other to obtain road announcements from the vehicle networks for separate verification, thereby ensuring that the road condition judgment results of the road condition judgment module are more accurate and improving the safety of vehicle assisted driving.

[0073] In this embodiment, the decision command further includes: obtaining the driving trajectories of other vehicles within a preset distance based on the vehicle image, determining the lane condition of the current lane based on the driving trajectories of the other vehicles, and generating a decision command.

[0074] In the scene prediction step, the driving status information of other vehicles in the current lane is judged based on the vehicle image. The lane condition of the current lane is predicted based on the driving status information as the verification result. Assuming that the vehicle in the adjacent lane decelerates, and it can be determined that the adjacent vehicle has the purpose of traveling in the same direction as the current vehicle, if the adjacent vehicle is traveling in the opposite direction to one side, the adjacent vehicle is used as the basis for decision command. If the adjacent vehicle is driving normally, the driving situation of the vehicles behind is judged. This decision command method is mainly based on the vehicle in front. If there are other vehicles in front of the vehicle, the vehicles in front can be used as the basis for decision command.

[0075] In this embodiment of the invention, the driving status of other vehicles in the current lane and the driving status of vehicles in adjacent lanes can be used as the basis for decision commands, providing more reference for road condition judgment, making the road condition results more accurate, improving the safety of vehicle assisted driving, and providing accurate driving trajectories. For example, if the trajectory of the preceding vehicle A is used, the vehicle can directly use the trajectory of vehicle A to verify the result of its judgment. If the adjacent vehicle B is driving entirely in the current lane, it can output that the current lane is normal. If the adjacent vehicle B changes lanes entirely to the opposite lane, it can be judged that the current lane is not safe to drive in, and it is necessary to learn the trajectory of the adjacent vehicle.

[0076] In this embodiment, in order to promptly synchronize the current road condition information with other autonomous vehicles that need to pass through the current road segment, and to avoid causing inconvenience to other vehicles, the autonomous driving assistance method further includes the following steps:

[0077] Step 14: Upload the road condition results and the driving trajectory to the vehicle network.

[0078] Device Examples

[0079] According to embodiments of the present invention, an autonomous driving assistance device for road surface fault scenarios at intersections is provided, applicable to scenarios where vehicles are traveling at intersections, such as... Figure 2 The diagram shown is a structural schematic of an embodiment of the present invention. The device includes:

[0080] The element extraction module 21 is used to collect vehicle images taken during driving, extract shape elements and / or text elements from the vehicle images, and filter and determine key elements based on the importance of the acquired shape elements and / or text elements.

[0081] The traffic condition acquisition module 22 is used to search for a first category corresponding to the key element in the vehicle network, filter the first category by calculating similarity to determine the target category, and use a deep neural network to obtain traffic condition information based on the target category. The traffic condition information includes: road segment name and road segment conditions.

[0082] The decision generation module 23 is used to verify the driving data and / or road announcements obtained from the vehicle network with the road condition information to obtain a verification result. When the verification result is consistent with the road condition information, a decision command is generated according to the road condition information, and the corresponding vehicle plans the driving trajectory according to the decision command.

[0083] The decision commands include lane changing in the same direction and lane changing in the opposite direction.

[0084] In this embodiment of the invention, by acquiring in-vehicle images captured during vehicle operation and extracting key elements from these images, real-time information on road conditions, lane information, and the driving status of other vehicles can be obtained. This allows for information interaction with all vehicles that have opted for autonomous driving. The system indexes categories corresponding to these key elements within the vehicle network, filters target categories within those categories, and obtains road condition results from these target categories. The obtained road condition information is then indexed and filtered to obtain real-time road condition results, enabling accurate acquisition and analysis of road conditions. Furthermore, the system obtains road notices and driving data through the vehicle network and integrates these road notices with… The driving data is verified to obtain a verification result. When the verification result matches the road condition result, the road condition result is used as a decision command to intercept or guide the road at the intersection ahead. The obtained road condition result is compared with the information in the vehicle network to determine the real situation of the current road. This automatic assisted driving system can judge and verify the road result, provide guidance for driving planning, plan the driving trajectory according to the decision command, and upload the decision command to the vehicle network. The system plans the driving trajectory according to the decision command obtained from the road condition result, so that the vehicle can accurately select the driving trajectory to improve the safety of autonomous driving, and upload the result to the vehicle network for other vehicles to refer to.

[0085] In this embodiment, the road condition acquisition module 22 is used to search for a first category corresponding to key elements in the vehicle network, and to determine the target category by calculating similarity to filter the first category. Specifically, it is configured as follows:

[0086] Draw the key elements in the vehicle image, and find the category corresponding to each frame in the Internet of Vehicles based on each frame to obtain a complete category map;

[0087] The target categories in the complete category diagram are filtered using priority relationships.

[0088] The system collects in-vehicle images captured during vehicle operation and extracts key elements from these images. These key elements include textual and graphic elements, and each key element has a corresponding category in the Internet of Vehicles (IoV).

[0089] The vehicle images can be acquired through vehicle-mounted cameras, surveillance cameras on the current road segment, or other devices capable of capturing images, without any specific limitations. The key elements in the vehicle images mainly include various elements such as text elements and graphic elements. Among them, text elements include various language symbols, and graphic elements include flowers, grass, trees, guardrails, various directional signs, obstacles, and vehicles, etc. In the Internet of Vehicles (IoV), there are categories that correspond one-to-one with the key elements. The corresponding categories in the IoV can be indexed through text elements and / or graphic elements. Categories also include general terms, such as various language symbols belonging to the text category, flowers, grass, and trees belonging to the vegetation category, and various vehicles belonging to the vehicle category, etc.

[0090] In the Internet of Vehicles (IoV), there are categories that correspond one-to-one with the aforementioned key elements. These categories can be indexed using textual and / or graphical elements. Specifically, firstly, through a category association step, the corresponding category outlines are drawn in the vehicle image based on each key element. For example, if the vehicle image includes the sky, road surface, guardrails, vegetation, and obstacles, then the categories are defined based on these key elements. That is, the area containing each category is outlined with a frame, and then each indexed category is mapped to the frame to obtain a complete category map. For example, category A is marked in area A, category B is marked in area B, and so on.

[0091] Secondly, through the category ignoring step, the target categories in the complete category map are obtained according to the priority relationship. Specifically, the complete category map contains several categories. For example, from the analysis of road conditions, it can be seen that road surface images, obstacle images, and guardrail conditions are all analyzable objects. Considering the priority, factors such as vegetation and sky can be ignored. Therefore, the interfering categories are vegetation and sky. Interfering categories are excluded by priority relationship. The interfering categories in the complete category map are ignored and filtered to obtain only the analyzable target categories. The target categories mainly include obstacles, guide signs, lane guardrails, and driving vehicles.

[0092] In this embodiment of the invention, a complete category diagram is formed by drawing the category diagrams corresponding to each key element, thus forming complete road information. A filtering strategy is used to filter out interfering categories in the complete category diagram to obtain the target category. This can eliminate interference by removing irrelevant interfering factors and other misleading categories in the image, so as to avoid excessive processing and confusion in subsequent category analysis, and ensure the high accuracy of the road condition information provided.

[0093] This specification provides one or more embodiments of a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described autonomous driving assistance method.

[0094] Exemplary vehicle

[0095] A block diagram of a vehicle 800 is shown according to an exemplary embodiment. The vehicle 800 may be a gasoline vehicle, a hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicles.

[0096] Reference Figure 3 The vehicle 800 may include multiple subsystems, such as a drive system 810, a control system 820, a sensing system 830, a communication system 840, an information display system 850, and a computing processing system 860. The vehicle 800 may also include more or fewer subsystems, and each subsystem may include multiple components, which will not be described in detail here.

[0097] The drive system 810 includes components that provide power to the vehicle 800. These include, for example, an engine, an energy source, and a transmission.

[0098] The control system 820 includes components that provide control for the vehicle 800. These include, for example, vehicle control, cockpit equipment control, and driver assistance control.

[0099] The perception system 830 includes components that provide the vehicle 800 with perception of its surroundings. These include, for example, a vehicle positioning system, a laser sensor, a voice sensor, an ultrasonic sensor, and camera equipment.

[0100] The communication system 840 includes components that provide communication connectivity for the vehicle 800. These may include, for example, mobile communication networks (e.g., 3G, 4G, 5G networks), WiFi, Bluetooth, and vehicle-to-everything (V2X) connectivity.

[0101] The information display system 850 includes components that provide various information displays for the vehicle 800. These include, for example, vehicle information displays, navigation information displays, and entertainment information displays.

[0102] The computing processing system 860 includes components that provide data computing and processing capabilities for the vehicle 800. The computing processing system 860 may include at least one processor 861 and a memory 862. The processor 861 can execute instructions stored in the memory 862.

[0103] The processor 861 can be any conventional processor, such as a commercially available CPU. The processor may also include, for example, a Graphics Processing Unit (GPU), a Field Programmable Gate Array (FPGA), a System on Chip (SOC), an Application Specific Integrated Circuit (ASIC), or a combination thereof.

[0104] The memory 862 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0105] In this embodiment of the present disclosure, a set of instructions is stored in the memory 862, and the processor 861 can execute the set of instructions to implement all or part of the steps of the autonomous driving assistance method in the intersection road surface fault scenario described in any of the exemplary embodiments above.

[0106] The embodiments of the present invention are system embodiments corresponding to the above method embodiments. The specific operations of each module processing step can be understood with reference to the description of the method embodiments, and will not be repeated here. Those skilled in the art will understand that all or part of the processes in the above embodiment methods can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, database, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0107] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for apparatus or system embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The apparatus and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and the contents not described in detail in the specification of the present invention are well known to those skilled in the art.

Claims

1. An autonomous driving assistance method for intersection road surface fault scenarios, characterized in that, Including the following steps: The vehicle-mounted images captured during driving are collected, and shape elements and / or text elements are extracted from the vehicle-mounted images. The obtained shape elements and / or text elements are filtered and key elements are determined according to the importance of the information. In the Internet of Vehicles, the first category corresponding to the key element is searched, and the target category is determined by filtering the first category through similarity calculation. A deep neural network is used to obtain road condition information based on the target category. The road condition information includes: road segment name and road segment condition. The driving data and / or road announcements obtained from the vehicle network are verified with the road condition information to obtain a verification result. When the verification result matches the road condition information, a decision command is generated based on the road condition information, and the corresponding vehicle plans its driving trajectory according to the decision command. The decision commands include lane changing in the same direction and lane changing in the opposite direction.

2. The autonomous driving assistance method for intersection road surface fault scenarios as described in claim 1, characterized in that, The step of finding the first category corresponding to the key elements in the Internet of Vehicles and determining the target category by filtering the first category through similarity calculation includes: outlining each key element in the vehicle image, and finding the category corresponding to each outline in the Internet of Vehicles based on each outline to obtain a complete category map; The target categories in the complete category diagram are filtered using priority relationships.

3. The autonomous driving assistance method for intersection road surface fault scenarios as described in claim 1, characterized in that, The step of using a deep neural network to obtain road condition information based on the target category includes: The target categories are sorted according to their priority to obtain a judgment sequence list; Extract key information from the target category, and combine the extracted key information with the judgment sequence table to obtain a road condition analysis table; Traffic information is obtained by searching and judging based on the semantic information in the traffic analysis table.

4. The autonomous driving assistance method for intersection road surface fault scenarios as described in claim 1, characterized in that, The verification results obtained by verifying the driving data and road notices acquired from the vehicle network include: The vehicle obtains driving data and road condition information fed back to the vehicle network from the vehicle network, and determines the current location of the vehicle based on the driving data and road condition information fed back by the other vehicles. Based on the vehicle's current location, search for the corresponding road announcement in the vehicle network, and filter the road announcement to obtain the road segment name and road segment information; Verify whether the selected road segment conditions and road condition information are consistent with the road condition information to obtain the verification results.

5. The autonomous driving assistance method for intersection road surface fault scenarios as described in claim 1, characterized in that, The decision command also includes: The driving trajectories of other vehicles within a preset distance are obtained based on the vehicle-mounted image. The lane condition of the current lane is determined based on the driving trajectories of the other vehicles, and a decision command is generated.

6. The autonomous driving assistance method for intersection road surface fault scenarios as described in claim 1, characterized in that, It also includes the following steps: The road condition results and the driving trajectory are uploaded to the vehicle network.

7. An automated driving assistance device for road surface fault scenarios at intersections, characterized in that, include: The element extraction module is used to collect vehicle images taken during driving, extract shape elements and / or text elements from the vehicle images, and filter and determine key elements based on the importance of the acquired shape elements and / or text elements. The traffic condition acquisition module is used to find the first category corresponding to the key element in the vehicle network, filter the first category by calculating similarity to determine the target category, and use a deep neural network to obtain traffic condition information based on the target category. The traffic condition information includes: road segment name and road segment condition. The decision generation module is used to verify the driving data and / or road announcements and road condition information obtained from the vehicle network to obtain the verification result. When the verification result is consistent with the road condition information, a decision command is generated according to the road condition information, and the corresponding vehicle plans the driving trajectory according to the decision command. The decision commands include lane changing in the same direction and lane changing in the opposite direction.

8. The automatic driving assistance device for intersection road surface fault scenarios as described in claim 7, characterized in that, The traffic condition acquisition module is used to find the first category corresponding to the key elements in the vehicle network, and to determine the target category by calculating the similarity of the first category. Specifically, it is configured as follows: Each key element is outlined in the in-vehicle image. Based on each outline, the category corresponding to each outline is found in the Internet of Vehicles to obtain a complete category diagram. The target categories in the complete category diagram are filtered using priority relationships.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of autonomous driving assistance in the intersection road surface fault scenario as described in any one of claims 1 to 6.

10. A vehicle, characterized in that, A set of instructions is stored, which is executed by the vehicle to implement the autonomous driving assistance method for intersection road surface failure scenarios as described in any one of claims 1 to 6; Including the autonomous driving assistance device for intersection road surface failure scenarios as described in any one of claims 7 to 8.

Images