Road indication sign rendering method and electronic equipment

Abstract

The invention discloses a road indication sign rendering method and electronic equipment. The method comprises the steps that in the vehicle driving process, at least one road image collected by a vehicle is acquired; identifying a road indication mark in the at least one road image to obtain an indication type and endpoint information of the road indication mark, the endpoint information being used for representing an indication direction of the road indication mark; adjusting a road indication mark in the at least one road image based on the indication type and the end point information to obtain a target road image; and rendering the road indication mark in the target road image to an interactive interface of the vehicle. According to the invention, the technical problem that the rendering effect of the road indication sign is poor in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the fields of autonomous driving and computer vision, and in particular to a method for rendering a road sign and an electronic device. Background Art

[0002] With the development of autonomous driving technology, the rendering technology of vehicle display screens plays a vital role in driving safety and experience. In particular, the rendering of road signs, such as traffic arrows, stop lines and crosswalk signs, is crucial to the driving safety of autonomous vehicles. However, in the prior art, directly rendering road signs in road images to vehicles will have the defects of rendering position offset, rendering not smooth or unstable rendering effect.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present invention provide a method for rendering a road sign and an electronic device, so as to at least solve the technical problem of poor rendering effect of a road sign in the related art.

[0005] According to one aspect of an embodiment of the present invention, a method for rendering road signs is provided, comprising: acquiring at least one road image collected by a vehicle during driving of a vehicle; identifying a road sign in at least one road image to obtain an indication type and endpoint information of the road sign, wherein the endpoint information is used to indicate a direction of the road sign; adjusting the road sign in at least one road image based on the indication type and endpoint information to obtain a target road image; and rendering the road sign in the target road image onto an interactive interface of the vehicle.

[0006] Furthermore, the road sign in at least one road image is adjusted based on the indication type and endpoint information to obtain a target road image, including: adjusting the road sign in at least one road image based on the indication type and endpoint information to obtain an initial road image; adjusting the road sign based on the lane lines in the initial road image to obtain a target road image.

[0007] Furthermore, a road sign in at least one road image is identified to obtain an indication type and endpoint information of the road sign, including: identifying a road sign in at least one road image to obtain a first area and a second area of ​​the road sign, wherein the first area is used to indicate an area where a starting point indicating the direction of travel of the vehicle is located, and the second area is used to indicate an area where an end point indicating the direction of travel is located; and determining the endpoint information and indication type based on the first area and the second area.

[0008] Furthermore, endpoint information and indication type are determined based on the first area and the second area, including: determining at least one first endpoint corresponding to the starting point in the first area; determining a target line segment corresponding to the end point in the second area, and determining the midpoint of the target line segment as the second endpoint; determining endpoint information based on at least one first endpoint and a second endpoint, and determining the indication type based on a line connecting at least one first endpoint and the second endpoint.

[0009] Furthermore, the road sign in at least one road image is adjusted based on the indication type and endpoint information to obtain an initial road image, including: in the case where there are multiple road sign signs, deduplication processing is performed on the first road sign sign in the road sign based on the indication type and endpoint information, and deletion processing is performed on the second road sign sign in the road sign sign to obtain a sign processing result, wherein the first road sign sign is used to indicate that repeated road sign signs appear in the road sign signs, and the second road sign sign is used to indicate that the road sign signs are within a preset area in the road sign signs; at least one road image is screened based on the sign processing result to obtain an initial road image.

[0010] Furthermore, at least one road image is screened based on the identification processing result to obtain an initial road image, including: in the case where there are multiple road images, state information of road signs in the multiple road images is determined based on the identification processing result, wherein the state information is used to indicate whether the display state of the road signs has changed; and at least one road image is screened based on the state information to obtain an initial road image.

[0011] Furthermore, the road signs are adjusted based on the lane lines in the initial road image to obtain the target road image, including: determining lane line position information of the lane lines and sign position information of the road signs based on the initial road image; determining a rectangular frame of the road sign based on the lane line position information and the sign position information; and adjusting the road signs in the initial road image based on the rectangular frame position information of the rectangular frame to obtain the target road image.

[0012] Furthermore, the road signs in the initial road image are adjusted based on the rectangular frame position information of the rectangular frame to obtain a target road image, including: adjusting the road signs in the initial road image based on the rectangular frame position information of the rectangular frame to obtain an adjusted road image, wherein the road signs in the adjusted road image are located within the rectangular frame; in response to the rectangular frame in the adjusted road image and the preset road signs in the adjusted road image being in an overlapping area, executing lane line position information and sign position information of the lane line determined based on the initial road image until there is no overlapping area between the rectangular frame in the adjusted road image and the preset road signs, and determining that the adjusted road image is the target road image.

[0013] Furthermore, rendering the road sign in the target road image onto the interactive interface of the vehicle includes: determining a mapping direction of the road sign based on a vertex order of the rectangular frame; and rendering the road sign onto the interactive interface based on the mapping direction.

[0014] According to another aspect of an embodiment of the present invention, a rendering device for road signs is also provided, including: an acquisition module, used to acquire at least one road image collected by a vehicle during driving of the vehicle; an identification module, used to identify the road signs in the at least one road image, and obtain the indication type and endpoint information of the road signs, wherein the endpoint information is used to indicate the indication direction of the road signs; an adjustment module, used to adjust the road signs in the at least one road image based on the indication type and endpoint information, and obtain a target road image; and a rendering module, used to render the road signs in the target road image onto an interactive interface of the vehicle.

[0015] According to another aspect of an embodiment of the present invention, there is further provided an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein the method in each embodiment of the present invention is executed when the program is running.

[0016] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present invention.

[0017] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0018] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0019] According to another aspect of the embodiments of the present invention, a computer program is further provided. When the computer program is executed by a processor, the methods in the embodiments of the present invention are implemented.

[0020] In an embodiment of the present application, a method for rendering a road sign is provided, including: obtaining at least one road image collected by the vehicle during vehicle driving; identifying the road sign in the road image to obtain the indication type and endpoint information; adjusting the road sign based on the indication type and endpoint information to obtain the target road image; and rendering the road sign in the target road image to the vehicle's interactive interface. It is easy to notice that the present application accurately identifies the indication type and endpoint information of the road sign in the road image, and further adjusts the road sign according to the indication type and endpoint information to ensure the rendering accuracy of the road sign, thereby accurately rendering the rendered road sign on the interactive interface, improving the rendering effect of the interactive interface, and facilitating the driver or the autonomous driving vehicle to understand the road conditions, adjust the driving direction in time, and avoid traffic accidents caused by entering the wrong lane, thereby solving the technical problem of poor rendering effect of road signs in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0022] Figure 1 is a flow chart of a method for rendering a road indication sign according to an embodiment of the present invention;

[0023] Figure 2 is a schematic diagram of a road sign rendering system according to an embodiment of the present invention;

[0024] Figure 3 It is a schematic diagram of a device for rendering a road sign according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] According to an embodiment of the present invention, an embodiment of a method for rendering a road sign is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0028] Figure 1 is a method for rendering a road sign according to an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0029] Step S102: acquiring at least one road image captured by the vehicle during the driving of the vehicle.

[0030] The above-mentioned vehicle may be a moving vehicle, which travels according to the path indicated by the navigation. The vehicle may be a vehicle assisted in driving by an automatic driving system, or a vehicle driven by a driver; the vehicle may be a vehicle using gasoline or diesel as a power source, or a vehicle using a battery or an electric motor as a power source, such as an electric car, an electric bicycle, etc. The type of vehicle is not limited here and can be determined according to actual needs. The above-mentioned vehicle is equipped with a display device, which may be one of the following: an instrument panel, a display screen, or a head-up display. It is not limited here and can be determined according to actual needs. The display function can be expanded by the above-mentioned display device. An interactive interface is provided on the display device for users to perform interactive operations, and a road image or road information in a road image is rendered on the interactive interface.

[0031] The above-mentioned road image can be used to characterize the image of the surrounding environment of the road on which the vehicle is traveling. The above-mentioned road image can be an image to be processed. The road image may include but is not limited to: road signs, lane lines, traffic signal signs, pedestrians, and road infrastructure, etc. There is no limitation here and it can be determined according to actual needs. In the field of autonomous driving, accurate and real-time road images are the basis for identifying road signs, so that road signs, obstacles, etc. can be effectively identified, so as to make safe driving decisions. Through road images, the surrounding road environment of the vehicle's driving road can be accurately judged, and then the road signs on the driving road can be accurately identified and displayed to the user. The acquisition of road images is an important step in realizing the recognition of road signs. The clarity, brightness, angle, etc. of the road image affect the subsequent recognition accuracy and efficiency.

[0032] In an optional embodiment, road images can be acquired through multiple cameras mounted on the vehicle. Road images of the road on which the vehicle is traveling are acquired in real time through multiple cameras. The cameras are mounted on the front, rear, sides and bottom of the vehicle to provide all-round field of view coverage. This multi-angle acquisition can capture road images and ensure that key road signs of the road image are recognized even in complex driving environments, such as multiple lanes and intersections. For example, images blocked by the vehicle in front or close to the ground can be captured by side or bottom cameras.

[0033] In another optional embodiment, the road image can also be obtained from a high-precision map. The high-precision map includes but is not limited to road images, obstacles, green belts, and road edges. This is not limited here and can be determined according to actual needs. Through the high-precision map, the road image in front of the vehicle can be obtained in advance, so as to process the road image and obtain road signs. With the help of high-precision map data, the vehicle's environmental perception ability is enhanced, and the stability of image acquisition and sign recognition is also improved.

[0034] Step S104: identifying a road sign in at least one road image to obtain the indication type and endpoint information of the road sign.

[0035] The endpoint information is used to indicate the direction indicated by the road sign.

[0036] The above-mentioned road sign can be used to indicate the direction of vehicle travel. The road sign may include but is not limited to: an indicator icon, an indicator type, endpoint information, etc. This is not limited here and can be determined according to actual needs. The indicator icon is the image information of the road sign, which can intuitively display the direction of guidance. The road sign can be obtained by recognizing the road image. Through accurate road signs, the direction of vehicle travel can be guided, and potential dangers such as sharp turns, downhill, construction areas, etc. can be warned, so that preventive measures can be taken to reduce the possibility of accidents and improve road use efficiency.

[0037] The indication type of the above-mentioned road sign can be used to characterize the type indicated by the road sign, and the indication type includes but is not limited to: arrow sign, warning sign, prohibition sign, etc. It is not limited here, and it can be determined according to actual needs. Through the indication type, the meaning of the road sign can be further judged. Optionally, the road sign can be compared with a preset sign library to determine the indication type of the road sign, or the indication type of the road sign can be determined by a first recognition model, and the first recognition model can be one of the following: support vector machine, random forest, logistic regression, neural network. It is not limited here, and it can be determined according to actual needs.

[0038] The endpoint information of the above-mentioned road sign can be used to indicate the indicated direction of the road sign. The indicated direction can be one of the following: go straight, turn left, turn right, etc., which is not limited here and can be determined according to actual needs. The endpoint information of the road sign can be used to indicate the direction of vehicle travel. The endpoint information of the road sign can be obtained by recognizing the road image. Optionally, the endpoint information of the road sign can be identified by a second recognition model, and the second recognition model can be one of the following: support vector machine, random forest, logistic regression, neural network. It is not limited here and can be determined according to actual needs.

[0039] In an optional embodiment, a road image can be identified through a neural network model to obtain a road sign, and then identify the indication type and endpoint information of the road sign. The neural network model can be one of the following: a convolutional neural network, a target detection model. This is not limited here and can be determined according to actual needs. Taking the convolutional neural network model as an example, the convolutional neural network model has been trained with a large number of annotated traffic sign image data sets, and can effectively identify different types of road signs and determine their positions in the image. Furthermore, through the edge detection algorithm, the edge of the road sign is detected, the endpoint information is determined, and then the direction indicated by the endpoint information is identified.

[0040] In another optional embodiment, the road sign can also be obtained through feature matching and pattern recognition. First, feature points are extracted from the road image, and then the feature points are matched with the road sign in the sign library using a feature matching algorithm. By calculating the similarity between the feature points, the indication type of the road sign in the road image can be determined. Furthermore, the shape of the road sign is analyzed through pattern recognition to obtain the endpoint information of the road sign, thereby determining the geometric features of the road sign, such as the top and bottom segments of the arrow, and determining its indication direction.

[0041] Step S106: adjusting the road indication mark in at least one road image based on the indication type and the endpoint information to obtain a target road image.

[0042] The above-mentioned target road image can be an image of the surrounding environment of the road on which the vehicle is traveling after adjustment. The target road image can be displayed on the interactive interface of the vehicle to prompt the user of the road signs on the road to facilitate accurate navigation. The target road image may include but is not limited to: road signs, lane lines, traffic signal signs, warning information, etc. There is no limitation here and it can be determined according to actual needs. In the field of autonomous driving, accurate and real-time target road images are key elements of assisted driving. The accurate display position and display stability of the target road image can ensure the delicacy of the display of road signs, so that users can accurately determine the road signs on the road, and then confirm the direction of travel, reducing the situation where drivers go the wrong way due to entering the wrong lane.

[0043] The above adjustment may refer to a process of deduplicating, deleting or locating a road sign in at least one road image, and the specific manner of the adjustment is not limited herein.

[0044] When at an intersection, there may be one or more identical road signs for the same traffic sign to remind the vehicle of the direction of travel. Alternatively, during driving, the road image will contain road signs for multiple lanes. At this time, the user pays more attention to the road sign of the current lane of the vehicle to ensure that it drives in accordance with traffic regulations. Therefore, displaying the road sign of the current lane on the vehicle interaction interface is more in line with user needs and avoids displaying redundant information. Therefore, it is necessary to adjust the identified road signs to obtain the target road image to display on the vehicle interaction interface, so that more accurate navigation instructions can be provided, especially in multi-lane or complex intersection environments. It is also possible to reduce the redundant information seen by the driver on the interaction interface by deduplicating and deleting the road signs, especially in dense sign areas, to improve the visual effect and information transmission, thereby improving the clarity of the interaction interface and the accuracy of navigation.

[0045] In an optional embodiment, the indication type and endpoint information of the road sign can be matched with a preset road sign set, which is a pre-stored collection of road sign signs on the driving road. By comparing the indication type and endpoint information of the road sign, it is determined whether there are multiple identical road sign signs, and one road sign sign is determined from the multiple identical road sign signs to reduce the redundancy of the target road image.

[0046] In another optional embodiment, the position of the road sign in the road image can be corrected to ensure that the display position and direction of the road sign are consistent with the display position and direction on the actual road. The image coordinates of the road sign can be converted into vehicle display coordinates through coordinate conversion, thereby adjusting the road sign to obtain the target road image displayed on the interactive interface.

[0047] Step S108: Rendering the road indication mark in the target road image onto the interactive interface of the vehicle.

[0048] The above-mentioned interactive interface can be an interface for users to interact with the vehicle. Through the interactive interface, the user can interact with the vehicle and display the road signs of the vehicle's driving road. Through the interactive interface, accurate road signs are provided to reduce the display redundancy of road images, improve driving safety and navigation efficiency.

[0049] In an optional embodiment, the road sign can be rendered to the interactive interface of the vehicle through a graphics rendering engine. The road sign can also be rendered through a hardware accelerator. Alternatively, the road sign can be rendered to the interactive interface of the vehicle by building a rendering pipeline. Furthermore, the resolution, refresh rate, color depth, and ambient light conditions of the vehicle display device can be adjusted through a rendering strategy to optimize the display effect, thereby ensuring the stability and clarity of the road sign display. Optionally, the rendering strategy can be one of the following: resolution scaling, color correction, motion blur, and anti-aliasing, which is not limited here and can be set according to requirements.

[0050] In an embodiment of the present application, a method for rendering a road sign is provided, including: obtaining at least one road image collected by the vehicle during vehicle driving; identifying the road sign in the road image to obtain the indication type and endpoint information; adjusting the road sign based on the indication type and endpoint information to obtain the target road image; and rendering the road sign in the target road image to the vehicle's interactive interface. It is easy to notice that the present application accurately identifies the indication type and endpoint information of the road sign in the road image, and further adjusts the road sign according to the indication type and endpoint information to ensure the rendering accuracy of the road sign, thereby accurately rendering the rendered road sign on the interactive interface, improving the rendering effect of the interactive interface, and facilitating the driver or the autonomous driving vehicle to understand the road conditions, adjust the driving direction in time, and avoid traffic accidents caused by entering the wrong lane, thereby solving the technical problem of poor rendering effect of road signs in related technologies.

[0051] Figure 2 is a schematic diagram of a rendering system for a road sign according to an embodiment of the present application, such as Figure 2 As shown, the rendering system includes an autonomous driving domain controller and a cockpit domain controller. The autonomous driving domain controller includes a perception module, a fusion module, and a correction module. The cockpit domain controller includes a rendering module.

[0052] The above-mentioned perception module can be used to collect at least one road image of the vehicle during the vehicle's driving process, and transmit the road image to the fusion module.

[0053] The above-mentioned fusion module can be used to receive the road image obtained by the perception module, and identify the road signs in the road image to obtain the indication type and endpoint information of the road sign, wherein the endpoint information is used to indicate the indication direction of the road sign, and then transmit the indication type and endpoint information to the correction module.

[0054] The above-mentioned correction module can be used to receive the indication type and endpoint information transmitted by the correction module, and further adjust the road indication signs in at least one road image based on the indication type and endpoint information to obtain the target road image, and then transmit the target road image to the rendering module.

[0055] The above-mentioned rendering module can be used to receive the target road image transmitted by the correction module, and render the road indication signs in the target road image onto the interactive interface of the vehicle.

[0056] It should be noted here that the examples and application scenarios implemented by the rendering system are the same as the rendering method of the above-mentioned road sign, but are not limited to the contents disclosed in the above-mentioned embodiments.

[0057] Furthermore, the road sign in at least one road image is adjusted based on the indication type and endpoint information to obtain a target road image, including: adjusting the road sign in at least one road image based on the indication type and endpoint information to obtain an initial road image; adjusting the road sign based on the lane lines in the initial road image to obtain a target road image.

[0058] The above-mentioned initial road image can be an image of the surrounding environment of the road on which the vehicle is traveling, obtained by adjusting the road signs in the road image based on the indication type and endpoint information. The initial road image may include but is not limited to: road signs, lane lines, traffic signal signs, warning information, etc. This is not limited here and can be determined according to actual needs. The road signs in the initial road image can be road signs for the lane where the vehicle is currently traveling, so as to remind the user to drive according to the road signs, confirm the driving direction, and avoid vehicle steering errors.

[0059] In an optional embodiment, the road sign can be first deduplicated and adjusted using the indication type and endpoint information, and then the road sign can be positioned and adjusted based on the lane lines to ensure that the road sign in the target road image matches the current direction of travel of the vehicle, especially in multi-lane or complex intersection environments, to provide more accurate navigation instructions. Optionally, redundant road signs can be deleted through deduplication adjustment to obtain an initial road image. The initial road image may have improvements in clarity, memory, etc., and can also reduce resource usage. Then, the road sign can be positioned and adjusted using the lane lines to obtain the target road image, thereby ensuring that the road sign in the target road image can be used to indicate the direction of travel of the vehicle's current lane, improving the quality and accuracy of the image, and making subsequent rendering tasks more accurate and reliable.

[0060] Furthermore, a road sign in at least one road image is identified to obtain an indication type and endpoint information of the road sign, including: identifying a road sign in at least one road image to obtain a first area and a second area of ​​the road sign, wherein the first area is used to indicate an area where a starting point indicating the direction of travel of the vehicle is located, and the second area is used to indicate an area where an end point indicating the direction of travel is located; and determining the endpoint information and indication type based on the first area and the second area.

[0061] The first area mentioned above can be used to indicate the area where the starting point of the vehicle's driving direction is located. The second area mentioned above can be used to indicate the area where the end point of the driving direction is located. For example, if the road sign is a traffic arrow, the first area is the area where the top of the traffic arrow is located, and the second area is the area where the bottom of the traffic arrow is located. Through this recognition process, different types of road signs can be accurately distinguished, providing a basis for subsequent adjustment and rendering, and are applicable to various road environments, including urban roads, highways or rural roads.

[0062] In an optional embodiment, the road sign can be identified by semantic segmentation to obtain the areas where the starting and ending points of the road sign are located. The first area and the second area of ​​the road sign can also be identified by a deep learning model. The deep learning model can be used to identify specific areas of an image. Alternatively, the edge of the road sign can be identified by an edge detection algorithm, and then the contour information of the first area and the second area of ​​the road sign can be identified by morphological processing. By determining the first area and the second area of ​​the road sign, it is convenient to accurately determine the content of the road sign, and determine the type and endpoint information of the road sign.

[0063] Furthermore, endpoint information and indication type are determined based on the first area and the second area, including: determining at least one first endpoint corresponding to the starting point in the first area; determining a target line segment corresponding to the end point in the second area, and determining the midpoint of the target line segment as the second endpoint; determining endpoint information based on at least one first endpoint and a second endpoint, and determining the indication type based on a line connecting at least one first endpoint and the second endpoint.

[0064] The above-mentioned first endpoint may be a point protruding from the starting point of the first area. There may be a corresponding first endpoint or multiple corresponding first endpoints in the starting point of the first area, which is not limited here and can be determined as needed. For example, there may be a corresponding first endpoint in the starting point of the first area, indicating that the road sign can indicate a driving direction, for example, indicating straight ahead or left turn. There may be multiple corresponding first endpoints in the starting point of the first area, indicating that the road sign can indicate multiple driving directions, for example, the road sign indicates that the vehicle traveling in the lane can turn left or go straight, or it may indicate that there are multiple road signs with the same driving direction, for example, at the ramp intersection of the expressway, in order to remind vehicles to get on and off the ramp in time, there will be multiple road signs indicating right turns in the road image, and then there will be multiple first endpoints in the first area.

[0065] The second endpoint may be the midpoint of the target line segment corresponding to the end point in the second area. For example, when the road sign is a traffic arrow, the vertex of the traffic arrow is the first endpoint, and the second endpoint is the midpoint of the bottom line segment of the traffic arrow. Further, the type of the traffic arrow is determined according to the shape of the line segment connecting the vertex and the midpoint.

[0066] The connection between the endpoints is obtained through the endpoint information, and then the direction of the road sign can be accurately captured through the direction and angle of the connection. This is especially critical for instructions that require turning or going straight, and can help drivers prepare for driving in advance and avoid driving errors caused by inaccurate information.

[0067] Furthermore, the road sign in at least one road image is adjusted based on the indication type and endpoint information to obtain an initial road image, including: in the case where there are multiple road sign signs, deduplication processing is performed on the first road sign sign in the road sign based on the indication type and endpoint information, and deletion processing is performed on the second road sign sign in the road sign sign to obtain a sign processing result, wherein the first road sign sign is used to indicate that repeated road sign signs appear in the road sign signs, and the second road sign sign is used to indicate that the road sign signs are within a preset area in the road sign signs; at least one road image is screened based on the sign processing result to obtain an initial road image.

[0068] The above-mentioned first road sign can be used to indicate repeated road signs in road signs. For example, when at an intersection, there may be one or more identical road signs for the same traffic sign to strengthen the reminder of the direction of vehicle travel. However, there is no need to render multiple identical road signs to the vehicle interaction interface, which makes the interaction interface content more, reduces the usability of the road sign, and may cause the user to be unable to judge the road sign of the current lane in time. Therefore, the first road sign is deduplicated and the repeated road signs are deleted to improve the visual effect and information transmission, thereby improving the clarity of the interaction interface and the accuracy of navigation.

[0069] The second road sign mentioned above can be used to indicate the road sign in the preset area among the road sign. The preset area mentioned above can be a driving area, or an area outside the current driving lane. Then the road sign in the preset area can be an interference sign during driving, and there is no need to render the road sign in the preset area to the interactive interface. Therefore, the road sign in the preset area can be deleted, thereby reducing the resource occupation of the initial road image, improving the availability of the initial road image, and improving the efficiency of subsequent rendering.

[0070] The above-mentioned identification processing result can be used to identify the road sign obtained after deduplication and deletion of the road sign. The road sign in the identification processing result is different and is located in the current lane of the vehicle to indicate the drivable direction of the current lane. Through the identification processing result, the road image can be further screened to obtain the initial road image, thereby improving the accuracy of subsequent rendering tasks.

[0071] By deduplicating, deleting, and filtering road signs, the redundant information of the initial road image can be reduced, and resource usage can be reduced. This can significantly improve the clarity and accuracy of the road image, especially in densely marked areas such as city centers or large intersections.

[0072] In an optional embodiment, a stability analysis can be performed on the road image to achieve screening. Optionally, first determine the coordinates of the road sign in the image, and then determine whether the change value of this coordinate in 5 consecutive frames of images is less than a preset threshold. The 5 frames here are only used as an example, and the number of image frames can be adjusted according to actual needs. In response to the change value being less than the preset threshold, it indicates that the coordinates of the road sign remain stable in the consecutive frames, and the road sign is screened out to obtain the initial road image.

[0073] Furthermore, at least one road image is screened based on the identification processing result to obtain an initial road image, including: in the case where there are multiple road images, state information of road signs in the multiple road images is determined based on the identification processing result, wherein the state information is used to indicate whether the display state of the road signs has changed; and at least one road image is screened based on the state information to obtain an initial road image.

[0074] The above-mentioned status information can be used to indicate whether the display status of the road sign has changed. The status information may include, but is not limited to: location information, display information, etc., which are not limited here and can be determined as needed. The location information may be the coordinates of the road sign in the image. The display information may be information such as the display clarity and display time of the road sign. The display status may be one of the following: stability, flickering degree, etc., which are not limited here and can be determined as needed. The status information can be used to determine whether the display status of the road sign has changed, and then the road sign whose display status has not changed can be filtered. When the display status has not changed, for example, the road sign appears in 5 consecutive frames, the road image where the road sign is located is filtered to obtain the initial road image.

[0075] In an optional embodiment, the state information can be compared with the preset state information to screen at least one road image. Optionally, the comparison can be implemented by a comparison model, and the result output by the comparison model can be used to determine whether the display state of the road sign has changed. The comparison model can be one of the following: decision tree, logistic regression, etc., which is not limited here and can be determined as needed. It is also possible to determine whether the state information is consistent with the preset state information stored in the database through query and comparison operations.

[0076] Furthermore, the road signs are adjusted based on the lane lines in the initial road image to obtain the target road image, including: determining lane line position information of the lane lines and sign position information of the road signs based on the initial road image; determining a rectangular frame of the road sign based on the lane line position information and the sign position information; and adjusting the road signs in the initial road image based on the rectangular frame position information of the rectangular frame to obtain the target road image.

[0077] The lane lines mentioned above can be used to represent the boundaries of drivable lanes in the road image. The lane lines can be solid lines or dotted lines. The lane in which the vehicle is currently traveling can be determined by the lane lines, and then the road sign of the current lane can be determined, and the position of the road sign displayed in the current lane can be adjusted.

[0078] The lane line position information can be used to characterize the position of the lane line in the road image. The lane line position information can be used to accurately determine the position of the lane line, and then combined with the sign position information of the road sign to determine the road sign displayed in the rectangular frame corresponding to the lane line.

[0079] The above-mentioned sign position information can be used to characterize the position of the road sign in the road image. The position of the road sign can be accurately determined by the sign position information, and then the road sign can be displayed in the rectangular frame corresponding to the lane line in combination with the lane line position information.

[0080] The rectangular frame may be a quadrilateral surrounding the road sign. The rectangular frame may be determined by the lane line position information and the sign position information. Optionally, the lane where the road sign is located and the lane line position information of the lane may be determined based on the initial road image, and the road sign may be placed at the center of the lane line in combination with the sign position information of the road sign, thereby determining a rectangular frame that can surround the road sign.

[0081] The above-mentioned rectangular frame position information can be used to characterize the position of the rectangular frame surrounding the road sign in the road image. The road sign can be positioned by the rectangular frame position information so as to stably display the road sign in the road image.

[0082] By adjusting the rectangular frame position of the road sign, it can be ensured that the road sign is displayed clearly and accurately on the interactive interface, avoiding confusion between the road sign and the lane line, which is beneficial for navigation and driving in complex road environments.

[0083] Furthermore, the road signs in the initial road image are adjusted based on the rectangular frame position information of the rectangular frame to obtain a target road image, including: adjusting the road signs in the initial road image based on the rectangular frame position information of the rectangular frame to obtain an adjusted road image, wherein the road signs in the adjusted road image are located within the rectangular frame; in response to the rectangular frame in the adjusted road image and the preset road signs in the adjusted road image being in an overlapping area, executing lane line position information and sign position information of the lane line determined based on the initial road image until there is no overlapping area between the rectangular frame in the adjusted road image and the preset road signs, and determining that the adjusted road image is the target road image.

[0084] The above-mentioned preset road sign may be a road sign other than the road indication sign in the road image. The preset road sign may be one of the following: a pedestrian crossing, a warning sign, which is not limited here and can be determined as needed. The preset road sign is different from the road indication sign and cannot provide guidance on the driving direction.

[0085] The above-mentioned overlap can be to adjust the rectangular frame in the road image to overlap with the preset road sign. The type of overlap can be complete overlap, intersection, etc., which is not limited here and can be determined as needed. By identifying the overlapping area through the rectangular frame position information and then adjusting the road sign, it can effectively avoid the occlusion between the road sign and ensure the visibility of each road sign. For overlapping situations, such as tunnel entrances, complex overpasses, etc., it can provide better visual effects and information transmission.

[0086] The above-mentioned adjusted road image can be an image of the surrounding environment of the road on which the vehicle is traveling after the position of the road sign is adjusted. In the adjusted road image, the road sign is located in a rectangular frame, which is convenient for further determining whether the road sign has an overlapping area with the preset road sign, and then adjusting the road sign in time, so as to improve the rendering effect of subsequent rendering tasks.

[0087] In an optional embodiment, the road signs in the initial road image can be moved to achieve adjustments. For example, when intersecting a stop line and a crosswalk, if they are directly rendered to the interactive interface, it will be difficult for users to judge the road signs of the current lane. Therefore, a strategy of moving back in small steps is adopted to reposition the road signs so that the road signs can be displayed intuitively.

[0088] Furthermore, rendering the road sign in the target road image onto the interactive interface of the vehicle includes: determining a mapping direction of the road sign based on a vertex order of the rectangular frame; and rendering the road sign onto the interactive interface based on the mapping direction.

[0089] By precisely controlling the direction of the map, it is possible to ensure that the direction of the road signs displayed on the interactive interface is consistent with the actual road direction, which is crucial for drivers to understand the road signs, especially when turning or changing lanes, and can provide intuitive navigation guidance.

[0090] In an optional embodiment, the vertex order of the rectangular frame surrounding the road sign can be determined first to determine the orientation of the road sign, and then the mapping direction of the road sign can be determined to ensure that the displayed direction of the road sign is the same as the actual direction on the road, so as to improve rendering efficiency. Optionally, the mapping direction can be changed by rotation. Further, the road sign is rendered on the interactive interface based on the mapping direction to provide the correct guidance direction of the road sign for user observation. The road sign can be rendered on the interactive interface of the vehicle through a graphics rendering engine. The road sign can also be rendered through a hardware accelerator.

[0091] According to an embodiment of the present invention, an embodiment of a device for rendering a road sign is provided. It should be noted that the device can be used to execute the above-mentioned method for rendering a road sign.

[0092] Figure 3 is a schematic diagram of a rendering device for a road sign according to an embodiment of the present application, such as Figure 3 As shown, the device includes the following: an acquisition module 32 , a recognition module 34 , an adjustment module 36 and a rendering module 38 .

[0093] The acquisition module is used to acquire at least one road image captured by the vehicle during the driving of the vehicle.

[0094] The recognition module is used to recognize a road sign in at least one road image to obtain the indication type and endpoint information of the road sign, wherein the endpoint information is used to indicate the indication direction of the road sign.

[0095] The adjustment module is used to adjust the road indication mark in at least one road image based on the indication type and endpoint information to obtain a target road image.

[0096] The rendering module is used to render the road sign in the target road image onto the interactive interface of the vehicle.

[0097] It should be noted that the acquisition module, identification module, adjustment module and rendering module correspond to steps S102 to S108 in the above embodiment, and the four modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above embodiment. It should be noted that the above modules or units can be hardware components or software components stored in a memory and processed by one or more processors.

[0098] Furthermore, the adjustment module is also used to adjust the road indication signs in at least one road image based on the indication type and endpoint information to obtain an initial road image; and adjust the road indication signs based on the lane lines in the initial road image to obtain a target road image.

[0099] Furthermore, the recognition module is also used to identify road signs in at least one road image to obtain a first area and a second area of ​​the road sign, wherein the first area is used to indicate an area where a starting point indicating the direction of travel of the vehicle is located, and the second area is used to indicate an area where an end point indicating the direction of travel is located; and endpoint information and indication type are determined based on the first area and the second area.

[0100] Furthermore, the identification module is also used to determine at least one first endpoint corresponding to the starting point in the first area; determine the target line segment corresponding to the end point in the second area, and determine the midpoint of the target line segment as the second endpoint; determine endpoint information based on at least one first endpoint and the second endpoint, and determine the indication type based on the connection line between at least one first endpoint and the second endpoint.

[0101] Furthermore, the adjustment module is also used to deduplicate the first road sign in the road sign based on the indication type and endpoint information, and delete the second road sign in the road sign, when there are multiple road sign signs, to obtain a sign processing result, wherein the first road sign is used to indicate that repeated road sign signs appear in the road sign signs, and the second road sign is used to indicate that the road sign signs are within a preset area; and at least one road image is screened based on the sign processing result to obtain an initial road image.

[0102] Furthermore, the adjustment module is also used to determine the status information of the road signs in the multiple road images based on the sign processing results when there are multiple road images, wherein the status information is used to indicate whether the display status of the road signs has changed; and to filter at least one road image based on the status information to obtain an initial road image.

[0103] Furthermore, the adjustment module is also used to determine the lane line position information of the lane line and the sign position information of the road sign based on the initial road image; determine the rectangular frame of the road sign based on the lane line position information and the sign position information; and adjust the road sign in the initial road image based on the rectangular frame position information of the rectangular frame to obtain the target road image.

[0104] Furthermore, the adjustment module is also used to adjust the road signs in the initial road image based on the rectangular frame position information of the rectangular frame to obtain an adjusted road image, wherein the road signs in the adjusted road image are located within the rectangular frame; in response to the rectangular frame in the adjusted road image and the preset road signs in the adjusted road image being in an overlapping area, the lane line position information of the lane line and the sign position information of the road signs are determined based on the initial road image until there is no overlapping area between the rectangular frame in the adjusted road image and the preset road signs, and the adjusted road image is determined to be the target road image.

[0105] Furthermore, the rendering module is also used to determine the mapping direction of the road sign based on the vertex sequence of the rectangular frame; and render the road sign onto the interactive interface based on the mapping direction.

[0106] An embodiment of the present application further provides an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of the present invention when running.

[0107] An embodiment of the present application further provides a computer-readable storage medium, which includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present invention.

[0108] An embodiment of the present application further provides a computer program product, including a computer program, which implements the methods in various embodiments of the present invention when executed by a processor.

[0109] An embodiment of the present application further provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0110] The embodiments of the present application further provide a computer program, which implements the methods in the above-mentioned embodiments of the present invention when executed by a processor.

[0111] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0112] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0113] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0114] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0115] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0116] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for rendering a road sign, characterized in that: include: During the driving of the vehicle, acquiring at least one road image collected by the vehicle; Identify a road sign in the at least one road image to obtain an indication type and endpoint information of the road sign, wherein the endpoint information is used to indicate an indication direction of the road sign; Adjusting the road indication mark in the at least one road image based on the indication type and the endpoint information to obtain a target road image; The road indication mark in the target road image is rendered on an interactive interface of the vehicle.

2. The method for rendering a road sign according to claim 1, characterized in that: The method further comprises: adjusting the road indication mark in the at least one road image based on the indication type and the endpoint information to obtain a target road image, including: Adjusting the road indication mark in the at least one road image based on the indication type and the endpoint information to obtain an initial road image; The road indication mark is adjusted based on the lane line in the initial road image to obtain the target road image.

3. The method for rendering a road sign according to claim 2, characterized in that: Identifying a road indication mark in the at least one road image to obtain an indication type and endpoint information of the road indication mark includes: Identify the road indicator in the at least one road image to obtain a first area and a second area of ​​the road indicator, wherein the first area is used to indicate an area where a starting point indicating a driving direction of the vehicle is located, and the second area is used to indicate an area where an end point indicating the driving direction is located; The endpoint information and the indication type are determined based on the first area and the second area.

4. The method for rendering a road sign according to claim 3, characterized in that: Determining the endpoint information and the indication type based on the first area and the second area includes: Determine at least one first endpoint corresponding to the starting point in the first area; Determine a target line segment corresponding to the end point in the second region, and determine the midpoint of the target line segment as the second end point; The endpoint information is determined based on the at least one first endpoint and the second endpoint, and the indication type is determined based on a connection between the at least one first endpoint and the second endpoint.

5. The method for rendering a road sign according to claim 2, characterized in that: The method further comprises: adjusting the road indication mark in the at least one road image based on the indication type and the endpoint information to obtain an initial road image, including: In the case where there are multiple road sign identifications, a first road sign identification among the road sign identifications is deduplicated based on the indication type and the endpoint information, and a second road sign identification among the road sign identifications is deleted to obtain a sign processing result, wherein the first road sign identification is used to indicate that duplicate road sign identifications appear among the road sign identifications, and the second road sign identification is used to indicate that the road sign identifications among the road sign identifications are within a preset area; The at least one road image is screened based on the identification processing result to obtain the initial road image.

6. The method for rendering a road sign according to claim 5, characterized in that: The at least one road image is screened based on the identification processing result to obtain the initial road image, including: In the case where there are multiple at least one road image, determining state information of the road indication signs in the multiple road images based on the sign processing result, wherein the state information is used to indicate whether the display state of the road indication signs changes; The at least one road image is screened based on the state information to obtain the initial road image.

7. The method for rendering a road sign according to claim 2, characterized in that: The road indication mark is adjusted based on the lane line in the initial road image to obtain a target road image, including: Determine lane line position information of the lane line and sign position information of the road indication sign based on the initial road image; Determine a rectangular frame of the road indication mark based on the lane line position information and the mark position information; The road indication mark in the initial road image is adjusted based on the rectangular frame position information of the rectangular frame to obtain the target road image.

8. The method for rendering a road sign according to claim 7, characterized in that: The road indication mark in the initial road image is adjusted based on the rectangular frame position information of the rectangular frame to obtain the target road image, including: adjusting the road indication mark in the initial road image based on the rectangular frame position information of the rectangular frame to obtain an adjusted road image, wherein the road indication mark in the adjusted road image is located within the rectangular frame; In response to the rectangular box in the adjusted road image and the preset road sign in the adjusted road image being in an overlapping area, the lane line position information of the lane line and the sign position information of the road sign based on the initial road image are determined until the rectangular box in the adjusted road image and the preset road sign no longer have the overlapping area, and the adjusted road image is determined to be the target road image.

9. The method for rendering a road sign according to claim 8, characterized in that: Rendering the road indication mark in the target road image onto an interactive interface of the vehicle includes: Determining the mapping direction of the road sign based on the vertex order of the rectangular frame; The road indication sign is rendered on the interactive interface based on the mapping direction.

10. An electronic device, characterized in that: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 9 when running.

Images