Road vehicle information extraction method, device, equipment, medium and program product

Abstract

The present application provides a road vehicle signal extraction method, device, equipment, medium and program product, which relates to the field of electronic map technology. The method includes: according to the element detection model corresponding to the map element, the element recognition is performed on the current road image of the target vehicle to obtain the recognition result of the map element, and the recognition result of the map element includes the lane line recognition result; according to the lane line recognition result, the lane boundary on the lane where the target vehicle is traveling is determined, and the sub-lane where the target vehicle is located is determined according to the lane boundary, and the road vehicle signal corresponding to the sub-lane is extracted; wherein the road vehicle signal is used to indicate the vehicle driving road surface information of the target vehicle. Through the above method, the extraction accuracy of the road vehicle signal on the current lane of the vehicle can be effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of electronic maps, and in particular to a method, device, equipment, medium and program product for extracting road vehicle information. Background Art

[0002] With the advancement of science and technology and the acceleration of urbanization, vehicle driving safety has also received more and more attention. Among them, road traffic information (hereinafter referred to as "road vehicle information") plays an important role in scenarios such as automatic map construction, vehicle navigation and path planning.

[0003] Taking the automatic map construction process as an example, identifying and extracting various road elements is the key to ensuring map accuracy. During vehicle driving, it is particularly important to accurately extract the road information of the vehicle's current road, which can include lane information, traffic signs, and other dynamic and static data related to driving. This information not only affects the driving safety and efficiency of the vehicle, but is also directly related to the decision-making and control capabilities of the autonomous driving system.

[0004] Therefore, there is an urgent need to propose a technical solution that can accurately extract vehicle road information, thereby improving the driving safety of vehicles. Summary of the invention

[0005] The present application provides a road vehicle signal extraction method, device, equipment, medium and program product, which can effectively improve the accuracy of road vehicle signal extraction.

[0006] According to one aspect of the present application, a road vehicle signal extraction method is provided, comprising: performing element recognition on a current road image of a target vehicle according to an element detection model corresponding to a map element to obtain a recognition result of the map element, wherein the recognition result of the map element includes a lane line recognition result; determining a lane boundary on a driving lane of the target vehicle according to the lane line recognition result, and determining a sub-lane in which the target vehicle is located according to the lane boundary to extract a road vehicle signal corresponding to the sub-lane; wherein the road vehicle signal is used to indicate vehicle driving surface information of the target vehicle.

[0007] In a possible implementation, determining the sub-lane in which the vehicle is located based on the lane line recognition result includes: performing straight line fitting on lane lines in the same direction based on the lane line recognition result to obtain multiple fitted lane lines; and determining the left lane boundary and the right lane boundary of the target vehicle based on the lane line slope of each fitted lane line to obtain the sub-lane in which the target vehicle is located.

[0008] In a possible implementation, determining the left lane boundary and the right lane boundary of the vehicle based on the lane line slope includes: for each fitted lane line in the sorting result based on the left and right sorting, if the positive and negative values ​​of the lane line slopes of the adjacent lane lines are opposite, then determining the adjacent lane lines as the left lane boundary and the right lane boundary of the vehicle; wherein, among the adjacent lane lines, the lane line with a negative lane line slope is the left lane boundary, and the lane line with a positive lane line slope is the right lane boundary.

[0009] In a possible implementation, the method further includes at least one of the following: sorting the fitted lane lines from left to right according to the slope of each fitted lane line to obtain a sorting result of each fitted lane line; sorting the center points of each fitted lane line from left to right according to the center points of each fitted lane line to obtain a sorting result of each fitted lane line; wherein the sorting result is used to determine the sub-lane in which the target vehicle is located.

[0010] In a possible implementation, the recognition result of the map element includes at least one of a curb recognition result and a guardrail recognition result; the extracting of the road vehicle signal corresponding to the sub-lane includes: obtaining the position of the curb in the road image according to the curb recognition result to determine the relative position between the curb and the lane boundary; determining the periphery of the boundary of the curb according to the relative position of the curb; and filtering out invalid road vehicle signals located outside the boundary of the curb to obtain the road vehicle signal corresponding to the sub-lane.

[0011] In a possible implementation, the extracting of the road vehicle signal corresponding to the sub-lane includes: obtaining the position of the guardrail in the road image according to the guardrail recognition result to determine the relative position between the guardrail and the lane boundary; determining the periphery of the boundary of the guardrail according to the relative position of the guardrail; and filtering out invalid road vehicle signals located outside the boundary of the guardrail to obtain the road vehicle signal corresponding to the sub-lane.

[0012] In a possible implementation, determining the perimeter boundary of the curb based on the relative position of the curb includes determining a valid curb among the curbs based on the relative position of the curb; wherein the valid curbs include a curb located on the left side of a left lane corresponding to the lane boundary, and a curb located on the right side of a right lane corresponding to the lane boundary; and determining the perimeter boundary of the curb by locating the valid curb away from one side of the lane boundary.

[0013] In a possible implementation, determining the perimeter of the boundary of the guardrail based on the relative position of the guardrail includes: determining an effective guardrail in the guardrail based on the relative position of the guardrail; wherein the effective guardrail includes a guardrail located on the left side of a left lane corresponding to the lane boundary, and a guardrail located on the right side of a right lane corresponding to the lane boundary; and determining the perimeter of the boundary of the guardrail by moving the effective guardrail away from the lane boundary.

[0014] In a possible implementation, the feature detection model is a model trained based on each type of historical map feature data, including at least one of the following: a lane line detection model, a guardrail detection model and a curb detection model; the feature recognition of the vehicle's current road image to obtain a recognition result of the map element corresponding to the road image includes at least one of the following: based on the lane line detection model, detecting the lane lines in the road image to obtain a lane line recognition result; based on the guardrail detection model, detecting the guardrails in the road image to obtain a guardrail recognition result; based on the curb detection model, detecting the curbs in the road image to obtain a curb recognition result.

[0015] In a possible implementation, the method further includes at least one of the following: based on a vehicle detection model, detecting vehicles in the road image to obtain a vehicle recognition result; wherein the vehicle detection model is a model trained based on historical vehicle data; based on a road arrow detection model, detecting road arrows in the road image to obtain an arrow recognition result; wherein the road arrow detection model is a model trained based on historical road arrow data; extracting the road vehicle signal corresponding to the sub-lane includes: extracting the road vehicle signal corresponding to the sub-lane according to at least one of the vehicle recognition result and the arrow recognition result, the road vehicle signal including the vehicle and / or road arrow corresponding to the sub-lane.

[0016] According to a second aspect of the present application, a road vehicle signal extraction device is provided, comprising: an element recognition module, which is configured to perform element recognition on a current road image of a target vehicle according to an element detection model corresponding to a map element, and obtain a recognition result of the map element, wherein the recognition result of the map element includes a lane line recognition result; a determination and extraction module, which is configured to determine a lane boundary on a driving lane of the target vehicle according to the lane line recognition result, and determine a sub-lane in which the target vehicle is located according to the lane boundary, so as to extract the road vehicle signal corresponding to the sub-lane; wherein the road vehicle signal is used to indicate vehicle driving surface information of the target vehicle.

[0017] In a possible implementation, the determination and extraction module includes: a lane line fitting module, which is configured to perform straight line fitting on lane lines in the same direction according to the lane line recognition results to obtain multiple fitted lane lines; a boundary determination module, which is configured to determine the left lane boundary and the right lane boundary of the target vehicle according to the lane line slopes of each fitted lane line to obtain the sub-lane where the target vehicle is located.

[0018] In a possible implementation, the boundary determination module is specifically configured as follows: for each fitted lane line in the sorting result based on the left-right sorting, if the positive and negative values ​​of the lane line slopes of the adjacent lane lines are opposite, then the adjacent lane lines are determined to be the left lane boundary and the right lane boundary of the vehicle; wherein, the lane line with a negative lane line slope among the adjacent lane lines is the left lane boundary, and the lane line with a positive lane line slope is the right lane boundary.

[0019] In a possible implementation, the device further includes at least one of the following: a first sorting module, which is configured to sort the fitted lane lines from left to right according to the slope of each fitted lane line to obtain a sorting result for each fitted lane line; a second sorting module, which is configured to sort the center points of each fitted lane line from left to right according to the center points of each fitted lane line to obtain a sorting result for each fitted lane line; wherein the sorting result is used to determine the sub-lane in which the target vehicle is located.

[0020] In a possible implementation, the recognition result of the map element includes at least one of a curb recognition result and a guardrail recognition result; the determination and extraction module includes: a first relative position determination unit, which is configured to obtain the position of the curb in the road image based on the curb recognition result to determine the relative position between the curb and the lane boundary; a first periphery determination unit, which is configured to determine the periphery of the boundary of the curb based on the relative position of the curb; and a first filtering unit, which is configured to filter out invalid road vehicle signals located at the periphery of the boundary of the curb to obtain the road vehicle signals corresponding to the sub-lane.

[0021] In a possible implementation, the determination and extraction module includes: a second relative position determination unit, which is configured to obtain the position of the guardrail in the road image based on the guardrail recognition result to determine the relative position between the guardrail and the lane boundary; a second periphery determination unit, which is configured to determine the periphery of the boundary of the guardrail based on the relative position of the guardrail; and a second filtering unit, which is configured to filter out invalid road vehicle signals located at the periphery of the boundary of the guardrail to obtain the road vehicle signals corresponding to the sub-lane.

[0022] In a possible implementation manner, the first periphery determination unit is specifically configured to: determine a valid curb among the curbs according to the relative positions of the curbs; wherein the valid curbs include curbs located on the left side of a left lane corresponding to the lane boundary, and curbs located on the right side of a right lane corresponding to the lane boundary; and determine the periphery of the boundary of the curb by locating the valid curb away from one side of the lane boundary.

[0023] In a possible implementation, the second periphery determination unit includes: determining a valid guardrail among the guardrails according to the relative position of the guardrails; wherein the valid guardrails include a guardrail located on the left side of a left lane corresponding to the lane boundary, and a guardrail located on the right side of a right lane corresponding to the lane boundary; and determining the periphery of the boundary of the guardrail by moving the valid guardrail away from one side of the lane boundary.

[0024] In a possible implementation, the feature detection model is a model trained based on each type of historical map feature data, including at least one of the following: a lane line detection model, a guardrail detection model and a curb detection model; the feature recognition module includes at least one of the following: a lane line recognition unit, which is configured to detect lane lines in the road image based on the lane line detection model to obtain lane line recognition results; a guardrail recognition unit, which is configured to detect guardrails in the road image based on the guardrail detection model to obtain guardrail recognition results; and a curb recognition unit, which is configured to detect curbs in the road image based on the curb detection model to obtain curb recognition results.

[0025] In a possible embodiment, the device also includes at least one of the following: based on a vehicle detection model, detecting the vehicle in the road image to obtain a vehicle recognition result; wherein the vehicle detection model is a model trained based on historical vehicle data; based on a road arrow detection model, detecting the road arrows in the road image to obtain an arrow recognition result; wherein the road arrow detection model is a model trained based on historical road arrow data; the determination and extraction module includes: a vehicle signal extraction unit, which is configured to extract the road vehicle signal corresponding to the sub-lane according to at least one of the vehicle recognition result and the arrow recognition result, and the road vehicle signal includes the vehicle and / or road arrow corresponding to the sub-lane.

[0026] According to the third aspect of the present application, an electronic device is provided, comprising: a memory and a processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the road vehicle information extraction method provided in any one of the first aspects above.

[0027] According to the fourth aspect of the present application, a computer-readable storage medium is provided, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, they are used to implement the road vehicle information extraction method provided in any one of the first aspects above.

[0028] According to a fifth aspect of the present application, a computer program product is provided, which includes a computer program, and when the computer program is executed by a processor, the road vehicle information extraction method provided in any one of the first aspects above is implemented.

[0029] The road vehicle information extraction method, device, equipment, medium and program product provided by the present application perform element recognition on the current road image of the target vehicle, and use the lane line recognition result to determine the lane boundary of the target vehicle, and determine the sub-lane where the target vehicle is located according to the lane boundary to extract the road vehicle information corresponding to the sub-lane. The road vehicle information corresponding to the sub-lane more accurately locates the driving surface information of the vehicle's current road, thereby realizing the accurate extraction of the vehicle's current road vehicle information. The map automatically constructed or updated based on the road vehicle information has higher accuracy and helps to improve the safety of vehicle driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0031] Figure 1 A schematic diagram of a process for extracting road vehicle information provided in an embodiment of the present application;

[0032] Figure 2 This is an example diagram of the lane line recognition result in the embodiment of the present application;

[0033] Figure 3 This is an example diagram of the guardrail recognition result in the embodiment of the present application;

[0034] Figure 4 This is an example diagram of the curb recognition result in the embodiment of the present application;

[0035] Figure 5 for Figure 1 Schematic diagram of the process of step S102;

[0036] Figure 6 This is an example diagram after lane line fitting in an embodiment of the present application;

[0037] Figure 7 This is an example diagram of the left and right boundaries of a sub-lane in an embodiment of the present application;

[0038] Figure 8 This is an example diagram of sorting based on the center points of lane lines in an embodiment of the present application;

[0039] Fig. 9 This is an example diagram of the road arrow recognition result in the embodiment of the present application;

[0040] Fig.10 This is an example diagram of the vehicle recognition result in the embodiment of the present application;

[0041] Fig.11a This is one of the example diagrams of the road arrows and vehicle recognition results after filtering out in the embodiment of the present application;

[0042] Fig.11b This is one of the example diagrams of the road arrows and vehicle recognition results after filtering out in the embodiment of the present application;

[0043] Fig.12 A schematic diagram of a process for extracting road vehicle information provided by an exemplary embodiment of the present application;

[0044] Fig.13 A schematic diagram of the structure of a road vehicle information extraction device provided in an embodiment of the present application;

[0045] Fig.14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0046] Fig.15 It is a block diagram of a terminal device provided by an exemplary embodiment of the present application.

[0047] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0048] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0049] It can be understood that in the embodiments of the present application, the term "and / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0050] With the advancement of science and technology and the acceleration of urbanization, road structure and traffic environment are undergoing rapid changes. This change is not only reflected in the construction of new roads and the renovation of old roads, but also includes the update of traffic signs such as road arrows and signal lights, as well as the dynamic changes of vehicles on the road during driving. This dynamic traffic environment puts higher requirements on map services, especially in the context of the increasing popularity of autonomous driving and advanced driving assistance systems (ADAS). The timely update and accuracy of maps have become crucial, and the accurate extraction of road vehicle information is a very important part of map automation construction.

[0051] In the related technology, the extraction of road vehicle information usually involves the identification and acquisition of map elements in the overall road structure. The amount of data that needs to be processed is very large, and the identification results contain a lot of invalid information, which affects the accuracy of road vehicle information extraction. For example, when a vehicle is traveling on multiple lanes, lane information (such as road arrows) on multiple lanes will be extracted at the same time, while road arrows on other lanes (especially opposite lanes) are usually meaningless for vehicle driving planning. In other words, the current road vehicle information extraction scheme lacks a technical solution for the accurate extraction of road vehicle information on the lane where the vehicle is currently traveling.

[0052] In view of this, an embodiment of the present application proposes a technical solution for extracting road vehicle information, which performs element recognition on the current road image of the target vehicle, and uses the lane line recognition result to determine the lane boundary on the target vehicle's driving lane, and determines the sub-lane where the target vehicle is located based on the lane boundary to extract the road vehicle information corresponding to the sub-lane. The road vehicle information corresponding to the sub-lane more accurately locates the driving surface information of the vehicle's current road, thereby realizing the accurate extraction of the vehicle's current road vehicle information. The map automatically constructed or updated based on the road vehicle information has higher accuracy and helps to improve the safety of vehicle driving.

[0053] The following is a detailed description of the technical solution of the present application and how the technical solution of the present application solves the above technical problems in conjunction with the accompanying drawings and specific embodiments. It should be noted that these specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0054] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0055] In a possible application scenario, the executor of the technical solution provided in this embodiment may be a microelectronic device deployed in the on-board camera of the target vehicle to accurately extract the vehicle's current road vehicle information elements, thereby assisting in autonomous driving. In some application scenarios, the technical solution provided by the embodiment of this application can also be applied to the application scenario of intelligent driving, such as the application scenario of autonomous driving based on vehicle cloud computing. The executor of the method provided by the embodiment of this application may be a server, such as a server of a high-precision map producer, which can identify the current lane status through the on-board video stream and assist in updating the map's vehicle information results on the current road. Alternatively, in some embodiments, it can also be applied to the review scenario of element information to facilitate the review of element information by relevant algorithm developers and testers. The relevant applications will not be repeated one by one in this embodiment. The following is an introduction to the executor of the method provided by the embodiment of this application using an electronic device as the embodiment.

[0056] Figure 1 A schematic diagram of a road vehicle information extraction method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method may include the following steps S201 and S102.

[0057] Step S101: perform element recognition on a current road image of a target vehicle according to an element detection model corresponding to a map element to obtain a recognition result of the map element, wherein the recognition result of the map element includes a lane line recognition result.

[0058] In this embodiment, the feature detection model may adopt a target detection model, such as a YOLO detection model, an SSD detection model, and the like.

[0059] In this embodiment, the map elements may include one or more combinations of lane lines, road arrows, guardrails, and curbs. In some embodiments, they may also be other elements, such as traffic lights, etc. Among them, the element detection model can be a unified model that is trained based on the element data corresponding to multiple map elements and can be used to detect multiple map elements separately at the same time, or it can be a single model that is trained based on the element data corresponding to one map element and detects specific intent elements. This embodiment does not specifically limit the specific form of the target detection model. For example, in the following examples, only a single model is used to extract elements as an example to introduce the embodiments of the present application, but the technical solution of the present application is not limited.

[0060] The target vehicle may be a vehicle for which road information needs to be extracted (for automatic map construction or path planning). The road image may be a road image captured by a vehicle-mounted camera, or a road image with the target vehicle as the base point captured by a camera on the road, or a road image with the target vehicle as the base point captured by a remote sensing satellite, etc.

[0061] In one possible implementation, the above-mentioned feature detection model may be a model trained based on each type of historical map feature data, for example, it may be one or more of a lane line detection model, a guardrail detection model, and a curb detection model. In some embodiments, it may also be other detection models trained with other types of map feature data.

[0062] In the above steps, the elements of the current road image of the vehicle are recognized to obtain the recognition result of the map elements corresponding to the road image, which can be achieved in the following manner:

[0063] In one example, based on a lane line detection model, lane lines in the road image are detected to obtain lane line recognition results.

[0064] In this example, the lane line detection model can use a historical image dataset containing lane lines, in which the images in the historical image dataset have lane lines annotated, i.e., annotated data (corresponding to historical map element data), and the annotated data is converted into a format that the model can understand (such as a binary mask image) for model training. A suitable deep learning model architecture can be selected, such as a convolutional neural network (CNN), a fully convolutional network (FCN), etc. as the initial model, and the annotated data is input into the initial model for training. When the corresponding loss conditions are met (determined by a person skilled in the art in combination with the actual application adaptability), a lane line detection model is obtained to ensure the recognition accuracy of the lane lines.

[0065] Figure 2 This is an example of using the lane line detection model to detect lane lines in a road image. The marked position is the identified lane line. It can be seen that on the road where the vehicle is driving, there are multiple lanes and multiple lane lines. If all the lane lines are used as road vehicle information, it will be difficult to meet the accuracy of vehicle navigation due to too much data.

[0066] In another example, based on the guardrail detection model, the guardrail in the road image is detected to obtain a guardrail recognition result.

[0067] It should be noted that the training process of the guardrail detection model can refer to the training process of the lane line mentioned above, and the relevant description will not be repeated here. Figure 3This is an example of using the guardrail detection model to detect guardrails in road images. The position of the marked box a is the identified guardrail. It can be seen that the lane on the other side of the guardrail is the opposite lane. The road traffic information in the opposite lane is invalid or even confusing information for the target vehicle, which will affect the road traffic information in the vehicle's current lane.

[0068] In yet another example, based on the curb detection model, curbs in the road image are detected to obtain curb recognition results.

[0069] It should be noted that the training process of the curb detection model can refer to the training process of the lane line above, and the relevant description will not be repeated here. Figure 4 This is an example of using the curb detection model to detect curbs in road images. The marked box b is the identified curb. It can be understood that curbs are also called "curbs" or "curbs". They are precast concrete blocks or masonry that are built between the roadway and the sidewalk, higher than the road surface and basically level with the sidewalk. Its main function is to protect pedestrian safety and form drainage ditches at the edge of the roadway. Similar to guardrails, curbs are usually used to divide the roadway and the sidewalk. When a curb appears on the vehicle's driving path, the elements outside the curb are usually invalid or even confusing information, which will affect the road traffic information of the vehicle's current lane.

[0070] In some examples, the above examples can be combined to simultaneously detect lane lines, guardrails, curbs and even more map elements in the road image to further improve the accuracy of road vehicle information extraction.

[0071] Continue to see Figure 1 , step S102, determining the lane boundary on the target vehicle's driving lane according to the lane line recognition result, and determining the sub-lane where the target vehicle is located according to the lane boundary, so as to extract the road vehicle information corresponding to the sub-lane; wherein the road vehicle information is used to indicate the vehicle driving road surface information of the target vehicle.

[0072] In this embodiment, the two lane lines closest to the target vehicle among the identified lane lines can be used to determine the lane boundaries, and the area formed between the lane boundaries can be used to determine the sub-lane where the vehicle is located. In some embodiments, the sub-lane where the target vehicle is located can also be determined in combination with the guardrail recognition results or the curb recognition results. For example, if a row of guardrails is identified in the road image, the lane on one side of the lanes on both sides of the guardrail where the target vehicle is located can be determined as the initial lane of the vehicle, and then the lane line recognition results can be used to determine the sub-lane from the initial lane, and the curb recognition process is the same. It can be understood that guardrails and curbs are usually used to divide forward and reverse lanes. When there are multiple sub-lanes in the forward and reverse lanes, the sub-lane where the vehicle is located can be quickly located, and then the lane line recognition results can be used to determine the sub-lane where the vehicle is located, which has a smaller range and higher efficiency.

[0073] In this embodiment, road vehicle information may include road information or vehicle information on the road surface on which the vehicle is traveling, such as road arrows (such as the number of road arrows, the type of road arrows, etc.), lane information (such as the number of lanes, the type of lanes), and other information, and vehicle information such as the number of vehicles, the type of vehicle (large vehicle, small vehicle), and so on.

[0074] It should be noted that, after determining the sub-lane in which the vehicle is located, the road vehicle signals extracted in this embodiment may be road vehicle signals on the sub-lane, or road vehicle signals on the sub-lane and its surrounding (same-direction) lanes (e.g., the number of lane lines in the same-direction lane), and this embodiment does not specifically limit this.

[0075] Through the above technical solution, by performing element recognition on the current road image of the target vehicle, and using the recognition results of the map elements to determine the sub-lane where the target vehicle is located, the sub-lane includes the lane boundary on the target vehicle's driving lane, so as to extract the road vehicle information corresponding to the sub-lane. The road vehicle information corresponding to the sub-lane more accurately locates the driving surface information of the vehicle's current road, and realizes the accurate extraction of the vehicle's current road vehicle information. The map automatically constructed or updated based on the road vehicle information has higher accuracy and helps to improve the safety of vehicle driving.

[0076] As mentioned above, using lane line recognition results to determine sub-lanes has a more efficient range and higher accuracy. Next, using lane lines to identify the sub-lanes where the target vehicle is located is further introduced. In this embodiment, the lane boundaries include left lane boundaries and right lane boundaries. Figure 5 As shown, the above step S102 may include the following steps:

[0077] Step S1021: According to the lane line recognition result, linear fitting is performed on the lane lines in the same direction to obtain multiple fitted lane lines.

[0078] In actual applications, lane lines may be solid lane lines or dashed lane lines. There may be multiple lane lines in the same direction. The model will identify these lane lines as separate lane lines. To facilitate subsequent calculations, this embodiment first performs straight-line fitting on the lane lines in the same direction. The fitting method can use the straight-line formula y=kx+b. Substitute the data of multiple lane lines into the formula for fitting, and obtain the k and b values ​​to obtain the fitted lane lines. It can be understood that the same direction in this embodiment is the direction in which the vehicle travels along the lane line, that is, the direction in which the vehicle moves. In one example, the schematic diagram of straight-line fitting for lane lines in the same direction is as follows: Figure 6As shown in the figure, the blue line indicates the forward direction of the lane line, that is, the same direction of the lane line. Multiple lane lines in the same direction are fitted into a fitted lane line. It can be seen that after straight line fitting, the lane line is more intuitive and clear, which is convenient for subsequent calculations and identification of lane boundaries.

[0079] Continue as Figure 5 As shown, step S1022, according to the lane line slopes of each fitted lane line, determine the left lane boundary and the right lane boundary of the target vehicle to obtain the sub-lane where the target vehicle is located.

[0080] The lane line slope of each fitted lane line is the k value in the above straight line formula. In some embodiments, the lane line slope may also be calculated by other related technologies, which is not particularly limited in this embodiment.

[0081] Compared with using the two lane lines closest to the vehicle (left and right) as the left and right lane boundaries of the vehicle, since the slope of the lane lines follows a specific rule (the slope increases from left to right on the left side of the vehicle), it has a corrective effect when there are misidentified lane lines (if the slope of the misidentified lane lines does not conform to the above rule, they can be eliminated), which effectively improves the lane recognition accuracy.

[0082] In a possible implementation, the positive and negative values ​​of the lane line slope can be used to determine the left and right lane boundaries of the vehicle to improve the efficiency and accuracy of sub-lane determination. Specifically, the above step S1022 determines the left lane boundary and the right lane boundary of the vehicle according to the lane line slope in the following manner:

[0083] For each fitted lane line, in the sorting result based on the left-right sorting, if the lane line slopes of the adjacent lane lines have opposite positive and negative values, the adjacent lane lines are determined to be the left lane boundary and the right lane boundary of the vehicle;

[0084] Among them, the lane line with a negative lane line slope among the adjacent lane lines is the left lane boundary, and the lane line with a positive lane line slope is the right lane boundary.

[0085] Exemplarily, the K values ​​of the sorted lane lines are compared. The left and right boundaries of the current vehicle driving sub-lane are calculated through the adjacent slope K values. From the sorted lane lines, the lane line K value slopes are obtained from left to right for calculation and comparison. Suppose the slope of one lane line is lane[i].k, then the slope of the lane line on the right is lane[i+1].k, where i represents the i-th lane line, if the following conditions are met (the slope K calculation is based on the image coordinate system):

[0086] lane[i].k*lane[i+1].k<0 and lane[i].k<0 and lane[i+1].k>0

[0087] Then lane lines lane[i] and lane[i+1] are the left and right boundaries of the current vehicle driving sub-lane, respectively. The lane line where lane[i].k<0 is the left lane boundary. Otherwise, continue to loop down and search to calculate the left and right boundaries of the current vehicle driving sub-lane as follows: Figure 7 shown.

[0088] In some embodiments, in addition to using the positive and negative values ​​of the slope to determine the left and right lane boundaries of the vehicle, other methods can be used to determine the left and right lane boundaries. For example, the left and right lane boundaries can be determined by using the size of the slope value. For example, on a road with a larger curvature, the lane line close to the vehicle will have a larger slope change. By comparing the absolute value of the slope size, it can be inferred which line is more likely to be the left boundary or the right boundary.

[0089] In some embodiments, the fitted lane lines can be sorted by the slope of each lane line or the center point of each lane line to improve sorting efficiency and accuracy. Specifically, the sorting results of the fitted lane lines based on the left-right sorting can be as follows:

[0090] In one example, the fitted lane lines can be sorted from left to right according to the slope of each fitted lane line to obtain the sorting result of each fitted lane line. Figure 7 It can be seen that the slope increases from left to right, and the fitted lane lines are sorted in order from small to large slopes.

[0091] In another example, according to the center points of the fitted lane lines, the center points are sorted from left to right to obtain the sorting results of the fitted lane lines. Figure 8 As shown, the lane lines can be sorted from left to right along the center point of each lane line in the x-axis direction.

[0092] In some examples, in addition to using the slope size and center point sorting of the fitted lane lines, other methods can also be used for sorting. For example, by calculating the intersection of the lane lines and a reference line (such as a horizontal line at the bottom or top of the image), sorting is performed according to the horizontal coordinate of the intersection. This embodiment does not specifically limit the sorting method of the fitted lane lines.

[0093] Among them, the sorting result can be used to determine the sub-lane where the target vehicle is located (for example, in the above example, it is determined by the positive and negative values ​​of the slopes of adjacent lane lines in the sorting result, or in some embodiments, it is determined by the size of the slope of the lane line in the sorting result).

[0094] Next, in order to further improve the extraction accuracy of road vehicle signals, the road vehicle signals can be extracted in combination with the relative positions of the curb and the guardrail. Specifically, the recognition result of the map element includes at least one of the curb recognition result and the guardrail recognition result. The road vehicle signal corresponding to the sublane in the above step S102 can be extracted in the following manner:

[0095] In one example, the position of the curb in the road image can be obtained based on the curb recognition result to determine the relative position between the curb and the lane boundary; based on the relative position, the boundary periphery of the curb is determined; and invalid road vehicle signals located outside the boundary periphery of the boundary periphery of the curb are filtered out to obtain the road vehicle signals corresponding to the sub-lane.

[0096] For example, the identified curb position can be marked to determine its specific position in the image to ensure the accuracy of the curb position, and the curb can be linearly fitted (the fitting method is lane line fitting, which will not be described here) to reduce the amount of calculation. Then the relative position between the curb and the lane boundary is calculated. In this embodiment, the relative position of the curb and the lane boundary can include the left and right positions between the curb and the lane boundary, that is, whether the curb is on the left relative to the left lane boundary, or whether the curb is on the right relative to the right lane boundary. And the relative position is used to determine the boundary periphery of the curb. In this embodiment, the boundary periphery is the range of the curb away from the side of the vehicle, such as the curb located at the left lane boundary, and the range on the left side of the curb is the boundary periphery. The road vehicle signal on the left side of the curb is invalid for the vehicle. By filtering out this part of invalid road vehicle signal, the accuracy of the overall road vehicle signal can be effectively improved.

[0097] In another example, based on the guardrail recognition result, the position of the guardrail in the road image is obtained to determine the relative position between the guardrail and the lane boundary; based on the relative position, the boundary periphery of the guardrail is determined; and invalid road vehicle signals located outside the boundary periphery of the guardrail are filtered out to obtain the road vehicle signals corresponding to the sub-lane.

[0098] It should be noted that for the guardrail recognition results and filtering out invalid road vehicle signals outside the boundary of the guardrail, please refer to the above-mentioned curb example, and the relevant instructions will not be repeated here.

[0099] In some examples, the above-mentioned curb recognition results and guardrail recognition results can also be combined to filter out invalid road vehicle signals outside the curb boundary and the guardrail boundary. For relevant instructions, please refer to the above examples and no further details will be given here.

[0100] Next, the determination of the boundary periphery of the curb and / or the boundary periphery of the guardrail according to the relative position in the above example is further described:

[0101] In one example, determining the boundary periphery of the curb according to the relative position of the curb includes: determining the effective curb in the curb according to the relative position of the curb; wherein, the effective curb includes the curb located on the left side of the left lane corresponding to the lane boundary and the curb located on the right side of the right lane corresponding to the lane boundary; determining the boundary periphery of the curb by the range on the side of the effective curb away from the lane boundary.

[0102] As mentioned above, in the process of determining the sub-lane of the vehicle, the left and right lane boundaries of the sub-lane in which the current vehicle is traveling have been determined. To improve the accuracy of curb recognition, in this embodiment, the curb is first screened. If the curb (the center point of the curb can be used) satisfies curb[j].cx < lane[i].cx and curb[j].k < 0, then this curb is an effective left curb; otherwise, the recognition result of this curb is not credible and is discarded. (cx, cy) refers to the center point of the polygon such as the curb; similarly, if the center point of the curb satisfies curb[j].cx > lane[i].cx and curb[j].k > 0, then this curb is an effective right curb; otherwise, the recognition result of this curb is not credible and is discarded. Among them, the schematic diagram of the effective curb is as Figure 3 shown ( Figure 3 the shown curb is an effective curb).

[0103] In another example, determining the boundary periphery of the guardrail according to the relative position of the guardrail includes: determining the effective guardrail in the guardrail according to the relative position of the guardrail; wherein, the effective guardrail includes the guardrail located on the left side of the left lane corresponding to the lane boundary and the guardrail located on the right side of the right lane corresponding to the lane boundary; determining the boundary periphery of the guardrail by the range on the side of the effective guardrail away from the lane boundary.

[0104] Correspondingly, for the determination process of the effective guardrail, based on the left and right boundaries of the sub-lane in which the current vehicle is traveling, if the center point of the guardrail satisfies barrier[i].cx < barrier[i].cx and barrier[i].k < 0, then this guardrail is an effective left guardrail; otherwise, the recognition result of this guardrail is not credible and is discarded. (cx, cy) refers to the center point of the polygon; similarly, if the center point of the guardrail satisfies barrier[i].cx > barrier[i].cx and barrier[i].k > 0, then this guardrail is an effective right guardrail; otherwise, the recognition result of this guardrail is not credible and is discarded. Among them, the schematic diagram of the effective guardrail is as Figure 4 shown ( Figure 4 the shown one in the figure is an effective guardrail after transposition).

[0105] In some examples, the above two examples may be combined to jointly determine the perimeter of the curb and guardrail boundary, and the relevant description will not be repeated here.

[0106] Through the above technical solution, by identifying effective curbs and / or effective guardrails, the accuracy of the boundary periphery of the curbs and / or guardrails can be improved, thereby improving the filtering accuracy of invalid road vehicle signals.

[0107] In some embodiments, the vehicle information and road arrows on the lane are detected by a vehicle detection model and a road arrow detection model to further improve the extraction accuracy of road vehicle information. Specifically, the method may also include at least one of the following:

[0108] Based on a vehicle detection model, the vehicle in the road image is detected to obtain a vehicle recognition result; wherein the vehicle detection model is a model trained based on historical vehicle data; based on a road arrow detection model, the road arrow in the road image is detected to obtain an arrow recognition result; wherein the road arrow detection model is a model trained based on historical road arrow data;

[0109] The above-mentioned step S102 extracts the road vehicle information corresponding to the sub-lane, specifically: according to at least one of the vehicle recognition result and the arrow recognition result, extracts the road vehicle information corresponding to the sub-lane, and the road vehicle information includes the vehicle and / or road arrow corresponding to the sub-lane.

[0110] It should be noted that the above model training process and recognition process can refer to the above lane line model training and recognition process, and the relevant description will not be repeated here. Correspondingly, the vehicle detection model and the road arrow detection model can also use the same model to simultaneously recognize vehicles and arrows in the road image, or more elements.

[0111] Fig. 9 This is the recognition result obtained by detecting and identifying road arrows through the target detection algorithm, such as Fig. 9 From the visual recognition results, we can see that the recognition results include the current lane arrow and the non-current lane arrow. The arrow on the left side of the guardrail is the non-current lane arrow. Fig.10 It is the recognition result of detecting and identifying the vehicle through the target detection algorithm, such as Fig.10 From the visual recognition results, we can see that the recognition results include vehicles in the current lane as well as vehicles in non-current lanes. The vehicle on the left side of the guardrail is not a vehicle in the current lane.

[0112] Taking the relative position between the guardrail, curb and lane boundary as an example, non-current targets are filtered by the curb and guardrail: traverse all arrows and vehicle recognition results, and calculate the target center point obj(cx,cy) (the calculation method of the center point is convenient for calculation); if the slope k of the current curb and guardrail is <0, then the guardrail and curb belong to the left segmentation element (that is, the element to the left of the left lane boundary), and the linear fitting line expression of the current segmentation element is y=kx+b, and a point on the line is (point_x, point_y). Target point y_res=k*(point_x-cx)+cy-point_y

[0113] If y_res<0, the target is to the left of the left guardrail (left curb), and the target should be filtered out.

[0114] Correspondingly, if the slope k of the current curb and guardrail is greater than 0, the guardrail and curb belong to the right segmentation element. Suppose the linear fitting line expression of the current segmentation element is y=kx+b; suppose a point on the line is (point_x, point_y), the target point y_res=k*(point_x-cx)+cy-point_y, if y_res>0, the target is to the right of the right guardrail (right curb), and the target should be filtered out.

[0115] Among them, the ground arrows (i.e. road arrows) and vehicle recognition results after filtering out the relative position of the guardrail are as follows: Fig.11a As shown in the figure, the ground arrows and vehicle detection results after the curb filtering are as follows: Fig.11b shown.

[0116] Through the above technical solution, the vehicle detection model and the road arrow detection model are used to detect the vehicle information and road arrows on the lane, which can further improve the extraction accuracy of road vehicle information.

[0117] In an exemplary embodiment, in combination Fig.12 As shown, the method for extracting road vehicle information includes the following steps:

[0118] S1. Get the current lane image;

[0119] S2. The ground arrow detection model (i.e., the road surface detection model) obtains the position and corresponding category of the ground arrow (i.e., the road surface arrow), and the center point P (arrow_cx, arrow_cy) of the ground arrow;

[0120] S3, the vehicle detection model obtains the location and corresponding category of the vehicle, and the center point P (vehicle_cx, vehicle_cy) of the vehicle;

[0121] S4, the curb segmentation model (i.e., the curb detection model) obtains a curb segmentation result (i.e., a curb recognition result);

[0122] S5. The guardrail segmentation model (i.e., the guardrail detection model) obtains the guardrail segmentation result (i.e., the guardrail recognition result);

[0123] S6. The lane line segmentation model (i.e., the lane line detection model) obtains the lane line recognition result;

[0124] S7. Perform straight line fitting on the lane line to obtain the expression y=kx+b of the lane line and the center point P (lane_cx, lane_cy) of the lane line;

[0125] S8. Sort the lane lines from left to right using the center points of the lane lines;

[0126] S9, using the lane line slope to calculate and determine the sub-lane position where the current vehicle is located;

[0127] S10, determining the relative positions of current curbs, guardrails and other elements through the lane line where the sublane is located;

[0128] S11, perform linear fitting on elements such as curbs and guardrails, calculate the y=kx+b expression of the element, and the center point P (obj_cx, obj_cy) of each segmentation element;

[0129] S12, filtering out elements such as ground arrows and vehicles that are not on the current road through the y=kx+b expression of elements such as curbs and guardrails, as well as the center points and relative positions of the elements;

[0130] S13, obtaining the final recognition result of the ground arrows and vehicles on the current road, that is, the final extracted road vehicle information.

[0131] Through the above technical solution, the current sub-lane recognition technology is introduced to assist in determining the relative positions of elements such as guardrails and curbs, and further filter out ground arrows and vehicles that are not on the current road, thereby improving the recognition accuracy of vehicle information. In addition, based on the lane segmentation principle, the lane line slope is calculated, and the lane lines are sorted in order, which can effectively determine the lane lines around the current sub-lane. By determining the lane lines around the current sub-lane, the relative positions of elements such as diversion areas, guardrails, and curbs are further assisted in determining, so that the relative position of each lane can be clearly known. In addition, the entire process does not require human participation, which reduces the cost of manual quality inspection, achieves cost reduction and efficiency improvement, and improves the overall map vehicle information update accuracy.

[0132] Fig.13 A schematic diagram of a road vehicle information extraction device provided in an embodiment of the present application is shown in FIG. Fig.13As shown, the device 130 includes an element recognition module 131 and a determination and extraction module 132, wherein:

[0133] An element recognition module 131 is configured to perform element recognition on a current road image of a target vehicle according to an element detection model corresponding to a map element, and obtain a recognition result of the map element, wherein the recognition result of the map element includes a lane line recognition result;

[0134] A determination and extraction module 132, which is configured to determine the lane boundary of the target vehicle's driving lane according to the lane line recognition result, and determine the sub-lane where the target vehicle is located according to the lane boundary, so as to extract the road vehicle information corresponding to the sub-lane;

[0135] The road vehicle information is used to indicate the vehicle driving road surface information of the target vehicle.

[0136] In a possible implementation, the determination and extraction module 132 includes:

[0137] A lane line fitting module, which is configured to perform straight line fitting on lane lines in the same direction according to the lane line recognition result to obtain multiple fitted lane lines;

[0138] The boundary determination module is configured to determine the left lane boundary and the right lane boundary of the target vehicle according to the lane line slopes of each fitted lane line, so as to obtain the sub-lane where the target vehicle is located.

[0139] In a possible implementation manner, the boundary determination module is specifically configured to:

[0140] For each fitted lane line, in the sorting result based on the left-right sorting, if the lane line slopes of the adjacent lane lines have opposite positive and negative values, the adjacent lane lines are determined to be the left lane boundary and the right lane boundary of the vehicle;

[0141] Among them, the lane line with a negative lane line slope among the adjacent lane lines is the left lane boundary, and the lane line with a positive lane line slope is the right lane boundary.

[0142] In a possible implementation, the device further includes at least one of the following:

[0143] A first sorting module is configured to sort the fitted lane lines from left to right according to the slopes of the fitted lane lines to obtain sorting results of the fitted lane lines;

[0144] A second sorting module is configured to sort the center points of the fitted lane lines from left to right according to the center points of the fitted lane lines to obtain a sorting result of the fitted lane lines;

[0145] The sorting result is used to determine the sub-lane where the target vehicle is located.

[0146] In a possible implementation, the recognition result of the map element includes at least one of a curb recognition result and a guardrail recognition result; and the determination and extraction module includes:

[0147] a first relative position determination unit, configured to obtain the position of the curb in the road image according to the curb recognition result, so as to determine the relative position between the curb and the lane boundary;

[0148] A first periphery determination unit, configured to determine a boundary periphery of the curb according to a relative position of the curb;

[0149] A first filtering unit, configured to filter out invalid road vehicle signals located outside the boundary of the curb to obtain the road vehicle signals corresponding to the sub-lane;

[0150] In a possible implementation, the determination and extraction module includes:

[0151] a second relative position determination unit, configured to obtain the position of the guardrail in the road image according to the guardrail recognition result, so as to determine the relative position between the guardrail and the lane boundary;

[0152] a second perimeter determination unit configured to determine a boundary perimeter of the guardrail according to a relative position of the guardrail;

[0153] The second filtering unit is configured to filter out invalid road vehicle signals located outside the boundary of the guardrail to obtain the road vehicle signals corresponding to the sub-lane.

[0154] In a possible implementation manner, the first peripheral determination unit is specifically configured to:

[0155] Determining effective curbs among the curbs according to the relative positions of the curbs; wherein the effective curbs include curbs located on the left side of the left lane corresponding to the lane boundary, and curbs located on the right side of the right lane corresponding to the lane boundary;

[0156] The effective curb is separated from the lane boundary by a range to determine the outer boundary of the curb.

[0157] In a possible implementation manner, the second peripheral determination unit includes:

[0158] Determine, according to the relative positions of the guardrails, effective guardrails among the guardrails; wherein the effective guardrails include a guardrail located on the left side of a left lane corresponding to the lane boundary, and a guardrail located on the right side of a right lane corresponding to the lane boundary;

[0159] The effective guardrail is moved away from the lane boundary to determine the periphery of the guardrail boundary.

[0160] In a possible implementation, the feature detection model is a model trained based on each type of historical map feature data, and includes at least one of the following: a lane detection model, a guardrail detection model, and a curb detection model;

[0161] The element identification module 131 includes at least one of the following:

[0162] A lane line recognition unit, which is configured to detect lane lines in the road image based on the lane line detection model to obtain a lane line recognition result;

[0163] a guardrail recognition unit, configured to detect the guardrail in the road image based on the guardrail detection model to obtain a guardrail recognition result;

[0164] A curb recognition unit is configured to detect curbs in the road image based on the curb detection model to obtain a curb recognition result.

[0165] In a possible implementation, the device further includes at least one of the following:

[0166] Based on a vehicle detection model, the vehicle in the road image is detected to obtain a vehicle recognition result; wherein the vehicle detection model is a model trained based on historical vehicle data;

[0167] Based on a road arrow detection model, detecting road arrows in the road image to obtain arrow recognition results; wherein the road arrow detection model is a model trained based on historical road arrow data;

[0168] The determination and extraction module 132 includes:

[0169] A vehicle signal extraction unit is configured to extract a road vehicle signal corresponding to the sub-lane based on at least one of the vehicle recognition result and the arrow recognition result, wherein the road vehicle signal includes a vehicle and / or a road arrow corresponding to the sub-lane.

[0170] The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the method embodiments of the present application, and no further elaboration is made here.

[0171] The present application also provides an electronic device, such as Fig.14 As shown, the electronic device includes: a processor 141, and a memory 142 communicatively connected to the processor 141;

[0172] The memory 142 stores computer-executable instructions;

[0173] The processor 141 executes the computer-executable instructions stored in the memory 142 to implement the road vehicle information extraction method, wherein the memory 142 and the processor 141 are connected via a bus 143.

[0174] The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the method embodiments of the present application, and no further elaboration is made here.

[0175] The embodiment of the present application also provides a chip, including a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory to execute the road vehicle information extraction method.

[0176] The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the method embodiments of the present application, and no further elaboration is made here.

[0177] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the road vehicle information extraction method provided in any embodiment of the method embodiment of the present application.

[0178] Among them, the computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0179] The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the method embodiments of the present application, and no further elaboration is made here.

[0180] An embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the road vehicle information extraction method provided by the method embodiment of the present application.

[0181] The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the method embodiments of the present application, and no further elaboration is made here.

[0182] Fig.15 It is a block diagram of a terminal device shown in an exemplary embodiment of the present application. The terminal device 800 can be a mobile phone, a computer, a digital broadcast terminal, a message transceiver device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0183] The terminal device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0184] The processing component 802 generally controls the overall operation of the terminal device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0185] The memory 804 is configured to store various types of data to support operations on the terminal device 800. Examples of such data include instructions for any application or method operating on the terminal device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0186] The power supply component 806 provides power to various components of the terminal device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the terminal device 800.

[0187] The multimedia component 808 includes a screen that provides an output interface between the terminal device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the terminal device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0188] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the terminal device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0189] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0190] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the terminal device 800. For example, the sensor assembly 814 can detect the open / closed state of the terminal device 800, the relative positioning of the components, such as the display and keypad of the terminal device 800, and the sensor assembly 814 can also detect the position change of the terminal device 800 or a component of the terminal device 800, the presence or absence of contact between the user and the terminal device 800, the orientation or acceleration / deceleration of the terminal device 800, and the temperature change of the terminal device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0191] The communication component 816 is configured to facilitate wired or wireless communication between the terminal device 800 and other devices. The terminal device 800 can access a wireless network based on a communication standard, such as WiFi, 3G, 4G, 5G or other standard communication networks, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0192] In an exemplary embodiment, the terminal device 800 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the method provided in any of the above-mentioned method embodiments of the present application.

[0193] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by a processor 820 of a terminal device 800 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0194] An embodiment of the present application also provides a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by the processor of the terminal device, the terminal device 800 can execute the method provided by any one of the above-mentioned method embodiments of the present application.

[0195] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0196] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0197] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for extracting road vehicle information, characterized in that: include: According to the feature detection model corresponding to the map feature, the feature recognition is performed on the current road image of the target vehicle to obtain the recognition result of the map feature, wherein the recognition result of the map feature includes the lane line recognition result; Determine the lane boundary of the target vehicle's driving lane according to the lane line recognition result, and determine the sub-lane where the target vehicle is located according to the lane boundary to extract the road vehicle information corresponding to the sub-lane; The road vehicle information is used to indicate the vehicle driving road surface information of the target vehicle.

2. The method according to claim 1, characterized in that The determining, according to the lane line recognition result, the sub-lane in which the vehicle is located includes: According to the lane line recognition result, linear fitting is performed on the lane lines in the same direction to obtain multiple fitted lane lines; According to the lane line slopes of each fitted lane line, the left lane boundary and the right lane boundary of the target vehicle are determined to obtain the sub-lane where the target vehicle is located.

3. The method according to claim 2, characterized in that Determining the left lane boundary and the right lane boundary of the vehicle according to the lane line slope includes: For each fitted lane line, in the sorting result based on the left-right sorting, if the lane line slopes of the adjacent lane lines have opposite positive and negative values, the adjacent lane lines are determined to be the left lane boundary and the right lane boundary of the vehicle; Among them, the lane line with a negative lane line slope among the adjacent lane lines is the left lane boundary, and the lane line with a positive lane line slope is the right lane boundary.

4. The method according to claim 2 or 3, characterized in that: Also includes at least one of the following: According to the slope of each fitted lane line, the fitted lane lines are sorted from left to right to obtain the sorting results of the fitted lane lines; According to the center points of the fitted lane lines, the center points are sorted from left to right to obtain the sorting results of the fitted lane lines; The sorting result is used to determine the sub-lane where the target vehicle is located.

5. The method according to claim 1, characterized in that The recognition result of the map element includes at least one of a curb recognition result and a guardrail recognition result; and the extracting of the road vehicle information corresponding to the sublane includes: Acquire the position of the curb in the road image according to the curb recognition result to determine the relative position between the curb and the lane boundary; Determining the perimeter of the curb according to the relative position of the curb; Filter out invalid road vehicle signals located outside the boundary of the curb to obtain road vehicle signals corresponding to the sub-lane; and / or, Acquiring the position of the guardrail in the road image according to the guardrail recognition result to determine the relative position between the guardrail and the lane boundary; Determining the perimeter of the boundary of the guardrail according to the relative position of the guardrail; Invalid road vehicle signals located outside the boundary of the guardrail are filtered out to obtain the road vehicle signals corresponding to the sub-lane.

6. The method according to claim 5, characterized in that The step of determining the perimeter of the curb according to the relative position of the curb comprises: Determining effective curbs among the curbs according to the relative positions of the curbs; wherein the effective curbs include curbs located on the left side of the left lane corresponding to the lane boundary, and curbs located on the right side of the right lane corresponding to the lane boundary; The effective curb is away from the lane boundary, and the outer boundary of the curb is determined; and / or, Determining the periphery of the boundary of the guardrail according to the relative position of the guardrail includes: Determine, according to the relative positions of the guardrails, effective guardrails among the guardrails; wherein the effective guardrails include a guardrail located on the left side of a left lane corresponding to the lane boundary, and a guardrail located on the right side of a right lane corresponding to the lane boundary; The effective guardrail is moved away from the lane boundary to determine the periphery of the guardrail boundary.

7. The method according to claim 1, characterized in that The feature detection model is a model trained based on each type of historical map feature data, including at least one of the following: a lane detection model, a guardrail detection model, and a curb detection model; The performing element recognition on the current road image of the vehicle to obtain a recognition result of the map element corresponding to the road image includes at least one of the following: Based on the lane line detection model, detect the lane lines in the road image to obtain a lane line recognition result; Based on the guardrail detection model, detect the guardrail in the road image to obtain a guardrail recognition result; Based on the curb detection model, curbs in the road image are detected to obtain curb recognition results.

8. The method according to any one of claims 1 to 7, characterized in that: Also includes at least one of the following: Based on a vehicle detection model, the vehicle in the road image is detected to obtain a vehicle recognition result; wherein the vehicle detection model is a model trained based on historical vehicle data; Based on a road arrow detection model, detecting road arrows in the road image to obtain arrow recognition results; wherein the road arrow detection model is a model trained based on historical road arrow data; The extracting the road vehicle information corresponding to the sub-lane includes: According to at least one of the vehicle recognition result and the arrow recognition result, a road vehicle signal corresponding to the sub-lane is extracted, where the road vehicle signal includes a vehicle and / or a road arrow corresponding to the sub-lane.

9. A road vehicle information extraction device, characterized in that: include: An element recognition module is configured to perform element recognition on a current road image of a target vehicle according to an element detection model corresponding to a map element, and obtain a recognition result of the map element, wherein the recognition result of the map element includes a lane line recognition result; A determination and extraction module, which is configured to determine the lane boundary on the lane where the target vehicle is traveling according to the lane line recognition result, and determine the sub-lane where the target vehicle is located according to the lane boundary, so as to extract the road vehicle information corresponding to the sub-lane; The road vehicle information is used to indicate the vehicle driving road surface information of the target vehicle.

10. An electronic device / computer-readable storage medium / computer program product, characterized in that: The electronic device comprises: a memory and a processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the road vehicle information extraction method according to any one of claims 1 to 8; and / or The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the road vehicle information extraction method according to any one of claims 1 to 8; and / or The computer program product comprises a computer program, which, when executed by a processor, implements the road vehicle information extraction method as described in any one of claims 1 to 8.

Images