Speed limit sign and lane matching method and device, and vehicle

Abstract

The application relates to the field of vehicles and provides a speed limit sign and lane matching method and device and a vehicle. The method comprises the following steps: detecting whether a speed limit sign exists in a collected road image; if the speed limit sign exists in the road image, the speed limit sign is cropped to obtain a speed limit sign image, and the speed limit type of the speed limit sign image is judged; the speed limit value in the speed limit sign image is recognized, the number of reliable speed limit sign images is determined according to the speed limit value; and the speed limit type and the speed limit value corresponding to the reliable speed limit sign image are matched to the corresponding lane according to the number of reliable speed limit sign images. The speed limit sign and lane matching method provided in the application can match the speed limit type and the speed limit value corresponding to multiple reliable speed limit sign images to the corresponding lane according to the number of the specific speed limit sign images, accurately judges the speed limit sign corresponding to the current lane in a scene with multiple lanes and multiple speed limit signs, and avoids traffic accidents.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicles, in particular to a speed limit sign and lane matching method and device and vehicle. BACKGROUND

[0002] With the acceleration of urbanization process in China, the explosive growth of the number of cars, the urban congestion problem is also increasingly serious, accompanied by frequent traffic accidents, noise pollution and air pollution, so that the urban carrying capacity and social operation efficiency have been severely challenged. Traffic sign recognition system as an important part of intelligent transportation system, is widely concerned by domestic and foreign scholars. Most of the major traffic accidents are caused by overspeed, and accurate speed limit sign recognition can provide protection for the successful construction of intelligent transportation system.

[0003] The common traffic speed limit sign recognition method only recognizes a single speed limit sign, which defaults that all lanes match a single speed limit sign. However, in actual road scenes, the road conditions are extremely complex, and there are multiple traffic speed limit signs due to multiple lanes. The placement forms of speed limit signs are various, and the method of recognizing only a single speed limit sign cannot match multiple speed limit signs with multiple lanes one by one, so it cannot accurately determine the corresponding speed limit sign of the current lane in the scene of multiple lanes and multiple speed limit signs, which is easy to cause traffic accidents. SUMMARY

[0004] The embodiments of the present application provide a speed limit sign and lane matching method, device and vehicle, to solve the technical problem that the corresponding speed limit sign of the current lane cannot be accurately determined in the scene of multiple lanes and multiple speed limit signs, which is easy to cause traffic accidents.

[0005] In a first aspect, the embodiments of the present application provide a speed limit sign and lane matching method, comprising:

[0006] Detecting whether there is a speed limit sign in the collected road image;

[0007] If there is a speed limit sign in the road image, the speed limit sign is cropped to obtain a speed limit sign image, and the speed limit type of the speed limit sign image is judged; the speed limit type includes the maximum speed limit, the minimum speed limit and the speed limit removal;

[0008] Recognizing the speed limit value in the speed limit sign image, and determining the number of reliable speed limit sign images according to the speed limit value;

[0009] According to the number of reliable speed limit sign images, the speed limit type and the speed limit value corresponding to the reliable speed limit sign image are matched to the corresponding lane.

[0010] In one embodiment, the matching the speed limit type and the speed limit value corresponding to the reliable speed limit sign image to the corresponding lane according to the number of reliable speed limit sign images comprises:

[0011] If the number of reliable speed limit sign images is equal to 1, the speed limit type and the speed limit value corresponding to the reliable speed limit sign image are matched to the whole lane.

[0012] In one embodiment, the matching the speed limit type and the speed limit value corresponding to the reliable speed limit sign image to the corresponding lane according to the number of reliable speed limit sign images comprises:

[0013] If the number of reliable speed limit sign images is equal to 2, and the speed limit type corresponding to one of the reliable speed limit sign images is the maximum speed limit, and the speed limit type corresponding to the other reliable speed limit sign image is the minimum speed limit, the speed limit type and the speed limit value corresponding to the reliable speed limit sign image are matched to the whole lane.

[0014] If the number of reliable speed limit sign images is equal to 2, and the speed limit type corresponding to both of the reliable speed limit sign images is the maximum speed limit, the number of lanes is determined to be 2, and the speed limit type and the speed limit value corresponding to the two reliable speed limit sign images are matched to the corresponding lanes.

[0015] In one embodiment, the matching the speed limit type and the speed limit value corresponding to the reliable speed limit sign image to the corresponding lane according to the number of reliable speed limit sign images comprises:

[0016] If the number of reliable speed limit sign images is N, and the speed limit type corresponding to each of the N reliable speed limit sign images is the maximum speed limit, the number of lanes is determined to be N, and the speed limit type and the speed limit value corresponding to each of the reliable speed limit sign images are matched to the corresponding lanes; wherein N is an integer greater than 2.

[0017] If the number of reliable speed limit sign images is N, and the speed limit type corresponding to one of the reliable speed limit sign images is the minimum speed limit, and the speed limit type corresponding to the remaining N-1 reliable speed limit sign images is the maximum speed limit, the number of lanes is determined to be N-1, the speed limit type and the speed limit value corresponding to the one reliable speed limit sign image are matched to the whole lane, and the speed limit type and the speed limit value corresponding to the remaining N-1 reliable speed limit sign images are matched to the corresponding lanes.

[0018] If the number of the reliable speed limit sign images is 2M, and the speed limit types corresponding to M reliable speed limit sign images are the highest speed limit, and the speed limit types corresponding to the other M reliable speed limit sign images are the lowest speed limit, the number of lanes is determined as M, and the reliable speed limit sign images with the highest speed limit type and the reliable speed limit sign images with the lowest speed limit type are matched, and the speed limit type and the speed limit value are matched to the corresponding lane according to the matching result; wherein M is an integer greater than 1.

[0019] In one embodiment, the matching of the reliable speed limit sign images with the highest speed limit type and the reliable speed limit sign images with the lowest speed limit type, and the matching of the speed limit type and the speed limit value to the corresponding lane according to the matching result, comprises:

[0020] A two-dimensional rectangular coordinate system is established in the road image; the Y-axis of the two-dimensional rectangular coordinate system is vertically downward;

[0021] The reliable speed limit sign images in the road image are grouped; wherein the absolute value of the difference of the image abscissa of any two reliable speed limit sign images in each group is within a threshold interval;

[0022] The average value of the image abscissa of the reliable speed limit sign images in each group is determined as a small group abscissa, and all groups are arranged in ascending order of the small group abscissa to obtain a first sequence;

[0023] The reliable speed limit sign images in each group in the first sequence are arranged in ascending order of the image ordinate to obtain a second sequence;

[0024] If the speed limit type corresponding to the second reliable speed limit sign image in the second sequence is the lowest speed limit, the Kth reliable speed limit sign image and the K+1th reliable speed limit sign image are matched, and the speed limit type and the speed limit value corresponding to the Kth reliable speed limit sign image and the speed limit type and the speed limit value corresponding to the K+1th reliable speed limit sign image are matched to the Kth lane; wherein K is an odd number greater than or equal to 1 and less than 2M. Lane; wherein K is an odd number greater than or equal to 1 and less than 2M.

[0025] In one embodiment, after the reliable speed limit sign images in each group in the first sequence are arranged in ascending order of the image ordinate to obtain a second sequence, it further comprises:

[0026] if the speed limit type corresponding to the second reliable speed limit sign image in the second sequence is the highest speed limit, then traversing the reliable speed limit sign images after the second reliable speed limit sign image in the second sequence to obtain a third sequence; the third sequence is an ordered sequence composed of M reliable speed limit sign images with the lowest speed limit type;

[0027] matching the Lth reliable speed limit sign image in the second sequence with the Lth reliable speed limit sign image in the third sequence, and matching the speed limit type and the speed limit value corresponding to the Lth reliable speed limit sign image in the second sequence and the speed limit type and the speed limit value corresponding to the Lth reliable speed limit sign image in the third sequence to the Lth lane; wherein L is an integer greater than or equal to 1 and less than or equal to M.

[0028] In one embodiment, after the reliable speed limit sign image in each group in the first sequence is arranged in ascending order of image longitudinal coordinate to obtain a second sequence, the method further comprises:

[0029] if the speed limit type corresponding to the second reliable speed limit sign image in the second sequence is the highest speed limit, then matching the Rth reliable speed limit sign image in the second sequence with the M+Rth reliable speed limit sign image in the second sequence, and matching the speed limit type and the speed limit value corresponding to the Rth reliable speed limit sign image in the second sequence and the speed limit type and the speed limit value corresponding to the M+Rth reliable speed limit sign image in the second sequence to the Rth lane; wherein R is an integer greater than or equal to 1 and less than or equal to M.

[0030] In one embodiment, the determining the number of reliable speed limit sign images according to the speed limit value comprises:

[0031] if the speed limit value is greater than the confidence threshold, then determining the speed limit sign image corresponding to the speed limit value as a reliable speed limit sign image;

[0032] adding up the reliable speed limit sign images to determine the number of reliable speed limit sign images.

[0033] In one embodiment, after the speed limit type and the speed limit value corresponding to the reliable speed limit sign image are matched to the corresponding lane, the method further comprises:

[0034] if there is a matched highest speed limit in the current lane and the current vehicle speed is greater than the highest speed limit, then starting vehicle voice warning and screen display to prompt the driver that the speed has been exceeded.

[0035] In a second aspect, the embodiments of the present application provide a speed limit sign and lane matching device, comprising:

[0036] The speed limit sign detection module is configured to detect whether a speed limit sign exists in the collected road image.

[0037] The speed limit sign image acquisition and type judgment module is configured to, if the speed limit sign exists in the road image, crop the speed limit sign to obtain a speed limit sign image, and judge a speed limit type of the speed limit sign image; the speed limit type includes a maximum speed limit, a minimum speed limit, and a speed limit removal.

[0038] The reliable speed limit sign image acquisition module is configured to identify a speed limit value in the speed limit sign image, and determine a number of reliable speed limit sign images according to the speed limit value.

[0039] The reliable speed limit sign and lane matching module is configured to match the speed limit type and the speed limit value corresponding to the reliable speed limit sign image to a corresponding lane according to the number of reliable speed limit sign images.

[0040] In a third aspect, an embodiment of the present application provides a vehicle including the speed limit sign and lane matching device described above.

[0041] The speed limit sign and lane matching method, device, and vehicle provided by the embodiments of the present application first detect whether a speed limit sign exists in a road image, if the speed limit sign exists, crop the speed limit sign to obtain a speed limit sign image, judge a speed limit type of the speed limit sign image, identify a speed limit value in the speed limit sign image, determine a number of reliable speed limit sign images according to the speed limit value, and finally match the speed limit type and the speed limit value corresponding to the reliable speed limit sign image to a corresponding lane according to the number of reliable speed limit sign images. Since the reliable speed limit sign image obtained from the road image can be one or multiple, when the reliable speed limit sign image is one, it means that the speed limit type and the speed limit value corresponding to the reliable speed limit sign image are applicable to all lanes, and when the reliable speed limit sign image is multiple, the speed limit type and the speed limit value corresponding to the multiple reliable speed limit sign images can be matched to corresponding lanes according to the number of the reliable speed limit sign images, so as to accurately judge the speed limit sign corresponding to the current lane in the scene of multiple lanes and multiple speed limit signs, and avoid traffic accidents. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative effort.

[0043] Figure 1is one of flowcharts of a speed limit sign and lane matching method provided by an embodiment of the present application;

[0044] Figure 2 is a schematic diagram of the speed limit sign and lane matching method provided by an embodiment of the present application when the reliable speed limit sign image is one;

[0045] Figure 3 is one of schematic diagrams of the speed limit sign and lane matching method provided by an embodiment of the present application when the reliable speed limit sign image is two;

[0046] Figure 4 is a schematic diagram of the speed limit sign and lane matching method provided by an embodiment of the present application when the reliable speed limit sign image is two;

[0047] Figure 5 is a schematic diagram of the speed limit sign and lane matching method provided by an embodiment of the present application when the reliable speed limit sign image is two;

[0048] Figure 6 is a schematic diagram of the speed limit sign and lane matching method provided by an embodiment of the present application when the reliable speed limit sign image is two;

[0049] Figure 7 is one of schematic diagrams of the speed limit sign and lane matching method provided by an embodiment of the present application when the reliable speed limit sign image is more than two;

[0050] Figure 8 is a schematic diagram of the speed limit sign and lane matching method provided by an embodiment of the present application when the reliable speed limit sign image is more than two;

[0051] Figure 9 is a schematic diagram of the speed limit sign and lane matching method provided by an embodiment of the present application when the reliable speed limit sign image is more than two;

[0052] Figure 10 is a schematic diagram of the speed limit sign and lane matching method provided by an embodiment of the present application when the reliable speed limit sign image is more than two;

[0053] Figure 11 is a schematic diagram of the speed limit sign and lane matching method provided by an embodiment of the present application when the reliable speed limit sign image is more than two;

[0054] Figure 12 is a schematic diagram of the speed limit sign and lane matching method provided by an embodiment of the present application when the reliable speed limit sign image is more than two;

[0055] Figure 13 is a schematic diagram of the speed limit sign and lane matching method provided by an embodiment of the present application when the reliable speed limit sign image is more than two;

[0056] Figure 14 Fig. 8 is a schematic view of eight reliable speed limit sign images when the number of reliable speed limit sign images is greater than two in the speed limit sign and lane matching method provided by the embodiments of the present application;

[0057] Figure 15 Fig. 2 is a flowchart of the speed limit sign and lane matching method provided by the embodiments of the present application.

[0058] Figure 16 Fig. 3 is a structural schematic view of the speed limit sign and lane matching device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0059] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0060] Figure 1 Fig. 1 is a flowchart of the speed limit sign and lane matching method provided by the embodiments of the present application. With reference to Fig. 1, the embodiments of the present application provide a speed limit sign and lane matching method, which can include the following steps. Figure 1

[0061] 101. Detecting whether there is a speed limit sign in a collected road image.

[0062] The speed limit sign in the collected road image can be detected by using a convolutional neural network. The road image can be an image obtained by converting a front road video collected by a camera configured on a vehicle in real time.

[0063] 102. If there is a speed limit sign in the road image, the speed limit sign is cropped to obtain a speed limit sign image, and the speed limit type of the speed limit sign image is determined.

[0064] The speed limit type includes the maximum speed limit, the minimum speed limit and the speed limit removal.

[0065] The speed limit sign is cropped, that is, the region corresponding to each speed limit sign is cropped from the road image.

[0066] 103. Identifying the speed limit value in the speed limit sign image, and determining the number of reliable speed limit sign images according to the speed limit value.

[0067] ​The speed limit value in the speed limit sign image can be recognized by using a convolutional neural network. The speed limit sign image is input into the convolutional neural network, the characters in the speed limit sign image are acquired, and the corresponding characters are arranged in ascending order according to the sum of the horizontal coordinates and the vertical coordinates, so as to form the speed limit value. For example, the characters in a certain speed limit sign image are recognized as "3" and "0" by using the convolutional neural network. The sum of the horizontal coordinates and the vertical coordinates of "3" is A, and the sum of the horizontal coordinates and the vertical coordinates of "0" is B. If A is less than B, the speed limit value in the speed limit sign image is 30.

[0068] It should be noted that the speed limit type can also be other speed limit types, which are not limited herein. In this embodiment, the speed limit type is the maximum speed limit, the minimum speed limit, and the speed limit removal.

[0069] 104. According to the number of reliable speed limit sign images, the speed limit type and the speed limit value corresponding to the reliable speed limit sign image are matched to the corresponding lane.

[0070] The speed limit sign and lane matching method provided in this embodiment first detects whether there is a speed limit sign in the road image. If there is a speed limit sign, the speed limit sign is cropped to obtain a speed limit sign image, and the speed limit type of the speed limit sign image is judged. Then, the speed limit value in the speed limit sign image is recognized, the number of reliable speed limit sign images is determined according to the speed limit value, and finally the speed limit type and the speed limit value corresponding to the reliable speed limit sign image are matched to the corresponding lane according to the number of reliable speed limit sign images. Since the reliable speed limit sign image obtained from the road image can be one or multiple, when the reliable speed limit sign image is one, it means that the speed limit type and the speed limit value corresponding to the reliable speed limit sign image are applicable to the whole lane. When the reliable speed limit sign image is multiple, the speed limit type and the speed limit value corresponding to the multiple reliable speed limit sign images can be matched to the corresponding lane according to the number of the reliable speed limit sign images, so as to accurately determine the speed limit sign corresponding to the current lane in the scene of multiple lanes and multiple speed limit signs, and avoid traffic accidents.

[0071] Figure 2 FIG. 1 is a schematic diagram of the speed limit sign and lane matching method provided in this embodiment when the reliable speed limit sign image is one. Referring to FIG. 1, Figure 2 In one embodiment, according to the number of reliable speed limit sign images, the speed limit type and the speed limit value corresponding to the reliable speed limit sign image are matched to the corresponding lane, which can include:

[0072] If the number of reliable speed limit sign images is equal to 1, the speed limit type and the speed limit value corresponding to the reliable speed limit sign image are matched to the whole lane.

[0073] When the number of reliable speed limit sign images is equal to 1, that is, there is only one speed limit sign in the front road, the speed limit type and speed limit value corresponding to the reliable speed limit sign image are matched to all lanes regardless of the number of lanes, so the speed limit type and speed limit value corresponding to the reliable speed limit sign image can be matched to all lanes.

[0074] In the embodiment, when the number of reliable speed limit sign images is 1, the speed limit type and speed limit value corresponding to the reliable speed limit sign image are matched to all lanes, which conforms to the actual situation that when there is only one speed limit sign in the front road, the speed limit sign is applicable to all lanes, and the speed limit sign and the lane are accurately matched.

[0075] The following describes the case that the number of reliable speed limit sign images is greater than or equal to 2. It should be noted that according to the setting of the speed limit sign on the actual road, when the number of reliable speed limit sign images divided in the same road image is greater than or equal to 2, the following limitation conditions exist:

[0076] 1. The speed limit release sign image must appear alone and cannot appear simultaneously with the highest speed limit sign image or the lowest speed limit sign image.

[0077] 2. The appearance of the lowest speed limit sign image is a sufficient condition for the appearance of the highest speed limit sign image, that is, if there is a lowest speed limit sign image in the same road image, there must be a highest speed limit sign image, but the existence of the highest speed limit sign image does not necessarily mean the existence of the lowest speed limit sign image.

[0078] Figure 3 is one of the schematic diagrams of the case that the number of reliable speed limit sign images is two in the speed limit sign and lane matching method provided by the embodiment of the application;

[0079] Figure 4 is the second schematic diagram of the case that the number of reliable speed limit sign images is two in the speed limit sign and lane matching method provided by the embodiment of the application;

[0080] Figure 5 is the third schematic diagram of the case that the number of reliable speed limit sign images is two in the speed limit sign and lane matching method provided by the embodiment of the application;

[0081] Figure 6 is the fourth schematic diagram of the case that the number of reliable speed limit sign images is two in the speed limit sign and lane matching method provided by the embodiment of the application;

[0082] Referring to Figures 3-6 In one embodiment, according to the number of reliable speed limit sign images, the speed limit type and speed limit value corresponding to the reliable speed limit sign image are matched to the corresponding lane, which can include:

[0083] If the number of reliable speed limit sign images is equal to 2, and one of the reliable speed limit sign images corresponds to the highest speed limit type and the other reliable speed limit sign image corresponds to the lowest speed limit type (see Figures 3-4 ), the speed limit type and speed limit value corresponding to the reliable speed limit sign images are matched to all lanes.

[0084] When the number of reliable speed limit sign images is equal to 2, and one of the reliable speed limit sign images corresponds to the highest speed limit type and the other reliable speed limit sign image corresponds to the lowest speed limit type, that is, there are two speed limit signs in front of the road, one is the highest speed limit sign and the other is the lowest speed limit sign, regardless of the number of lanes, both speed limit signs are applicable to all lanes, that is, all lanes have the same highest speed limit and lowest speed limit, so the speed limit type and speed limit value corresponding to the reliable speed limit sign images can be matched to all lanes.

[0085] If the number of reliable speed limit sign images is equal to 2, and both of the reliable speed limit sign images correspond to the highest speed limit type (see Figures 5-6 ), the number of lanes is determined to be 2, and the speed limit type and speed limit value corresponding to the two reliable speed limit sign images are matched to the corresponding lanes.

[0086] When the number of reliable speed limit sign images is equal to 2, and both of the reliable speed limit sign images correspond to the highest speed limit type, that is, there are two speed limit signs in front of the road, both of which are highest speed limit signs, since only one highest speed limit can be set for one lane, therefore, when there are two highest speed limits in front of the road, the number of lanes must be 2, and each lane corresponds to one highest speed limit, the speed limit type and speed limit value corresponding to the two reliable speed limit sign images can be matched to the corresponding lanes.

[0087] It should be noted that the coordinates of the two reliable speed limit sign images and the coordinates of the two lanes can be matched in a certain arrangement order to determine which reliable speed limit sign image corresponds to which lane, or other ways can be used to determine which reliable speed limit sign image corresponds to which lane, which is not limited here, as long as different reliable speed limit sign images can be matched to the correct lanes.

[0088] In this embodiment, when the number of reliable speed limit sign images is 2, different ways are used to match the speed limit type and speed limit value corresponding to the reliable speed limit sign images to the corresponding lanes according to the different speed limit types corresponding to the two reliable speed limit sign images, which conforms to the actual situation when there are two speed limit signs in front of the road, and the matching of the speed limit signs and the lanes is accurate. At the same time, whether the placement of the two image speed limit sign images is horizontal or vertical, it will not affect the matching effect of the reliable speed limit sign images and the lanes.

[0089] Figure 7 is one of the schematic diagrams when the number of reliable speed limit sign images is greater than two in the speed limit sign and lane matching method provided by the embodiments of the present application;

[0090] Figure 8 is the second schematic diagram when the number of reliable speed limit sign images is greater than two in the speed limit sign and lane matching method provided by the embodiments of the present application;

[0091] Figure 9 is the third schematic diagram when the number of reliable speed limit sign images is greater than two in the speed limit sign and lane matching method provided by the embodiments of the present application;

[0092] Figure 10 is the fourth schematic diagram when the number of reliable speed limit sign images is greater than two in the speed limit sign and lane matching method provided by the embodiments of the present application;

[0093] Figure 11 is the fifth schematic diagram when the number of reliable speed limit sign images is greater than two in the speed limit sign and lane matching method provided by the embodiments of the present application;

[0094] Figure 12 is the sixth schematic diagram when the number of reliable speed limit sign images is greater than two in the speed limit sign and lane matching method provided by the embodiments of the present application;

[0095] Figure 13 is the seventh schematic diagram when the number of reliable speed limit sign images is greater than two in the speed limit sign and lane matching method provided by the embodiments of the present application;

[0096] Figure 14 is the eighth schematic diagram when the number of reliable speed limit sign images is greater than two in the speed limit sign and lane matching method provided by the embodiments of the present application;

[0097] Referring to Figures 7-14 , in one embodiment, according to the number of reliable speed limit sign images, the speed limit type and speed limit value corresponding to the reliable speed limit sign images are matched to the corresponding lanes, which can include:

[0098] if the number of reliable speed limit sign images is N, and the speed limit types corresponding to the N reliable speed limit sign images are all the highest speed limit (see Figures 7-8 ), then the number of lanes is determined to be N, and the speed limit type and speed limit value corresponding to each reliable speed limit sign image are matched to the corresponding lane; wherein N is an integer greater than 2;

[0099] When the number of reliable speed limit sign images is N (N is an integer greater than 2), and the speed limit types corresponding to the N reliable speed limit sign images are all the highest speed limit, that is, there are N speed limit signs in front of the road, and all of them are the highest speed limit signs, since only one highest speed limit can be set in one lane, when there are N highest speed limits in front of the road, the number of lanes must be N, and each lane corresponds to one highest speed limit, so the speed limit types and speed limit values corresponding to the N reliable speed limit sign images can be matched to the corresponding lanes.

[0100] If the number of reliable speed limit sign images is N, and the speed limit type corresponding to one of the reliable speed limit sign images is the lowest speed limit, and the speed limit types corresponding to the remaining N-1 reliable speed limit sign images are all the highest speed limit (see Figures 9-10 ), it is determined that the number of lanes is N-1, and the speed limit type and speed limit value corresponding to the one reliable speed limit sign image are matched to all lanes, and the speed limit type and speed limit value corresponding to the remaining N-1 reliable speed limit sign images are matched to the corresponding lanes.

[0101] When the number of reliable speed limit sign images is N, and the speed limit type corresponding to one of the reliable speed limit sign images is the lowest speed limit, and the speed limit types corresponding to the remaining N-1 reliable speed limit sign images are all the highest speed limit, that is, there are N speed limit signs in front of the road, one of which is the lowest speed limit sign, and the others are the highest speed limit signs, since only one highest speed limit can be set in one lane, when there are N-1 highest speed limits in front of the road, the number of lanes must be N-1, at the same time, since there is only one lowest speed limit, this lowest speed limit sign is applicable to all lanes regardless of the number of lanes, that is, all lanes N-1 have the same lowest speed limit, but have their own highest speed limit, so the speed limit type and speed limit value corresponding to the one reliable speed limit sign image can be matched to all lanes, and the speed limit type and speed limit value corresponding to the remaining N-1 reliable speed limit sign images can be matched to the corresponding lanes.

[0102] If the number of reliable speed limit sign images is 2M, and the speed limit types corresponding to M reliable speed limit sign images are the highest speed limit, and the speed limit types corresponding to the remaining M reliable speed limit sign images are the lowest speed limit (see Figures 11-14 ), it is determined that the number of lanes is M, and the reliable speed limit sign images with the highest speed limit type and the reliable speed limit sign images with the lowest speed limit type are matched, and the speed limit type and speed limit value are matched to the corresponding lanes according to the matching result; wherein M is an integer greater than 1.

[0103] In this case, the highest speed limit sign and the lowest speed limit sign correspond to each other, each lane has its corresponding highest speed limit and lowest speed limit, therefore, the highest speed limit sign image and the lowest speed limit sign image in the 2M reliable speed limit sign images need to be correctly matched, and then the corresponding speed limit type and speed limit value are matched to the corresponding lane according to the matching result.

[0104] It should be noted that the coordinates of the plurality of reliable speed limit sign images and the coordinates of the plurality of lanes can be matched in a certain arrangement order to determine which reliable speed limit sign image corresponds to which lane, or other ways can be used to determine which reliable speed limit sign image corresponds to which lane, which is not limited here, as long as different reliable speed limit sign images can be matched to the correct lane.

[0105] When the number of reliable speed limit sign images is greater than 2, different ways are used to match the speed limit type and speed limit value corresponding to the reliable speed limit sign image to the corresponding lane according to the different speed limit types corresponding to the reliable speed limit sign image, which conforms to the actual situation when there are multiple speed limit signs in front of the road, and the speed limit sign and lane matching is accurate. At the same time, no matter whether the placement of the multiple image speed limit sign images is horizontal or vertical, it will not affect the matching effect of the reliable speed limit sign image and the lane.

[0106] Figure 15 is a flowchart of the speed limit sign and lane matching method provided by the embodiment of the application. Refer to Figures 11-15 In one embodiment, the reliable speed limit sign image with the highest speed limit type and the reliable speed limit sign image with the lowest speed limit type are matched, and the speed limit type and speed limit value are matched to the corresponding lane according to the matching result, which can include:

[0107] 201. Establish a two-dimensional rectangular coordinate system in the road image;

[0108] The Y-axis of the two-dimensional rectangular coordinate system is arranged vertically downward.

[0109] The two-dimensional rectangular coordinate system can include all reliable speed limit sign images in the road image.

[0110] 202. Group the reliable speed limit sign images in the road image;

[0111] Wherein, the absolute value of the difference of the image abscissa of any two reliable speed limit sign images in each group is within the threshold interval.

[0112] Although in actual road, the highest speed limit sign and the lowest speed limit sign appearing in pairs are always set on the same horizontal line or the same vertical line, i.e. have the same longitudinal coordinate or horizontal coordinate, in the image processing process, the horizontal coordinates of multiple reliable speed limit sign images arranged longitudinally are not exactly equal, and generally have a difference of 1-3 pixels, therefore, the threshold range can be set as 1-3 pixels, and the reliable speed limit sign images with the absolute value of the difference of the horizontal coordinates within the range are divided into a group, so as to divide the reliable speed limit sign images with similar horizontal coordinates into a group.

[0113] It should be noted that the reliable speed limit sign images with similar longitudinal coordinates can also be divided into a group by limiting the threshold range of the longitudinal coordinate difference, which is not limited here.

[0114] 203、Determine the average value of the image horizontal coordinates of the reliable speed limit sign images in the group as the group horizontal coordinate, and arrange all groups according to the group horizontal coordinates from small to large to obtain a first sequence;

[0115] As described above, the horizontal coordinates of the reliable speed limit sign images in each group have a difference, therefore, the average value of the horizontal coordinates of all reliable speed limit sign images in the group (i.e. image horizontal coordinates) is selected as the horizontal coordinate of the group (i.e. group horizontal coordinate), and the groups are arranged from small to large according to the group horizontal coordinates to obtain a first sequence.

[0116] 204、Arrange the reliable speed limit sign images in each group in the first sequence according to the image longitudinal coordinates from small to large to obtain a second sequence;

[0117] Thus, the sorting of all reliable speed limit sign images in the road image is completed.

[0118] 205、If the speed limit type corresponding to the second reliable speed limit sign image in the second sequence is the lowest speed limit, match the Kth reliable speed limit sign image with the K+1th reliable speed limit sign image, and match the speed limit type and speed limit value corresponding to the Kth reliable speed limit sign image and the speed limit type and speed limit value corresponding to the K+1th reliable speed limit sign image to the Kth lane; lane.

[0119] Wherein, K is an odd number greater than or equal to 1 and less than 2M.

[0120] Since in practice, the highest speed limit sign is always located before the lowest speed limit sign, therefore, the speed limit type corresponding to the first reliable speed limit sign image in the second sequence is the highest speed limit, therefore, when the speed limit type corresponding to the second reliable speed limit sign image is the lowest speed limit, it can be determined that the current speed limit sign placement mode is as shown in Figure 11or 12 or 13, i.e. in the second sequence, the highest speed limit sign image is always arranged adjacent to its corresponding lowest speed limit sign image, therefore, the Kth reliable speed limit sign image can be matched with the K+1th reliable speed limit sign image, and the speed limit type and speed limit value corresponding to the Kth reliable speed limit sign image and the speed limit type and speed limit value corresponding to the K+1th reliable speed limit sign image are matched to the first lane. lane.

[0121] For example, the first reliable speed limit sign image in the second sequence is matched with the second reliable speed limit sign image, and the speed limit type and speed limit value corresponding to the first reliable speed limit sign image and the speed limit type and speed limit value corresponding to the second reliable speed limit sign image are matched to the first lane, the third reliable speed limit sign image is matched with the fourth reliable speed limit sign image, and the speed limit type and speed limit value corresponding to the third reliable speed limit sign image and the speed limit type and speed limit value corresponding to the fourth reliable speed limit sign image are matched to the second lane, and so on.

[0122] 206, if the speed limit type corresponding to the second reliable speed limit sign image in the second sequence is the highest speed limit, then the reliable speed limit sign images after the second reliable speed limit sign image in the second sequence are traversed to obtain a third sequence;

[0123] The third sequence is an ordered sequence composed of M reliable speed limit sign images with the lowest speed limit type;

[0124] When the speed limit type corresponding to the second reliable speed limit sign image is the highest speed limit, it can be determined that the current speed limit sign arrangement mode is as shown in Figure 14 i.e. in the second sequence, all highest speed limit sign images are arranged in ascending order of the longitudinal coordinate, all lowest speed limit sign images are arranged in ascending order of the longitudinal coordinate, and all arranged lowest speed limit sign images are located after all arranged highest speed limit sign images, and the third sequence is obtained in this step, i.e. all arranged lowest speed limit sign images are obtained.

[0125] 207, the Lth reliable speed limit sign image in the second sequence is matched with the Lth reliable speed limit sign image in the third sequence, and the speed limit type and speed limit value corresponding to the Lth reliable speed limit sign image in the second sequence and the speed limit type and speed limit value corresponding to the Lth reliable speed limit sign image in the third sequence are matched to the Lth lane.

[0126] Wherein, L is an integer greater than or equal to 1 and less than or equal to M.

[0127] Since the highest speed limit sign image and the lowest speed limit sign image are both M, and the sorting rules are the same, the highest speed limit sign image and the lowest speed limit sign image corresponding to the sorting position must match each other, so the Lth reliable speed limit sign image in the second sequence can be matched with the Lth reliable speed limit sign image in the third sequence, and the speed limit type and the speed limit value corresponding to the Lth reliable speed limit sign image in the second sequence and the speed limit type and the speed limit value corresponding to the Lth reliable speed limit sign image in the third sequence are matched to the Lth lane.

[0128] For example, the first reliable speed limit sign image in the second sequence is matched with the first reliable speed limit sign image in the third sequence, the speed limit type and the speed limit value corresponding to the first reliable speed limit sign image in the second sequence and the speed limit type and the speed limit value corresponding to the first reliable speed limit sign image in the third sequence are matched to the first lane, the second reliable speed limit sign image in the second sequence is matched with the second reliable speed limit sign image in the third sequence, the speed limit type and the speed limit value corresponding to the second reliable speed limit sign image in the second sequence and the speed limit type and the speed limit value corresponding to the second reliable speed limit sign image in the third sequence are matched to the second lane, and so on.

[0129] It should be noted that the coordinates of multiple reliable speed limit sign images and the coordinates of multiple lanes can be matched according to a certain arrangement order to determine which reliable speed limit sign image corresponds to which lane, or other ways can be used to determine which reliable speed limit sign image corresponds to which lane, which is not limited here, as long as different reliable speed limit sign images can be matched to the correct lane.

[0130] In this embodiment, the reliable speed limit sign images are grouped and sorted according to the image coordinates and the group coordinates of the reliable speed limit sign images, and then the corresponding highest speed limit sign image and the lowest speed limit sign image are matched in different ways according to the sorting and the speed limit type corresponding to the second reliable speed limit sign image in the sequence, and the speed limit type and the speed limit value are matched to the corresponding lane according to the matching result, which meets the actual situation that the highest speed limit sign and the lowest speed limit sign appear in pairs in front of the road, and the speed limit sign and the lane are matched accurately. At the same time, no matter whether the placement of multiple image speed limit sign images is horizontal or vertical, it will not affect the matching effect of the reliable speed limit sign image and the lane.

[0131] Reference Figure 14 In one embodiment, if the speed limit type corresponding to the second reliable speed limit sign image in the second sequence is the highest speed limit, another way of matching the reliable speed limit sign image with the speed limit type of the highest speed limit and the reliable speed limit sign image with the speed limit type of the lowest speed limit can also be used, and the speed limit type and the speed limit value are matched to the corresponding lane according to the matching result, which can include:

[0132] If the speed limit type corresponding to the second reliable speed limit sign image in the second sequence is the highest speed limit, the Rth reliable speed limit sign image in the second sequence is matched with the (M+R)th reliable speed limit sign image in the second sequence, and the speed limit type and speed limit value corresponding to the Rth reliable speed limit sign image in the second sequence and the speed limit type and speed limit value corresponding to the (M+R)th reliable speed limit sign image in the second sequence are matched to the Rth lane.

[0133] wherein R is an integer greater than or equal to 1 and less than or equal to M.

[0134] Similarly, it can be determined that the current speed limit sign placement mode is as shown in Figure 14 In the second sequence, all the highest speed limit sign images are arranged in ascending order of the longitudinal coordinate, all the lowest speed limit sign images are arranged in ascending order of the longitudinal coordinate, and all the arranged lowest speed limit sign images are located after all the arranged highest speed limit sign images. Since the highest speed limit sign image and the lowest speed limit sign image are both M, the second sequence is a sequence in which M arranged highest speed limit sign images are in front and M arranged lowest speed limit sign images are behind, i.e., the sequence number of the highest speed limit sign image is 1 to M, and the sequence number of the lowest speed limit sign image is M+1 to M. Since the sorting rules of the highest speed limit sign image and the lowest speed limit sign image are the same, in the second sequence, the highest speed limit sign image with the sequence number 1 must be matched with the lowest speed limit sign image with the sequence number M+1, the highest speed limit sign image with the sequence number 2 must be matched with the lowest speed limit sign image with the sequence number M+2, and so on. That is, the Rth reliable speed limit sign image in the second sequence is matched with the (M+R)th reliable speed limit sign image in the second sequence, and the speed limit type and speed limit value corresponding to the Rth reliable speed limit sign image in the second sequence and the speed limit type and speed limit value corresponding to the (M+R)th reliable speed limit sign image in the second sequence are matched to the Rth lane.

[0135] In the second sequence, the second reliable speed limit sign image corresponds to the highest speed limit, and another way of matching the reliable speed limit sign image with the speed limit type of the highest speed limit and the reliable speed limit sign image with the speed limit type of the lowest speed limit is provided, and the speed limit type and speed limit value are matched to the corresponding lane according to the matching result. The actual situation that the highest speed limit sign and the lowest speed limit sign appear in pairs in front of the road is met, and the speed limit sign and the lane are accurately matched.

[0136] In one embodiment, determining the number of reliable speed limit sign images according to the speed limit value can include:

[0137] If the speed limit value is greater than the confidence threshold, the speed limit sign image corresponding to the speed limit value is determined as a reliable speed limit sign image.

[0138] Summing up the reliable speed limit sign images to determine the number of reliable speed limit sign images.

[0139] The embodiment filters and screens the speed limit value through the confidence threshold, so that the obtained speed limit sign image is more reliable.

[0140] In one embodiment, after matching the speed limit type and speed limit value corresponding to the reliable speed limit sign image to the corresponding lane, the following can be included:

[0141] If there is a matching highest speed limit in the current lane, and the current vehicle speed is greater than the highest speed limit, the vehicle voice warning and screen display are started to prompt the driver that the speed has been exceeded.

[0142] The voice warning and screen display can be "the highest speed limit of the front road is XX kilometers / hour", "the minimum speed limit of the front road is XX kilometers / hour", or "the speed limit of the front road is lifted XX kilometers / hour". If the current vehicle speed is greater than the highest speed limit of the lane, the voice warning and screen display can be "you have exceeded the speed, please slow down and pay attention to safety".

[0143] The embodiment reminds the driver of the speeding behavior through the voice warning and screen display, which can effectively avoid traffic accidents.

[0144] The speed limit sign and lane matching device provided by the embodiment of the application is described below, and the speed limit sign and lane matching device described below can be mutually corresponding and referred to the speed limit sign and lane matching method described above.

[0145] Figure 16 The structure diagram of the speed limit sign and lane matching device provided by the embodiment of the application is shown in FIG. 3. Referring to FIG. 3, the speed limit sign and lane matching device provided by the embodiment of the application can include: Figure 16 The speed limit sign and lane matching device provided by the embodiment of the application can include:

[0146] The speed limit sign detection module 301 is configured to detect whether there is a speed limit sign in the collected road image.

[0147] The speed limit sign image acquisition and type judgment module 302 is configured to, if there is a speed limit sign in the road image, crop the speed limit sign to obtain a speed limit sign image, and judge the speed limit type of the speed limit sign image. The speed limit type includes the highest speed limit, the minimum speed limit, and the speed limit lifting.

[0148] The reliable speed limit sign image acquisition module 303 is configured to identify the speed limit value in the speed limit sign image, and determine the number of reliable speed limit sign images according to the speed limit value.

[0149] The reliable speed limit sign and lane matching module 304 is configured to match the speed limit type and the speed limit value corresponding to the reliable speed limit sign image to a corresponding lane according to the number of reliable speed limit sign images.

[0150] The speed limit sign and lane matching device provided by the embodiment matches the speed limit type and the speed limit value corresponding to the reliable speed limit sign image to a corresponding lane according to the number of reliable speed limit sign images. When there is only one reliable speed limit sign image, the speed limit type and the speed limit value corresponding to the reliable speed limit sign image are applicable to all lanes. When there are multiple reliable speed limit sign images, the speed limit type and the speed limit value corresponding to the multiple reliable speed limit sign images are matched to corresponding lanes according to the number of reliable speed limit sign images, so that the speed limit sign corresponding to the current lane can be accurately determined in a multi-lane multi-speed limit sign scenario, and traffic accidents can be avoided.

[0151] In one embodiment, the reliable speed limit sign and lane matching module 304 is specifically configured to:

[0152] If the number of reliable speed limit sign images is equal to 1, the speed limit type and the speed limit value corresponding to the reliable speed limit sign image are matched to all lanes.

[0153] In one embodiment, the reliable speed limit sign and lane matching module 304 is specifically configured to:

[0154] If the number of reliable speed limit sign images is equal to 2, and the speed limit type corresponding to one of the reliable speed limit sign images is the maximum speed limit and the speed limit type corresponding to the other reliable speed limit sign image is the minimum speed limit, the speed limit type and the speed limit value corresponding to the reliable speed limit sign images are matched to all lanes.

[0155] If the number of reliable speed limit sign images is equal to 2, and the speed limit type corresponding to both of the reliable speed limit sign images is the maximum speed limit, the number of lanes is determined to be 2, and the speed limit type and the speed limit value corresponding to the two reliable speed limit sign images are matched to corresponding lanes.

[0156] In one embodiment, the reliable speed limit sign and lane matching module 304 is specifically configured to:

[0157] If the number of the reliable speed limit sign images is N, and the speed limit types corresponding to the N reliable speed limit sign images are all highest speed limit, the number of lanes is determined as N, and the speed limit type and the speed limit value corresponding to each of the reliable speed limit sign images are matched to the corresponding lane; wherein N is an integer greater than 2;

[0158] If the number of the reliable speed limit sign images is N, and the speed limit type corresponding to one of the reliable speed limit sign images is lowest speed limit, and the speed limit types corresponding to the remaining N-1 reliable speed limit sign images are all highest speed limit, the number of lanes is determined as N-1, and the speed limit type and the speed limit value corresponding to the one reliable speed limit sign image are matched to all lanes, and the speed limit type and the speed limit value corresponding to the remaining N-1 reliable speed limit sign images are matched to the corresponding lanes;

[0159] If the number of the reliable speed limit sign images is 2M, and the speed limit type corresponding to M of the reliable speed limit sign images is highest speed limit, and the speed limit type corresponding to the remaining M reliable speed limit sign images is lowest speed limit, the number of lanes is determined as M, and the reliable speed limit sign images with the highest speed limit type and the reliable speed limit sign images with the lowest speed limit type are matched, and the speed limit type and the speed limit value are matched to the corresponding lanes according to the matching result; wherein M is an integer greater than 1.

[0160] In one embodiment, the reliable speed limit sign and lane matching module 304 is specifically configured to:

[0161] A two-dimensional rectangular coordinate system is established in the road image; the Y-axis of the two-dimensional rectangular coordinate system is arranged vertically downward;

[0162] The reliable speed limit sign images in the road image are grouped; wherein the absolute value of the difference between the image abscissa of any two reliable speed limit sign images in each group is within a threshold interval;

[0163] The average value of the image abscissa of the reliable speed limit sign images in each group is determined as a small group abscissa, and all groups are arranged in ascending order of the small group abscissa to obtain a first sequence;

[0164] The reliable speed limit sign images in each group in the first sequence are arranged in ascending order of the image ordinate to obtain a second sequence;

[0165] if the speed limit type corresponding to the second reliable speed limit sign image in the second sequence is the minimum speed limit, the Kth reliable speed limit sign image is matched with the K+1th reliable speed limit sign image, and the speed limit type and the speed limit value corresponding to the Kth reliable speed limit sign image and the speed limit type and the speed limit value corresponding to the K+1th reliable speed limit sign image are matched to the Kth lane; a lane; wherein K is an odd number greater than or equal to 1 and less than 2M.

[0166] In an embodiment, the reliable speed limit sign and lane matching module 304 is specifically configured to:

[0167] if the speed limit type corresponding to the second reliable speed limit sign image in the second sequence is the maximum speed limit, the reliable speed limit sign images after the second reliable speed limit sign image in the second sequence are traversed to obtain a third sequence; the third sequence is an ordered sequence composed of M reliable speed limit sign images with the minimum speed limit;

[0168] the Lth reliable speed limit sign image in the second sequence is matched with the Lth reliable speed limit sign image in the third sequence, and the speed limit type and the speed limit value corresponding to the Lth reliable speed limit sign image in the second sequence and the speed limit type and the speed limit value corresponding to the Lth reliable speed limit sign image in the third sequence are matched to the Lth lane; wherein L is an integer greater than or equal to 1 and less than or equal to M.

[0169] In an embodiment, the reliable speed limit sign and lane matching module 304 is specifically configured to:

[0170] if the speed limit type corresponding to the second reliable speed limit sign image in the second sequence is the maximum speed limit, the Rth reliable speed limit sign image in the second sequence is matched with the M+Rth reliable speed limit sign image in the second sequence, and the speed limit type and the speed limit value corresponding to the Rth reliable speed limit sign image in the second sequence and the speed limit type and the speed limit value corresponding to the M+Rth reliable speed limit sign image in the second sequence are matched to the Rth lane; wherein R is an integer greater than or equal to 1 and less than or equal to M.

[0171] In an embodiment, the reliable speed limit sign image acquisition module 303 is specifically configured to:

[0172] if the speed limit value is greater than the confidence threshold, the speed limit sign image corresponding to the speed limit value is determined as a reliable speed limit sign image;

[0173] Sum up the reliable speed limit sign images to determine the number of reliable speed limit sign images.

[0174] In one embodiment, a pre-warning module (not shown in the figure) is further included for:

[0175] If there is a matched highest speed limit in the current lane, and the current vehicle speed is greater than the highest speed limit, then start vehicle voice pre-warning and screen display to prompt the driver that the speed has been exceeded.

[0176] The vehicle provided by the embodiments of the present application is described below, and the vehicle described below can be correspondingly referred to the speed limit sign and lane matching method and device described above.

[0177] The vehicle provided by the embodiments of the present application is described below, and the vehicle described below can be correspondingly referred to the speed limit sign and lane matching method and device described above.

[0178] The device embodiments described above are only schematic, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application. Those skilled in the art can understand and implement without creative labor.

[0179] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for matching speed limit signs with lanes, characterized in that, include: Detect whether speed limit signs are present in the collected road images; If a speed limit sign exists in the road image, the speed limit sign is cropped to obtain a speed limit sign image, and the speed limit type of the speed limit sign image is determined; the speed limit type includes maximum speed limit, minimum speed limit, and speed limit lifted. Identify the speed limit value in the speed limit sign image, and determine the number of reliable speed limit sign images based on the speed limit value; Based on the number of reliable speed limit sign images, the speed limit type and speed limit value corresponding to each reliable speed limit sign image are matched to the corresponding lane, including: If the number of reliable speed limit sign images is N, and the speed limit type corresponding to all N reliable speed limit sign images is the maximum speed limit, then the number of lanes is determined to be N, and the speed limit type and speed limit value corresponding to each reliable speed limit sign image are matched to the corresponding lane; where N is an integer greater than 2; If the number of reliable speed limit sign images is N, and the speed limit type corresponding to one of the reliable speed limit sign images is the minimum speed limit, while the speed limit type corresponding to the remaining N-1 reliable speed limit sign images is the maximum speed limit, then the number of lanes is determined to be N-1. The speed limit type and speed limit value corresponding to one of the reliable speed limit sign images are matched to the entire lane, and the speed limit type and speed limit value corresponding to the remaining N-1 reliable speed limit sign images are matched to the corresponding lanes. If the number of reliable speed limit sign images is 2M, and the speed limit type corresponding to M of the reliable speed limit sign images is the maximum speed limit, and the speed limit type corresponding to the remaining M reliable speed limit sign images is the minimum speed limit, then the number of lanes is determined to be M. The reliable speed limit sign images are grouped according to the horizontal coordinate of the images and sorted within each group according to the vertical coordinate of the images to obtain a sequence of the reliable speed limit sign images; If the speed limit type corresponding to the second reliable speed limit sign image in the sequence is the minimum speed limit, the highest speed limit sign image in the sequence and its adjacent minimum speed limit sign image are used as matching images. According to the order of the matching images in the sequence, the speed limit type and the speed limit value corresponding to the matching images are matched to the lanes in the corresponding order. If the speed limit type corresponding to the second reliable speed limit sign image in the sequence is the maximum speed limit, then each image in the sequence of the maximum speed limit sign images arranged in ascending order of image vertical coordinates is matched with the image in the same order of the minimum speed limit sign images arranged in ascending order of image vertical coordinates. The speed limit type and speed limit value corresponding to the matched images are then matched to the lanes in the corresponding order according to the order of the matched images in the sequence; where M is an integer greater than 1.

2. The method for matching speed limit signs with lanes according to claim 1, characterized in that, The step of matching the speed limit type and speed limit value corresponding to the reliable speed limit sign image to the corresponding lane based on the number of reliable speed limit sign images includes: If the number of reliable speed limit sign images is equal to 1, then the speed limit type and speed limit value corresponding to the reliable speed limit sign image are matched to the entire lane.

3. The method for matching speed limit signs with lanes according to claim 1, characterized in that, The step of matching the speed limit type and speed limit value corresponding to the reliable speed limit sign image to the corresponding lane based on the number of reliable speed limit sign images includes: If the number of reliable speed limit sign images is equal to 2, and the speed limit type corresponding to one of the reliable speed limit sign images is the maximum speed limit, and the speed limit type corresponding to the other reliable speed limit sign image is the minimum speed limit, then the speed limit type and speed limit value corresponding to the reliable speed limit sign image are matched to the entire lane. If the number of reliable speed limit sign images is equal to 2, and the speed limit type corresponding to both reliable speed limit sign images is the maximum speed limit, then the number of lanes is determined to be 2, and the speed limit type and speed limit value corresponding to the two reliable speed limit sign images are matched to the corresponding lanes.

4. The method for matching speed limit signs with lanes according to claim 1, characterized in that, Matching the reliable speed limit sign image with the highest speed limit type and the reliable speed limit sign image with the lowest speed limit type, and matching the speed limit type and the speed limit value to the corresponding lane based on the matching result, including: A two-dimensional rectangular coordinate system is established in the road image; the Y-axis of the two-dimensional rectangular coordinate system is set vertically downward; The reliable speed limit sign images in the road image are grouped; wherein, the absolute value of the difference between the horizontal coordinates of any two reliable speed limit sign images in each group is within a threshold range; The average value of the horizontal coordinate of the reliable speed limit sign images within a group is determined as the horizontal coordinate of the group. All groups are arranged in ascending order of the horizontal coordinate of the group to obtain the first sequence. Arrange the reliable speed limit sign images in each group of the first sequence from smallest to largest according to the vertical coordinate of the images to obtain the second sequence; If the speed limit type corresponding to the second reliable speed limit sign image in the second sequence is the minimum speed limit, then the Kth reliable speed limit sign image is matched with the (K+1)th reliable speed limit sign image, and the speed limit type and speed limit value corresponding to the Kth reliable speed limit sign image, as well as the speed limit type and speed limit value corresponding to the (K+1)th reliable speed limit sign image, are matched to the sequence... Lane; where K is an odd number greater than or equal to 1 and less than 2M.

5. The method for matching speed limit signs with lanes according to claim 4, characterized in that, After arranging the reliable speed limit sign images in each group of the first sequence in ascending order of their vertical coordinates to obtain the second sequence, the method further includes: If the speed limit type corresponding to the second reliable speed limit sign image in the second sequence is the highest speed limit, then the reliable speed limit sign images after the second reliable speed limit sign image are traversed and sorted to obtain the third sequence; the third sequence is an ordered sequence composed of M reliable speed limit sign images whose speed limit type is the lowest speed limit; The Lth reliable speed limit sign image in the second sequence is matched with the Lth reliable speed limit sign image in the third sequence, and the speed limit type and speed limit value corresponding to the Lth reliable speed limit sign image in the second sequence, as well as the speed limit type and speed limit value corresponding to the Lth reliable speed limit sign image in the third sequence, are matched to the Lth lane; where L is an integer greater than or equal to 1 and less than or equal to M.

6. The method for matching speed limit signs with lanes according to claim 4, characterized in that, After arranging the reliable speed limit sign images in each group of the first sequence in ascending order of their vertical coordinates to obtain the second sequence, the method further includes: If the speed limit type corresponding to the second reliable speed limit sign image in the second sequence is the maximum speed limit, then the Rth reliable speed limit sign image in the second sequence is matched with the (M+R)th reliable speed limit sign image in the second sequence, and the speed limit type and speed limit value corresponding to the Rth reliable speed limit sign image in the second sequence, as well as the speed limit type and speed limit value corresponding to the (M+R)th reliable speed limit sign image in the second sequence, are matched to the Rth lane; where R is an integer greater than or equal to 1 and less than or equal to M.

7. The method for matching speed limit signs with lanes according to claim 1, characterized in that, Determining the number of reliable speed limit sign images based on the speed limit value includes: If the confidence level of the speed limit value is greater than the confidence level threshold, then the speed limit sign image corresponding to the speed limit value is determined as a reliable speed limit sign image; The number of reliable speed limit sign images is determined by statistical analysis.

8. The method for matching speed limit signs with lanes according to claim 1, characterized in that, After matching the speed limit type and speed limit value corresponding to the reliable speed limit sign image to the corresponding lane, the process includes: If the current lane has a matching maximum speed limit, and the current vehicle speed exceeds the maximum speed limit, a vehicle voice warning and screen display will be activated to inform the driver that the speed has exceeded the limit.

9. A speed limit sign and lane matching device, characterized in that, The method for performing speed limit sign and lane matching as described in claim 1 includes: The speed limit sign detection module is used to detect whether speed limit signs exist in the acquired road images. The speed limit sign image acquisition and type determination module is used to: if a speed limit sign exists in the road image, crop the speed limit sign to obtain a speed limit sign image, and determine the speed limit type of the speed limit sign image; the speed limit type includes maximum speed limit, minimum speed limit, and speed limit lifted. A reliable speed limit sign image acquisition module is used to: identify the speed limit value in the speed limit sign image, and determine the number of reliable speed limit sign images based on the speed limit value; A reliable speed limit sign and lane matching module is used to match the speed limit type and speed limit value corresponding to the reliable speed limit sign image to the corresponding lane based on the number of reliable speed limit sign images.

10. A vehicle, characterized in that, Includes the speed limit sign and lane matching device as described in claim 9.

Images