Road marking fast straight line fitting method, device, equipment and medium
Through deep convolutional neural network and loss function compensation technology, the problem of slow fitting speed of road markings in the existing technology is solved, and a fast and accurate linear fitting effect is achieved.
Patent Information
- Application Number
- CN202111492513.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-08
- Publication Date
- 2025-05-16
- Estimated Expiration
- 2041-12-08
AI Technical Summary
In the review of traffic violation evidence, when identifying urban road markings and fitting them into straight lines, the calculation amount and time are too large, resulting in slow speed.
The deep convolutional neural network is used to obtain the ground identification model identification results of road marking pictures, and the linear equation is calculated through scaling, binarization processing and least squares method, combining loss function compensation and correction to achieve rapid fit.
While maintaining the effect, the calculation amount and time are significantly reduced, and the accurate linear equations are quickly fitted at any image size.
Smart Images

Figure CN114495048B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image processing and deep learning technology, and in particular to a method, device, equipment and medium for fast straight line fitting of road markings. Background Art
[0002] In practical applications such as reviewing evidence from traffic violation capture, it is often necessary to first identify the road markings in cities. In order to facilitate subsequent logical processing, the ground markings identified by the model need to be fitted into n-order curves. In real scenarios, most traffic violations occur at urban road intersections, and there are two types of ground markings that need to be fitted: lane lines and stop lines. Therefore, the lane lines and stop lines can be approximated as straight lines at this time. Therefore, the problem becomes: fitting the lane line information and stop line information of urban road intersections identified by the model into straight lines respectively.
[0003] Under normal practice, there may be two modes of results for the model to recognize ground signs: semantic segmentation and instance segmentation. Semantic segmentation outputs mask information of each category for the entire image, while instance segmentation outputs the category, box, and mask information of each sign, such as each stop line and each lane line.
[0004] The biggest drawback of the traditional method of fitting a straight line is that it is too slow, because the point pairs that need to be fitted are related to the size of the original image and the size of the target in the original image. In violation review, the resolution of evidence images is usually very large, and the stop line and lane lines are also nearby. Therefore, the number of point pairs is huge, resulting in a large amount of calculation for the least squares method. Summary of the invention
[0005] The purpose of the present invention is to overcome the defects of the prior art and provide a method, device, equipment and medium for fast straight line fitting of road markings, which can greatly reduce the amount of calculation and calculation time while maintaining the same effect as the traditional method.
[0006] The object of the present invention is achieved through the following technical solutions:
[0007] A road marking fast straight line fitting method, the method comprising:
[0008] Acquire multiple road marking images and obtain ground sign model recognition results of the multiple road marking images through a deep convolutional neural network;
[0009] Scale the ground landmark model recognition results to a new fixed size;
[0010] Each ground marker model recognition result scaled to a fixed size is binarized using a preset threshold to obtain a set of point pairs.
[0011] Construct a straight line equation, and calculate the first straight line equation, the second straight line equation, the first loss function and the second loss function for each point pair by the least squares method, wherein the first straight line equation is y=a1*x+b1, the second straight line equation is x=a2*y+b2, the first loss function is the loss function loss1 of the first straight line equation, and the second loss function is the loss function loss2 of the second straight line equation;
[0012] Compensating the first loss function and the second loss function according to a scaling ratio of the ground marker model recognition result;
[0013] Determine the straight line equation according to the compensated first loss function and the second loss function, and calculate the two original endpoints of the straight line;
[0014] The original endpoints are corrected according to the coordinates of the bounding box of the ground identification model recognition result to obtain the final fitted straight line equation.
[0015] Furthermore, the ground marker model recognition result includes a semantic segmentation result or an instance segmentation result.
[0016] Furthermore, when the ground marker model recognition result is a semantic segmentation result, scaling the ground marker model recognition result to a new fixed size is specifically:
[0017] Post-processing the semantic segmentation result to divide it into multiple instance-level information;
[0018] Calculate the bounding box coordinates based on the pixels corresponding to each instance;
[0019] Crop the instance pixel information inside the bounding box and scale it to the preset size;
[0020] When the ground marker model recognition result is an instance segmentation result, scaling the ground marker model recognition result to a new fixed size is specifically:
[0021] Scale the fixed-size mask corresponding to the bounding box to the preset size.
[0022] Furthermore, compensating the first loss function and the second loss function according to the scaling ratio of the ground marker model recognition result specifically includes:
[0023] The first loss function after compensation is loss1'=loss1*h / h', and the second loss function after compensation is loss2'=loss2*w / w';
[0024] Among them, h is the height of the bounding box of the ground marker model recognition result, w is the width of the bounding box of the ground marker model recognition result, h' is the height of the scaled ground marker model recognition result, and w' is the width of the scaled ground marker model recognition result.
[0025] Furthermore, the method further comprises:
[0026] After compensating the first loss function and the second loss function according to the scaling ratio of the ground marker model recognition result, determining the category of the ground marker model recognition result;
[0027] If the ground marking model recognition result is a lane line, the first loss function is compensated for the second time by a preset coefficient q, and the first loss function after the second compensation is loss1"=loss1'*q;
[0028] If the ground mark model recognition result is a stop line, the second loss function is compensated for the second time by a preset coefficient q, and the second loss function after the second compensation is loss2"=loss2'*q;
[0029] Among them, 1<q≤3.
[0030] Furthermore, determining the straight line equation according to the compensated first loss function and the second loss function, and calculating the two original endpoints of the straight line specifically includes:
[0031] If both loss1 and loss2 are non-numbers, the straight line fitting fails;
[0032] If only loss1 is non-number, the equation of the line is determined to be the equation of the second line;
[0033] If only loss2 is non-number, the equation of the straight line is determined to be the first straight line equation;
[0034] If loss1 and loss2 are both not non-numbers, and loss1<loss2, then the equation of the straight line is determined to be the first equation of the straight line;
[0035] If loss1 and loss2 are both not non-numbers, and loss2<loss1, then the equation of the straight line is determined to be the second straight line equation;
[0036] According to the determined straight line equation and the point with the smallest horizontal and vertical coordinates and the point with the largest horizontal and vertical coordinates in the corresponding ground identification model recognition result, the two endpoints of the corresponding line segment are calculated to obtain [x1, y1, x2, y2].
[0037] Furthermore, the step of correcting the original endpoints according to the coordinates of the bounding box of the ground marker model recognition result to obtain the final fitted straight line equation specifically includes:
[0038] According to the coordinates of the bounding box [xmin, ymin, xmax, ymax], the original endpoints [x1, y1, x2, y2] are corrected to
[0039] [x1*((xmax-xmin) / (w'-1))+xmin,y1*((ymax-ymin) / (h'-1))+ymin,x2*((xmax-xmin) / (w'-1))+xmin,y2*((ymax-ymin) / (h'-1))+ymin];
[0040] Then the final straight line equation is fitted based on the corrected endpoints.
[0041] On the other hand, the present invention also provides a road marking fast straight line fitting device, the device comprising:
[0042] A segmentation module, used to obtain a plurality of road marking images and obtain ground sign model recognition results of the plurality of road marking images through a deep convolutional neural network;
[0043] A scaling module, used to scale the ground marker model recognition result to a new fixed size;
[0044] A processing module is used to perform binarization processing on each ground marker model recognition result scaled to a fixed size through a preset threshold to obtain a set of point pairs;
[0045] A construction module is used to construct a straight line equation, and calculate the first straight line equation, the second straight line equation, the first loss function and the second loss function for each point pair by the least square method, wherein the first straight line equation is y=a1*x+b1, the second straight line equation is x=a2*y+b2, the first loss function is the loss function loss1 of the first straight line equation, and the second loss function is the loss function loss2 of the second straight line equation;
[0046] A compensation module, used to compensate the first loss function and the second loss function according to the scaling ratio of the ground marker model recognition result;
[0047] A calculation module, used to determine the straight line equation according to the compensated first loss function and the second loss function, and calculate two original endpoints of the straight line;
[0048] The fitting module is used to correct the original endpoints according to the coordinates of the bounding box of the ground identification model recognition result to obtain the final fitting straight line equation.
[0049] On the other hand, the present invention further provides a computer device, comprising a processor and a memory, wherein the memory stores a computer program, and the computer program is loaded and executed by the processor to implement any one of the above-mentioned road marking fast straight line fitting methods.
[0050] On the other hand, the present invention further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is loaded and executed by a processor to implement any of the above-mentioned methods for fast straight line fitting of road markings.
[0051] The beneficial effects of the present invention are:
[0052] The method, device, equipment and medium for fast straight line fitting of road markings provided by the present invention first fit the straight line at low resolution and then project it to the original image coordinates. In view of the disadvantage that the horizontal and vertical lines are sometimes not distinguishable in this approach, clever optimization is made to solve this problem. Ultimately, accurate straight line equations of horizontal stop lines and vertical lane lines can be fitted extremely quickly at any image size. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] Figure 1 It is a schematic diagram of the process flow of a method for fast straight line fitting of road markings provided by an embodiment of the present invention;
[0054] Figure 2 It is a structural block diagram of a road marking fast straight line fitting device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0055] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.
[0056] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making any creative work shall fall within the scope of protection of the present invention.
[0057] The traditional method of fitting straight lines works well, but its biggest drawback is that it is too slow, because the point pairs that need to be fitted are related to the size of the original image and the size of the target in the original image. In violation review, the resolution of evidence images is usually very large, and the stop line and lane lines are also nearby. Therefore, the number of point pairs is huge, resulting in a large amount of calculation for the least squares method.
[0058] In order to solve the above technical problems, the following embodiments of the method, device, equipment and medium for fast straight line fitting of road markings of the present invention are proposed.
[0059] Example 1
[0060] Reference Figure 1 ,like Figure 1 FIG. 1 is a flow chart of a method for fast straight line fitting of road markings provided in this embodiment, and the method specifically comprises the following steps:
[0061] Step S100: Acquire multiple road marking images and obtain ground sign model recognition results of the multiple road marking images through a deep convolutional neural network.
[0062] Specifically, for each input image, a deep convolutional neural network is used to obtain the result of semantic segmentation or the result of instance segmentation.
[0063] It should be noted that the above semantic segmentation results or instance segmentation results refer to the results of the stop line and lane line categories. All input images can be processed by either semantic segmentation or instance segmentation. In the subsequent steps, there are certain differences in the processing of semantic segmentation results and instance segmentation results.
[0064] Step S200: scaling the ground marker model recognition result to a new fixed size.
[0065] Specifically, for the results of the stop line and lane line categories, if the ground sign model recognition result is a semantic segmentation result, post-processing is first performed to split the original semantic segmentation result into multiple instance-level information, and the coordinates of its bounding box are calculated based on the pixels corresponding to each instance, and then the pixel information of the instance inside the bounding box is cropped and scaled to a fixed size, taking 28*28 as an example here. Among them, instance-level information refers to having an independent border for each ground sign like the instance segmentation result.
[0066] It should be noted that, since semantic segmentation labels each pixel in the image, the result is that all stop lines and lane lines are labeled as a whole, without distinguishing each stop line from the lane line, and without the coordinates of the bounding box. Therefore, the purpose of the above steps is to obtain the coordinates of the bounding box, distinguish each stop line from the lane line, and obtain the same pixel information as the mask in the instance segmentation result.
[0067] If the ground marker model recognition result is an instance segmentation result, then for each bounding box, the fixed-size mask corresponding to the bounding box is scaled to a new fixed size. In this embodiment, the scaling size of the instance segmentation result is also taken as 28*28 as an example.
[0068] It should be noted that the computational complexity of the scaling operation is only related to the size after scaling. Therefore, both model outputs here only involve a small amount of fixed computation for scaling to 28*28.
[0069] Step S300: Binarize each ground marker model recognition result scaled to a fixed size using a preset threshold to obtain a set of point pairs.
[0070] Specifically, for each box classified as a lane line or stop line and the corresponding scaled 28*28 mask, binarization is performed according to a set threshold such as 0.5 to obtain a set of point pairs.
[0071] Step S400: Construct a straight line equation, and calculate the first straight line equation, the second straight line equation, the first loss function, and the second loss function for each point pair by the least square method. Among them, the first straight line equation is y=a1*x+b1, the second straight line equation is x=a2*y+b2, the first loss function is the loss function loss1 of the first straight line equation, and the second loss function is the loss function loss2 of the second straight line equation. Among them, the first loss function is used to measure the change in the x direction, loss1=sum(Δx), and the second loss function is used to measure the change in the y direction, loss2=sum(Δy). Generally, the vertical line has a small change in the x direction and a sharp change in the y direction, and the horizontal line has a sharp change in the x direction and a small change in the y direction. Therefore, according to the size of loss1 and loss2, it can be determined whether the current straight line has a small change in the x direction or a small change in the y direction, thereby determining whether the current straight line is a long horizontal line or a long vertical line.
[0072] Calculating the first straight line equation and the second straight line equation by the least squares method mentioned in this embodiment refers to calculating the parameters a1, b1, a2 and b2 of the first straight line equation and the second straight line equation based on a set of point pairs obtained above, thereby obtaining the first straight line equation and the second straight line equation.
[0073] It should also be noted that, since the mask information is scaled in the above steps, the 28*28 mask cannot reflect the different pixel ratios caused by the different aspect ratios of the real mask, resulting in the calculated first loss function and second loss function being unable to truly measure whether the final fitted straight line equation uses the first straight line equation or the second straight line equation. Therefore, this embodiment provides the following method to solve this problem:
[0074] Step S500: Compensating the first loss function and the second loss function according to the scaling ratio of the ground marker model recognition result.
[0075] Specifically, the actual width w and height h can be obtained according to the bounding box of the ground marker model recognition result obtained in the above steps, and the first loss function and the second loss function are compensated according to the scaling ratio of the ground marker model recognition result:
[0076] The compensated first loss function loss1'=loss1*h / 28, and the compensated second loss function loss2'=loss2*w / 28.
[0077] In addition, in practical applications, there are still a few cases where the correct straight line equation cannot be selected according to the size of loss1' and loss2'. The aforementioned few cases include when the lane line or stop line is completely vertical or horizontal in the picture, and the pixels or masks inside the 28*28 box obtained in the aforementioned steps are close to a square. At this time, the calculated loss1' and loss2' are very close, and there is a situation where loss1' is slightly larger than loss2' for the lane line, and loss2' is slightly larger than loss1' for the stop line. Therefore, this embodiment proposes a method for solving the above problem by setting a compensation coefficient based on the characteristics of the horizontal and vertical lines of the stop line and the lane line at real intersections. The method is specifically as follows:
[0078] A compensation coefficient q is set, 1<q≤3. In the displayed intersection, the stop line is generally a long horizontal line, and the lane line is a long vertical line. Therefore, for the stop line, the probability of fitting as a long vertical line should be small, and for the lane line, the probability of fitting as a long horizontal line should be small. Based on the above-mentioned characteristics of the stop line and lane line, loss1' and loss2' are further compensated with the compensation coefficient q. If the ground marking model recognition result is a lane line, the first loss function is compensated for the second time by the preset coefficient q, and the first loss function loss1" after the second compensation is loss1'*q. If the ground marking model recognition result is a stop line, the second loss function is compensated for the second time by the preset coefficient q, and the second loss function loss2" after the second compensation is loss2'*q. The compensated loss function can suppress the situation where a long vertical line is fitted when the recognition object is a stop line and the situation where a long horizontal line is fitted when the recognition object is a lane line.
[0079] It should be noted that when q=1.5, the effect is better.
[0080] Step S600: determining a straight line equation according to the compensated first loss function and the second loss function, and calculating two original endpoints of the straight line.
[0081] Specifically, since division by zero may occur when calculating the least squares, causing loss1 and / or loss2 to be non-numbers, the final straight line equation needs to be determined by the following method.
[0082] If both loss1 and loss2 are non-numbers, the straight line fitting fails. If only loss1 is a non-number, the straight line equation is determined to be the second straight line equation. If only loss2 is a non-number, the straight line equation is determined to be the first straight line equation.
[0083] If loss1 and loss2 are both not non-numbers, and loss1<loss2, then the equation of the straight line is determined to be the first equation of the straight line.
[0084] If loss1 and loss2 are both not non-numbers, and loss2<loss1, then the equation of the straight line is determined to be the second equation of the straight line.
[0085] According to the determined straight line equation and the point with the smallest horizontal and vertical coordinates and the point with the largest horizontal and vertical coordinates in the corresponding ground identification model recognition result, the two endpoints of the corresponding line segment are calculated to obtain [x1, y1, x2, y2].
[0086] Step S700: Correct the original endpoints according to the coordinates of the bounding box of the ground marker model recognition result to obtain the final fitted straight line equation.
[0087] Specifically, after determining the equation of the line, according to the coordinates of the external frame [xmin, ymin, xmax, ymax], the original endpoints [x1, y1, x2, y2] are corrected to [x1*((xmax-xmin) / (w'-1))+xmin, y1*((ymax-ymin) / (h'-1))+ymin, x2*((xmax-xmin) / (w'-1))+xmin, y2*((ymax-ymin) / (h'-1))+ymin], and then the final equation of the line is fitted according to the corrected endpoints.
[0088] The fast straight line fitting method for road markings provided in this embodiment first fits a straight line at a low resolution and then projects it to the original image coordinates. In view of the disadvantage that this method sometimes cannot distinguish horizontal and vertical lines, a clever optimization is made to solve this problem. Ultimately, accurate straight line equations for horizontal stop lines and vertical lane lines can be fitted extremely quickly at any image size.
[0089] Example 2
[0090] Reference Figure 2 ,like Figure 2 FIG. 1 is a structural block diagram of a fast straight line fitting device for road markings provided in this embodiment, and the device specifically comprises:
[0091] The segmentation module 10 is used to obtain multiple road marking pictures and obtain ground sign model recognition results of the multiple road marking pictures through a deep convolutional neural network.
[0092] The scaling module 20 is used to scale the ground marker model recognition result to a new fixed size.
[0093] The processing module 30 is used to perform binarization processing on each ground marker model recognition result scaled to a fixed size through a preset threshold to obtain a set of point pairs.
[0094] Construction module 40 is used to construct a straight line equation, and calculate the first straight line equation, the second straight line equation, the first loss function and the second loss function for each point pair by the least squares method, wherein the first straight line equation is y=a1*x+b1, the second straight line equation is x=a2*y+b2, the first loss function is the loss function loss1 of the first straight line equation, and the second loss function is the loss function loss2 of the second straight line equation.
[0095] The compensation module 50 is used to compensate the first loss function and the second loss function according to the scaling ratio of the ground marker model recognition result.
[0096] The calculation module 60 is used to determine the straight line equation according to the compensated first loss function and the second loss function, and calculate the two original endpoints of the straight line.
[0097] The fitting module 70 is used to correct the original endpoints according to the coordinates of the bounding box of the ground mark model recognition result to obtain the final fitted straight line equation.
[0098] As an implementation manner, the ground marker model recognition result includes a semantic segmentation result or an instance segmentation result.
[0099] As an implementation mode, when the ground marker model recognition result is a semantic segmentation result, scaling the ground marker model recognition result to a new fixed size is specifically as follows:
[0100] Post-process the semantic segmentation results and divide them into multiple instance-level information;
[0101] Calculate the bounding box coordinates based on the pixels corresponding to each instance;
[0102] Crop the instance pixel information inside the bounding box and scale it to the preset size;
[0103] When the ground marker model recognition result is an instance segmentation result, scaling the ground marker model recognition result to a new fixed size is as follows:
[0104] Scale the fixed-size mask corresponding to the bounding box to the preset size.
[0105] As an implementation manner, compensating the first loss function and the second loss function according to the scaling ratio of the ground marker model recognition result specifically includes:
[0106] The first loss function after compensation is loss1'=loss1*h / h', and the second loss function after compensation is loss2'=loss2*w / w';
[0107] Among them, h is the height of the bounding box of the ground marker model recognition result, w is the width of the bounding box of the ground marker model recognition result, h' is the height of the ground marker model recognition result after scaling, and w' is the width of the ground marker model recognition result after scaling.
[0108] As an embodiment, the method further includes:
[0109] After compensating the first loss function and the second loss function according to the scaling ratio of the ground marker model recognition result, determining the ground marker model recognition result category;
[0110] If the ground mark model recognition result is a lane line, the first loss function is compensated for the second time by the preset coefficient q, and the first loss function after the second compensation is loss1"=loss1'*q;
[0111] If the ground mark model recognition result is a stop line, the second loss function is compensated for the second time by the preset coefficient q, and the second loss function after the second compensation is loss2"=loss2'*q;
[0112] Among them, 1<q≤3.
[0113] As an implementation mode, determining the straight line equation according to the compensated first loss function and the second loss function, and calculating the two original endpoints of the straight line specifically includes:
[0114] If both loss1 and loss2 are non-numbers, the straight line fitting fails;
[0115] If only loss1 is non-number, the equation of the line is determined to be the equation of the second line;
[0116] If only loss2 is non-number, the equation of the straight line is determined to be the first straight line equation;
[0117] If loss1 and loss2 are both not non-numbers, and loss1<loss2, then the equation of the straight line is determined to be the first equation of the straight line;
[0118] If loss1 and loss2 are both not non-numbers, and loss2<loss1, then the equation of the straight line is determined to be the second straight line equation;
[0119] According to the determined straight line equation and the point with the smallest horizontal and vertical coordinates and the point with the largest horizontal and vertical coordinates in the corresponding ground identification model recognition result, the two endpoints of the corresponding line segment are calculated to obtain [x1, y1, x2, y2].
[0120] As an implementation method, the original endpoints are corrected according to the coordinates of the bounding box of the ground marker model recognition result to obtain the final fitted straight line equation, which specifically includes:
[0121] According to the coordinates of the bounding box [xmin, ymin, xmax, ymax], the original endpoints [x1, y1, x2, y2] are corrected to
[0122] [x1*((xmax-xmin) / (w'-1))+xmin,y1*((ymax-ymin) / (h'-1))+ymin,x2*((xmax-xmin) / (w'-1))+xmin,y2*((ymax-ymin) / (h'-1))+ymin];
[0123] Then the final straight line equation is fitted based on the corrected endpoints.
[0124] The fast straight line fitting device for road markings provided in this embodiment first fits a straight line at a low resolution and then projects it to the original image coordinates. In view of the disadvantage that this method sometimes cannot distinguish horizontal and vertical lines, a clever optimization is made to solve this problem. Ultimately, it is possible to extremely quickly fit accurate straight line equations for horizontal stop lines and vertical lane lines at any image size.
[0125] Example 3
[0126] This preferred embodiment provides a computer device, which can implement the steps in any embodiment of the method for fast straight line fitting of road markings provided in the embodiments of the present application. Therefore, the beneficial effects of the method for fast straight line fitting of road markings provided in the embodiments of the present application can be achieved. For details, please refer to the previous embodiments and will not be repeated here.
[0127] Example 4
[0128] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling related hardware through instructions, and the instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. To this end, an embodiment of the present invention provides a storage medium, which stores a plurality of instructions, and the instructions can be loaded by a processor to execute the steps of any embodiment of the road marking fast straight line fitting method provided by the embodiment of the present invention.
[0129] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0130] Since the instructions stored in the storage medium can execute the steps in any of the embodiments of the method for fast straight line fitting of road markings provided in the embodiments of the present invention, the beneficial effects that can be achieved by any of the methods for fast straight line fitting of road markings provided in the embodiments of the present invention can be achieved. For details, please refer to the previous embodiments and will not be repeated here.
[0131] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for fast straight line fitting of road markings, characterized in that: The method comprises: Acquire multiple road marking images and obtain ground sign model recognition results of the multiple road marking images through a deep convolutional neural network; Scale the ground landmark model recognition results to a new fixed size; Each ground marker model recognition result scaled to a fixed size is binarized using a preset threshold to obtain a set of point pairs. Construct a straight line equation, and calculate the first straight line equation, the second straight line equation, the first loss function and the second loss function for each point pair by the least squares method, wherein the first straight line equation is y=a1*x+b1, the second straight line equation is x=a2*y+b2, the first loss function is the loss function loss1 of the first straight line equation, and the second loss function is the loss function loss2 of the second straight line equation; Compensating the first loss function and the second loss function according to a scaling ratio of the ground marker model recognition result; Determine the straight line equation according to the compensated first loss function and the second loss function, and calculate the two original endpoints of the straight line; The original endpoints are corrected according to the coordinates of the bounding box of the ground identification model recognition result to obtain the final fitted straight line equation.
2. The method for fast straight line fitting of road markings as claimed in claim 1, characterized in that: The ground marker model recognition result includes a semantic segmentation result or an instance segmentation result.
3. The road marking fast straight line fitting method as claimed in claim 2, characterized in that: When the ground marker model recognition result is a semantic segmentation result, scaling the ground marker model recognition result to a new fixed size is specifically: Post-processing the semantic segmentation result to divide it into multiple instance-level information; Calculate the bounding box coordinates based on the pixels corresponding to each instance; Crop the instance pixel information inside the bounding box and scale it to the preset size; When the ground marker model recognition result is an instance segmentation result, scaling the ground marker model recognition result to a new fixed size is specifically: Scale the fixed-size mask corresponding to the bounding box to the preset size.
4. The method for fast straight line fitting of road markings as claimed in claim 1, characterized in that: The compensating the first loss function and the second loss function according to the scaling ratio of the ground marker model recognition result specifically includes: The first loss function after compensation is loss1'=loss1*h / h', and the second loss function after compensation is loss2'=loss2*w / w'; Among them, h is the height of the bounding box of the ground marker model recognition result, w is the width of the bounding box of the ground marker model recognition result, h' is the height of the scaled ground marker model recognition result, and w' is the width of the scaled ground marker model recognition result.
5. The method for fast straight line fitting of road markings as claimed in claim 4, characterized in that: The method further comprises: After compensating the first loss function and the second loss function according to the scaling ratio of the ground marker model recognition result, determining the category of the ground marker model recognition result; If the ground marking model recognition result is a lane line, the first loss function is compensated for the second time by a preset coefficient q, and the first loss function after the second compensation is loss1"=loss1'*q; If the ground mark model recognition result is a stop line, the second loss function is compensated for the second time by a preset coefficient q, and the second loss function after the second compensation is loss2"=loss2'*q; Among them, 1<q≤3.
6. The method for fast straight line fitting of road markings as claimed in claim 5, characterized in that: Determining the straight line equation according to the compensated first loss function and the second loss function, and calculating the two original endpoints of the straight line specifically includes: If both loss1 and loss2 are non-numbers, the straight line fitting fails; If only loss1 is non-number, the equation of the line is determined to be the equation of the second line; If only loss2 is non-number, the equation of the straight line is determined to be the first straight line equation; If loss1 and loss2 are both not non-numbers, and loss1<loss2, then the equation of the straight line is determined to be the first equation of the straight line; If loss1 and loss2 are both not non-numbers, and loss2<loss1, then the equation of the straight line is determined to be the second straight line equation; According to the determined straight line equation and the point with the smallest horizontal and vertical coordinates and the point with the largest horizontal and vertical coordinates in the corresponding ground identification model recognition result, the two endpoints of the corresponding line segment are calculated to obtain [x1, y1, x2, y2].
7. The method for fast straight line fitting of road markings as claimed in claim 6, characterized in that: The step of correcting the original endpoints according to the coordinates of the bounding box of the ground marker model recognition result to obtain the final fitted straight line equation specifically includes: According to the coordinates of the external frame [xmin, ymin, xmax, ymax], correct the original endpoints [x1, y1, x2, y2] to [x1*((xmax-xmin) / (w'-1))+xmin, y1*((ymax-ymin) / (h'-1))+ymin, x2*((xmax-xmin) / (w'-1))+xmin, y2*((ymax-ymin) / (h'-1))+ymin]; Then the final straight line equation is fitted based on the corrected endpoints.
8. A road marking fast straight line fitting device, characterized in that: The device comprises: A segmentation module, used to obtain a plurality of road marking images and obtain ground sign model recognition results of the plurality of road marking images through a deep convolutional neural network; A scaling module, used to scale the ground marker model recognition result to a new fixed size; A processing module is used to perform binarization processing on each ground marker model recognition result scaled to a fixed size through a preset threshold to obtain a set of point pairs; A construction module is used to construct a straight line equation, and calculate the first straight line equation, the second straight line equation, the first loss function and the second loss function for each point pair by the least square method, wherein the first straight line equation is y=a1*x+b1, the second straight line equation is x=a2*y+b2, the first loss function is the loss function loss1 of the first straight line equation, and the second loss function is the loss function loss2 of the second straight line equation; A compensation module, used to compensate the first loss function and the second loss function according to the scaling ratio of the ground marker model recognition result; A calculation module, used to determine the straight line equation according to the compensated first loss function and the second loss function, and calculate two original endpoints of the straight line; The fitting module is used to correct the original endpoints according to the coordinates of the bounding box of the ground identification model recognition result to obtain the final fitting straight line equation.
9. A computer device, characterized in that: The computer device includes a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the road marking fast straight line fitting method as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, which is loaded and executed by a processor to implement the road marking fast straight line fitting method as claimed in any one of claims 1 to 7.
Citation Information
Patent Citations
Multi-scale fusion method for automatically extracting a point cloud traffic sign plate
CN109271861A
Multi-marked-line pavement crack identification method, device and equipment and storage medium
CN111242017A