A method, system and device for measuring the width of a slow-moving road

Abstract

The present invention discloses a method, system and device for measuring the width of a slow road. The method comprises obtaining a picture to be measured; using a small sample instance segmentation model based on FGN to detect and divide the picture to be measured, generating multiple mask matrices, each mask matrix identifying a slow road area and an area type; counting the pixel width of the slow road area in each mask matrix row by row, and averaging the multiple pixel widths to obtain the average pixel width of each slow road area; calculating the actual width of the slow road area according to the pixel width and the actual width scale and the average pixel width; wherein the pixel width and the actual width scale are the ratio of the average pixel width of the reference object to the standard width. The present invention can accurately measure the width of a road of a certain attribute, not limited to the distance between lane lines.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular relates to a method, system and device for measuring the width of a slow-moving road. Background Art

[0002] With the proposal of the dual carbon goals and the concern of urban citizens about health, the society has paid more attention to slow traffic, and the government has paid more attention to slow traffic year by year. Based on the slow traffic system, it is necessary to estimate the width of slow roads when necessary.

[0003] Traditional width estimation methods are implemented in two stages: lane line detection and width estimation. Among them, the detection methods of lane line detection mainly include traditional methods and deep learning methods, such as: (1) The method combining edge detection with Hough transform can detect the two lane lines that the vehicle is currently traveling in a simple scene, but this method relies on the accuracy of edge detection technology; (2) The color threshold method converts the image into a color space and sets a threshold for each channel in the color space to obtain the result, but this method relies on the threshold selection of each channel and may have poor robustness.

[0004] As can be seen from the above, the existing width estimation method mainly relies on the lane line detection technology and estimates the width distance of the lane based on two lane lines. The width estimation method only measures the width between the lane lines, but does not directly estimate the width of the entire lane. Therefore, the present invention provides a method that can measure the width based on the entire lane. Summary of the invention

[0005] In order to solve at least one technical problem in the background technology, the present invention proposes a slow road width measurement method, system and device, which are used to directly predict the complete lane and estimate the lane width in a slow traffic system to complete the width measurement task of the slow lane.

[0006] According to a first aspect of the present invention, the present invention first provides a method for measuring the width of a slow-moving road, comprising:

[0007] Get the picture to be measured;

[0008] The FGN-based small sample instance segmentation model is used to detect and divide the image to be measured, and multiple mask matrices are generated, each mask matrix identifies a slow road area and an area type; wherein the FGN-based small sample instance segmentation model is obtained by machine learning training using the COCO dataset and the SS dataset;

[0009] Counting the pixel widths of the slow road area in each mask matrix row by row, and averaging a plurality of the pixel widths to obtain an average pixel width of each slow road area;

[0010] The actual width of the slow road area is calculated according to the pixel width and the actual width scale and the average pixel width; wherein the pixel width and the actual width scale is the ratio of the average pixel width of a reference object to the standard width, and the reference object is selected from the slow road area with a known standard width in the mask matrix.

[0011] Furthermore, the process of training the FGN-based small sample instance segmentation model using the COCO dataset and the SS dataset through machine learning includes:

[0012] Using some instances in the COCO dataset as training sets, inputting them into the FGN-based small sample instance segmentation model to obtain an intermediate model;

[0013] Some instances in the SS data set and some instances in the training set are used as validation sets and input into the intermediate model for optimization to obtain a trained FGN-based small sample instance segmentation model.

[0014] Furthermore, the method for obtaining the SS data set includes:

[0015] Collect road video data of the target road section at different time periods and environments;

[0016] Performing frame extraction processing on the road video data to obtain a video image sequence and prepare a video data set;

[0017] The images in the video dataset are segmented and labeled according to the preset segmentation type, and a corresponding label is generated for each image to obtain the SS dataset, wherein the label includes the area range and the area type, and the preset segmentation type includes the sidewalk, non-motorized vehicle lane, parking space, and isolation belt.

[0018] Furthermore, after generating the mask matrix and before counting the pixel width of the slow-moving road area row by row, the method further comprises:

[0019] Perform perspective transformation processing on the image to be measured and the multiple mask matrices to obtain a transformed mask matrix.

[0020] Furthermore, the performing perspective transformation processing on the image to be measured and the multiple mask matrices includes:

[0021] Performing perspective transformation on the image to be measured to generate a bird's-eye view;

[0022] Based on the picture to be measured and the bird's-eye view, a perspective transformation matrix is ​​obtained;

[0023] Perform a perspective transformation on each of the mask matrices according to the perspective transformation matrix to obtain a transformed mask matrix.

[0024] Furthermore, after counting the pixel widths of the slow-moving road area in each of the mask matrices row by row and before averaging the plurality of pixel widths, the method further comprises:

[0025] According to the preset truncated mean strategy, outliers are removed from the pixel widths counted row by row to obtain multiple pixel widths after the outliers are removed.

[0026] According to a second aspect of the present invention, the present invention provides a slow-moving road width measurement system, comprising:

[0027] An acquisition module is used to acquire the image to be measured;

[0028] A segmentation module, used to detect and divide the image to be measured by using a small sample instance segmentation model based on FGN, and generate multiple mask matrices, each mask matrix identifying a slow road area and an area type; wherein the small sample instance segmentation model based on FGN is obtained by machine learning training using the COCO dataset and the SS dataset;

[0029] A first processing module is used to count the pixel widths of the slow road area in each of the mask matrix row by row, and average a plurality of the pixel widths to obtain an average pixel width of each slow road area;

[0030] The second processing module is used to calculate the actual width of the slow road area according to the ratio of pixel width to actual width and the average pixel width; wherein the ratio of pixel width to actual width is the ratio of average pixel width to standard width of a reference object, and the reference object is selected from the slow road area with known standard width in the mask matrix.

[0031] Furthermore, the slow-moving road width measurement system includes a perspective transformation module, and the perspective transformation module is used to perform perspective transformation processing on the image to be measured and the multiple mask matrices to obtain a transformed mask matrix.

[0032] Furthermore, the slow-moving road width measurement system includes an exception processing module, which is used to remove outliers from pixel widths counted row by row according to a preset truncated mean strategy to obtain multiple pixel widths after removing outliers.

[0033] According to a third aspect of the present invention, the present invention provides an electronic device, comprising a memory and a processor, wherein:

[0034] The memory is used to store the computer program;

[0035] The processor is used to execute the computer program to implement the slow-moving road width measurement method according to any one of claims 1 to 6.

[0036] Through the technical solution of the present invention, the following technical effects can be obtained:

[0037] The slow-moving road width measurement method and device provided by the present invention adopt a small sample instance segmentation framework based on a fully guided network (FGN) to detect and divide the slow-moving roads, which reduces the pressure of data set annotation and can accurately measure the width of a certain attribute road without being limited to the distance between lane lines; secondly, by using the perspective transformation technology to change the viewing angle, the influence of the camera viewing angle and distortion on the width measurement is reduced; thirdly, by introducing the truncated mean strategy, the influence of abnormal data on the measurement results can be reduced, and the credibility of the measurement results can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 A flow chart of a method for measuring the width of a slow lane provided by an embodiment of the present invention;

[0040] Figure 2 A schematic diagram of a model framework of a small sample instance segmentation model based on FGN provided in an embodiment of the present invention;

[0041] Figure 3 A schematic diagram of the working principle of the AG-RPN module provided in an embodiment of the present invention;

[0042] Figure 4 A schematic diagram of the working principle of the RG-DET module provided in an embodiment of the present invention;

[0043] Figure 5 A schematic diagram of the working principle of the AG-FCN module provided in an embodiment of the present invention;

[0044] Figure 6 A flowchart of a method for training a small sample instance segmentation model based on FGN provided by an embodiment of the present invention using a COCO dataset and a SS dataset through machine learning;

[0045] Figure 7 A flow chart of a method for obtaining the SS data set provided in an embodiment of the present invention;

[0046] Figure 8 A flow chart of a method for performing perspective transformation processing on the image to be measured and the multiple mask matrices provided by an embodiment of the present invention;

[0047] Fig. 9 A schematic diagram of the structure of a slow lane width measurement system provided by an embodiment of the present invention;

[0048] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] It should be noted that the terms "first", "second", "third" and "fourth" in the specification and claims of this application are used to distinguish different objects rather than to describe a specific order. The terms "including" and "having" in the embodiments of this application and any variations thereof are intended to cover non-exclusive inclusions.

[0051] Among the related technologies, most lane width detection solutions rely on lane line detection technology, which can only measure the width between lane lines, rather than the width of the entire lane; in addition, most of the current lane detection methods are committed to detecting the material of the lane, etc., rather than dividing the road attributes; most of the current instance segmentation algorithms require large-scale data sets, which are difficult to label.

[0052] Instance segmentation is an important research topic in the field of computer vision, which aims to accurately identify and segment individual object instances in an image. With the widespread application of convolutional neural networks (CNNs), instance segmentation technology has made significant progress. However, traditional instance segmentation tasks require pixel-level annotation, which is more cumbersome and time-consuming than other computer vision tasks such as object detection. In addition, instance segmentation technology often has weak generalization capabilities, especially when the training data is not diverse enough or when unseen scenes or objects appear, it will show poor performance.

[0053] There are many traditional methods to improve generalization ability, including using diverse training data sets, applying data enhancement techniques during training, introducing attention mechanisms, using cross-validation techniques to evaluate model performance, etc. These techniques often require large-scale data for training and still cannot avoid the problem of difficult labeling. To this end, we are committed to implementing a small sample instance segmentation method and ensuring the generalization ability of the model.

[0054] It should be noted that slow traffic usually refers to a mode of transportation that uses slow travel. In the urban transportation system, pedestrian traffic with a speed of 4-7 kilometers per hour and non-motorized vehicles (including bicycles and electric vehicles) with a speed of less than 20 kilometers per hour can both be called "slow traffic."

[0055] According to one embodiment of the present invention, Figure 1 The present invention provides a flow chart of a method for measuring the width of a slow-moving road. Figure 1 As shown, the present invention provides a method for measuring the width of a slow-moving road, the method comprising:

[0056] S10, obtaining the image to be measured;

[0057] In this step, the image to be measured is acquired according to the actual needs of the user, and the image to be measured includes the road conditions of the target road section. Optionally, the image to be measured can be collected by road monitoring equipment or manually through electronic devices such as mobile phones.

[0058] S20, using a small sample instance segmentation model based on FGN to detect and divide the image to be measured, and generate multiple mask matrices, each mask matrix identifying a slow road area and an area type; wherein the small sample instance segmentation model based on FGN is obtained by machine learning training using the COCO dataset and the SS dataset;

[0059] The purpose of this step is to segment and mark the slow road area in the image to be measured, and finally obtain a mask matrix of different types of slow road areas. The slow road area includes sidewalks, bicycle / electric lanes, parking spaces, and isolation belts. Optionally, in the mask matrix, different slow road areas are marked with different symbols, and the symbols can be colors, edge lines, etc. In this embodiment, the sidewalk is marked with a first color, such as green, and the bicycle lane is marked with a second color, such as red.

[0060] Considering the limited samples and the high cost of sample collection and annotation, the small sample instance segmentation model based on FGN described in this step uses the classic instance segmentation dataset COCO as sample support, and realizes the division of slow lanes by expanding the traditional two-stage instance segmentation network Mask R-CNN. In other words, the fully guided network FGN extends Mask R-CNN and combines it with the small sample learning paradigm to achieve small sample instance segmentation. Specifically, FGN uses different guidance mechanisms for the tasks of different modules in Mask R-CNN to achieve full guidance. The different modules include: Attention Guided RPN (AG-RPN), Relation Guided Detector (RG-DET), Attention Guided FCN (AG-FCN).

[0061] Figure 2 A schematic diagram of the model framework of the FGN-based small sample instance segmentation model is shown, as Figure 2 As shown, the support set is denoted by S, which contains K identification images of N types; the query image is denoted by X, that is, the image to be measured in the present invention, and S and X are both extracted by a shared feature extraction network CNN to obtain their own feature maps. The result of the support set encoding is denoted by , the encoding result of the query image is recorded as Y, and the dimension is H×W×C.

[0062] In the first stage, AG-RPN encodes the support set To generate candidates for Y. Then, in the second stage, for the aligned feature map Z j Each candidate of RG-DET is used to guide the classification head and bbox head, while AG-FCN guides the mask head. The following is a detailed introduction to the above three modules:

[0063] (1) Attention-Guided RPN (AG-RPN)

[0064] Figure 3 Figure 2 shows the working principle of the AG-RPN module. Figure 3 As shown, AG-RPN performs Perform global average pooling (GAP) and mean calculation to obtain new classes The corresponding class-aware attention vectors of N classes , the expression is as follows:

[0065] (1);

[0066] Then the class-aware attention vector of each class is multiplied by the feature map Y of the query image to obtain the objective score of the query image X on N classes. , the expression is as follows:

[0067] (2);

[0068] The objective score Used to guide RPN to focus on new categories and generate class-aware candidate boxes.

[0069] (2) Relation-Guided Detector (RG-DET)

[0070] Figure 4 FIG. 4 shows a schematic diagram of the working principle of the RG-DET module. Figure 4 As shown, the result of encoding the aligned support set is After taking the average, we get , With Z j Spliced ​​together, a multi-layer perceptron (MLP) is used to guide the detection branch in Mask R-CNN through an explicit contrast scheme to adapt to FSIS (Few Sample Instance Segmentation). Specifically, for the detection branch, Z j and That is, the matching score with the feature of the i-th class is combined with a combination (C + , C - ), which represent the scores of matching the i-th object and the background respectively. In order to achieve background correction, a vector is derived , the original combination score must have 2N, from which the positive scores of N categories are recorded as ,Will Denoted as:

[0071] (3);

[0072] Then, the vector Softmax processing is performed so that the confidence of the background needs to depend on the confidence of the best matching class to avoid background suppression problems.

[0073] (3) Attention-Guided FCN (AG-FCN)

[0074] Figure 5 The working principle diagram of the AG-FCN module is shown as follows: Figure 5 As shown in Figure 2, the structure of AG-FCN is similar to that of AG-RPN. It also obtains attention information from the support set to guide the mask segmentation part, which will not be repeated here.

[0075] Using the above-mentioned small sample instance segmentation model based on FGN to measure the width of slow-moving roads can reduce the pressure of data collection and annotation, thereby reducing labor costs. After obtaining the small sample instance segmentation model based on FGN, it is necessary to train the small sample instance segmentation model. Specifically, Figure 6 As shown in FIG. 1 , the process of training the FGN-based small sample instance segmentation model using the COCO dataset and the SS dataset through machine learning includes:

[0076] S21, using some instances in the COCO dataset as training sets, inputting them into the FGN-based small sample instance segmentation model to obtain an intermediate model;

[0077] S22, taking some instances in the SS data set and some instances in the training set as validation sets, inputting them into the intermediate model for optimization, and obtaining a trained FGN-based small sample instance segmentation model.

[0078] Specifically, COCO stands for Common Objects in Context, which is a dataset that can be used for image recognition. In this embodiment, the 80 categories of the COCO dataset are used as base classes. , the four categories of slow roads, such as bicycle lanes, sidewalks, parking spaces, and median strips, are included as new categories , and set the training set accordingly and support set They are used as the training set of COCO and the validation set of SS respectively. Different from Mask R-CNN, the training process of this embodiment includes two steps:

[0079] In the first step, only the base class of As training data;

[0080] The second step is to start from the base class and new class In this embodiment, the data obtained is from the entire support set and from Randomly sampled from The support set consists of 3K instances of each class in . The support set contains NK instances, which means that each class in N categories includes K instances.

[0081] In the specific training process, the training parameters are optionally set as follows: ResNet101 is used as the backbone of the model, the SGD (Stochastic Gradient Descent) optimizer is used, and the initial learning rate of SGD for training the first stage AG-RPN and the second stage RG-DET and AG-FCN is set to 0.01 and 0.001 respectively. The number of iterations is set to 60,000, and the learning rate is decayed by 10 times for the last 50% of the steps.

[0082] The SS data set includes photos of divided slow-moving road areas and marked area types. Figure 7 The flowchart of the method for obtaining the SS data set in the above step S20 is shown as follows: Figure 7 As shown, the method for obtaining the SS data set includes:

[0083] S23, collecting road video data of the target road section at different time periods and environments;

[0084] Optionally, the road video data may be real-time video data of roads at various intersections in a city, collected by monitoring equipment or the like.

[0085] S24, performing frame extraction processing on the road video data to obtain a video image sequence and generate a video data set;

[0086] As an optional embodiment, step S24 includes: using the ffmpeg tool, intercepting pictures every 8-12 frames in the road video data to obtain a video image sequence; and making multiple pictures in the video image sequence into a video data set. In this embodiment, it is selected to intercept pictures every 10 frames in the road video data to reduce the number of annotations in the video data set and the similarity of the annotated pictures, so as to prevent the subsequent training model from being too redundant.

[0087] The ffmpeg tool is used to convert the video file format and convert the video file into an image sequence. The video image sequence is a set consisting of a series of continuous images. In this embodiment, the video image sequence includes 15776 pictures, and these pictures are collected together to form a video data set.

[0088] S25, segmenting and labeling the images in the video dataset according to the preset segmentation type, generating a corresponding label for each image, and obtaining an SS dataset, wherein the label includes an area range and an area type, and the preset segmentation type includes a sidewalk, a non-motorized vehicle lane, a parking space, and a isolation belt.

[0089] Based on the video data set produced in step S24, step S25 performs high-quality annotation on each image in the video data set, and respectively annotates the division area and area type. In this embodiment, the labelme image segmentation and annotation tool is used for annotation, and the slow road area in each image is manually annotated, such as dividing bicycle lanes, pedestrian lanes, parking spaces, isolation belts, etc., to generate an SS data set (Slow Street Data) suitable for slow road division. Further, multiple images in the SS data set are randomly assigned, with a portion of them used as training set images and the remaining portion as validation set images.

[0090] As an optional embodiment, the slow-moving road width measurement method further includes:

[0091] S50, performing perspective transformation processing on the image to be measured and the multiple mask matrices to obtain a transformed mask matrix.

[0092] Normally, since there is a certain inclination angle between the camera and the ground when collecting the picture to be measured, the parallel roads will present different pixel widths at different distances in the field of view, and the pixel width will become narrower from near to far, thus affecting the calculation of the actual width and distance of the road; in addition, the distortion introduced by the deviation of the camera manufacturing accuracy and the assembly process may cause the distortion of the original image. Therefore, this embodiment introduces a perspective transformation method to correct the distortion of the image and change the observation angle. Figure 8 A flow chart of a method for performing perspective transformation processing on the image to be measured and the multiple mask matrices is shown, as shown in FIG. Figure 8 As shown, the step S50 specifically includes:

[0093] S51, performing perspective transformation on the image to be measured to generate a bird's-eye view;

[0094] In this step, optionally, four pixel points of the image to be measured are arbitrarily selected to obtain an intermediate perspective transformation matrix, and the image to be measured is perspective transformed according to the intermediate perspective transformation matrix to obtain a bird's-eye view.

[0095] S52, obtaining a perspective transformation matrix based on the image to be measured and the bird's-eye view;

[0096] In this step, quadrilateral coordinates are flexibly selected in the picture to be measured and the bird's-eye view respectively to obtain first coordinates and second coordinates, and the perspective transformation matrix is ​​obtained according to the first coordinates and the second coordinates.

[0097] S53, performing perspective transformation on the multiple mask matrices according to the perspective transformation matrix to obtain a transformed mask matrix.

[0098] In the mask matrix obtained in step S53, the slow-moving road areas are kept parallel from near to far, thereby avoiding the problem of increasingly narrow pixel widths, reducing the influence of camera viewing angle and distortion on width measurement, and thus improving the accuracy of width measurement.

[0099] S30, counting the pixel width of the slow road area in each of the mask matrix row by row, and averaging a plurality of the pixel widths to obtain an average pixel width of each slow road area;

[0100] After the small sample instance segmentation model predicts, step S20 outputs a mask matrix for each road instance, that is, a mask matrix is ​​output for the sidewalk and a mask matrix is ​​output for the parking space. The value of each position in the mask matrix represents the confidence that the pixel at that position belongs to this category. Each mask matrix is ​​used to display a mask image after the road instance is divided.

[0101] Specifically, the number of pixels in each row of the road instance identified in each mask matrix, that is, the pixel width of each row, is calculated, and the sum of the pixel widths of all rows is calculated, and then the average pixel width of the road instance is calculated based on the sum of the pixel widths of all rows and the number of rows. In this embodiment, along the column direction, the number of pixels in each row whose confidence is greater than a preset threshold is counted as the pixel width of this row, for example, the preset threshold mask_thresh=0.7, and the predicted category confidence is greater than the preset threshold, and it can be considered that the pixel belongs to this category.

[0102] Furthermore, the method includes: after counting pixel widths row by row, removing rows with zero pixel widths, and then calculating an average pixel width.

[0103] Considering that the image to be measured may not fully display the edge of the road, the truncated mean strategy can be used to process the mask matrix output by the model prediction to eliminate abnormal data, thereby offsetting the influence of a few extreme values ​​and improving the credibility of the measurement results. As an optional embodiment, after counting the pixel width of the slow road area in each mask matrix row by row and before averaging the multiple pixel widths, it further includes:

[0104] S60, according to a preset truncated mean strategy, outliers are removed from the pixel widths counted row by row to obtain a plurality of pixel widths after the outliers are removed.

[0105] Specifically, the truncated mean strategy includes:

[0106] Step a, find the mean of all rows with non-zero pixel width and variance ;

[0107] Step b, set the standard deviation threshold for truncation to 10

[0108] Step c: remove the values ​​that differ from the mean by *threshold line, if the pixel width of the current line Satisfies the following expression:

[0109] (4);

[0110] It is believed that It is an outlier and needs to be removed, and then the mean of the pixel width after removing the outlier is calculated;

[0111] S40, calculating the actual width of the slow road area according to the pixel width, the actual width ratio and the average pixel width; wherein the pixel width and the actual width ratio is the ratio of the average pixel width of a reference object to the standard width, and the reference object is selected from the slow road area of ​​known standard width in the mask matrix.

[0112] After calculating the pixel width, it is necessary to convert the pixel width into the actual space width, so a scale is used for conversion. The choice of the reference object is not limited, as long as the standard width of the reference object in the actual space is known in advance. Optionally, according to actual traffic conditions, there must be parking spaces on slow roads, and the state stipulates the standard width of parking spaces, so the reference object can be used as a parking space to calculate the ratio of pixel width to actual width. The scale expression is as follows:

[0113] (5);

[0114] The actual width is the standard width of the parking space, which is a constant of 2.5 m; the pixel width is the average pixel width of the mask matrix corresponding to the parking space, thereby obtaining the scale value.

[0115] For the road widths of other area types, the scale value and the average pixel width of the corresponding mask matrix are substituted into the above formula (4) to calculate the road width of the corresponding area type.

[0116] The slow-moving road width measurement method provided by the embodiment of the present invention measures the width of the entire lane without calculating the distance between lane lines, and realizes the estimation of road width by using attributes, and can simultaneously obtain the attributes and width information of the road; on this basis, the traditional two-stage instance segmentation network Mask R-CNN is expanded, and a small sample instance segmentation framework based on the fully guided network FGN (Fully Guided Network) is adopted to realize the detection and division of slow roads, which reduces the pressure of data collection and annotation and reduces labor costs; in addition, the perspective transformation method is combined to transform the observation angle, so as to reduce the influence of camera angle and distortion on width estimation; thirdly, the truncated mean strategy is adopted to process the output mask matrix to offset the influence of a few extreme values.

[0117] Based on a slow-moving road width measurement method provided in Example 1, this embodiment provides a system corresponding to the slow-moving road width measurement method. For the same content, please refer to the above method embodiment, which will not be described in detail in this embodiment. Fig. 9 A schematic diagram of a slow-moving road width measurement system is shown in FIG. Fig. 9 As shown, the slow-moving road width measurement system comprises:

[0118] An acquisition module 10 is used to acquire the image to be measured;

[0119] The segmentation module 20 is used to detect and divide the image to be measured by using a small sample instance segmentation model based on FGN, and generate multiple mask matrices, each mask matrix identifies a slow road area and an area type; wherein the small sample instance segmentation model based on FGN is obtained by machine learning training using the COCO dataset and the SS dataset;

[0120] A first processing module 30 is used to count the pixel widths of the slow-moving road area in each of the mask matrix row by row, and average a plurality of the pixel widths to obtain an average pixel width of each slow-moving road area;

[0121] The second processing module 40 is used to calculate the actual width of the slow road area according to the ratio of the pixel width to the actual width and the average pixel width; wherein the ratio of the pixel width to the actual width is the ratio of the average pixel width of a reference object to the standard width, and the reference object is selected from the slow road area with a known standard width in the mask matrix.

[0122] As an optional embodiment, the slow-moving road width measurement system further includes a perspective transformation module 50, and the perspective transformation module 50 is used to perform perspective transformation processing on the image to be measured and the multiple mask matrices to obtain a transformed mask matrix.

[0123] Furthermore, the perspective transformation module 50 is further used to: perform perspective transformation on the image to be measured to generate a bird's-eye view; obtain a perspective transformation matrix based on the image to be measured and the bird's-eye view; and perform perspective transformation on the multiple mask matrices according to the perspective transformation matrix to obtain a transformed mask matrix.

[0124] As an optional embodiment, the slow-moving road width measurement system further includes an exception processing module 60, which is used to remove outliers from the pixel widths counted row by row according to a preset truncated mean strategy to obtain multiple pixel widths after removing outliers.

[0125] The slow-moving road width measurement system provided by the embodiment of the present invention measures the width of the entire lane without calculating the distance between lane lines, thereby realizing the estimation of road width by using attributes, and being able to simultaneously obtain the attributes and width information of the road; on this basis, the traditional two-stage instance segmentation network Mask R-CNN is expanded, and a small sample instance segmentation framework based on the fully guided network FGN (Fully Guided Network) is adopted to realize the detection and division of slow-moving roads, thereby alleviating the pressure of data collection and annotation and reducing labor costs; in addition, the perspective transformation module is combined to transform the observation angle, thereby reducing the influence of the camera angle and distortion on the width estimation; thirdly, the output mask matrix is ​​processed by the exception processing module to offset the influence of a few extreme values.

[0126] like Fig.10 As shown, this embodiment further provides an electronic device 100, the electronic device 100 includes: a memory 110 and a processor 120, wherein:

[0127] The memory 110 is used to store computer programs;

[0128] The processor 120 is used to execute the computer program to achieve the slow-moving road width measurement as described in the above embodiment.

[0129] The memory 110 is used as a carrier for storing resources, and may be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc. The storage method may be temporary storage or permanent storage.

[0130] In addition, the electronic device 100 also includes a power supply 130, a communication interface 140, an input / output interface 150 and a communication bus 160; wherein the power supply 130 is used to provide working voltage for each hardware device on the electronic device 100; the communication interface 140 can create a data transmission channel between the electronic device 100 and external devices, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input / output interface 150 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0131] Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described device and equipment can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0132] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative.

[0133] Those skilled in the art should understand that the scope of the invention involved in the present invention is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) to form a technical solution.

[0134] It should be understood that the size of the serial numbers of each step in the content of the invention and the embodiments of the present invention does not absolutely mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

Claims

1. A method for measuring the width of a slow-moving road, characterized in that: include: Get the picture to be measured; The FGN-based small sample instance segmentation model is used to detect and divide the image to be measured, and multiple mask matrices are generated, each mask matrix identifies a slow road area and an area type; wherein the FGN-based small sample instance segmentation model is obtained by machine learning training using the COCO dataset and the SS dataset; Counting the pixel widths of the slow road area in each mask matrix row by row, and averaging a plurality of the pixel widths to obtain an average pixel width of each slow road area; The actual width of the slow road area is calculated according to the pixel width and the actual width scale and the average pixel width; wherein the pixel width and the actual width scale is the ratio of the average pixel width of a reference object to the standard width, and the reference object is selected from the slow road area with a known standard width in the mask matrix; After counting the pixel widths of the slow road area in each mask matrix row by row and before averaging the plurality of pixel widths, the method further includes: According to the preset truncated mean strategy, outliers are removed from the pixel widths counted row by row to obtain multiple pixel widths after the outliers are removed; The process of training the FGN-based small sample instance segmentation model using the COCO dataset and the SS dataset through machine learning includes: Using some instances in the COCO dataset as training sets, inputting them into the FGN-based small sample instance segmentation model to obtain an intermediate model; Some instances in the SS data set and some instances in the training set are used as validation sets, input into the intermediate model for optimization, and a trained FGN-based small sample instance segmentation model is obtained; the method for obtaining the SS data set includes: Collect road video data of the target road section at different time periods and environments; Performing frame extraction processing on the road video data to obtain a video image sequence and prepare a video data set; Segment and label the images in the video dataset according to the preset segmentation type, generate corresponding labels for each image, and obtain the SS dataset, wherein the labels include the area range and the area type, and the preset segmentation types include sidewalks, non-motorized vehicle lanes, parking spaces, and isolation belts; After generating a plurality of mask matrices and before counting the pixel widths of the slow-moving road area row by row, the method further comprises: Performing perspective transformation processing on the image to be measured and the multiple mask matrices to obtain a transformed mask matrix; The performing perspective transformation processing on the image to be measured and the multiple mask matrices includes: Performing perspective transformation on the image to be measured to generate a bird's-eye view; Based on the picture to be measured and the bird's-eye view, a perspective transformation matrix is ​​obtained; Perform a perspective transformation on each of the mask matrices according to the perspective transformation matrix to obtain a transformed mask matrix.

2. A slow-moving road width measurement system, characterized in that: include: An acquisition module is used to acquire the image to be measured; A segmentation module, used to detect and divide the image to be measured by using a small sample instance segmentation model based on FGN, and generate multiple mask matrices, each mask matrix identifying a slow road area and an area type; wherein the small sample instance segmentation model based on FGN is obtained by machine learning training using the COCO dataset and the SS dataset; A first processing module is used to count the pixel widths of the slow road area in each of the mask matrix row by row, and average a plurality of the pixel widths to obtain an average pixel width of each slow road area; A second processing module is used to calculate the actual width of the slow road area according to the ratio of the pixel width to the actual width and the average pixel width; wherein the ratio of the pixel width to the actual width is the ratio of the average pixel width to the standard width of a reference object, and the reference object is selected from the slow road area with a known standard width in the mask matrix; Further comprising a perspective transformation module, The perspective transformation module is used to perform perspective transformation processing on the image to be measured and the multiple mask matrices to obtain a transformed mask matrix; further comprising a transformation processing module, The transformation processing module is used to perform perspective transformation on the image to be measured to generate a bird's-eye view, obtain a perspective transformation matrix based on the image to be measured and the bird's-eye view, and perform perspective transformation on each of the mask matrices according to the perspective transformation matrix to obtain a transformed mask matrix; Further includes exception handling module: The exception processing module is used to remove outliers from pixel widths counted row by row according to a preset truncated mean strategy to obtain multiple pixel widths after removing outliers; Further including a machine learning training module, The machine learning training module is used to use some instances in the COCO dataset as training sets, input the FGN-based small sample instance segmentation model to obtain an intermediate model, use some instances in the SS dataset and some instances in the training set as validation sets, input the intermediate model for optimization, and obtain a trained FGN-based small sample instance segmentation model; Further includes SS dataset acquisition module: The SS data set acquisition module is used to collect road video data of the target road section in different time periods and environments; the road video data is subjected to frame extraction processing to obtain a video image sequence, and a video data set is prepared; the pictures in the video data set are segmented and labeled according to a preset segmentation type, and a corresponding label is generated for each picture to obtain an SS data set, wherein the label includes an area range and an area type, and the preset segmentation type includes a sidewalk, a non-motorized vehicle lane, a parking space, and a isolation belt.

3. An electronic device, characterized in that: comprising a memory and a processor, wherein: The memory is used to store the computer program; The processor is used to execute the computer program to implement the slow-moving road width measurement method as claimed in claim 1.

Images