Traffic cone detection method and device
By predicting the center trajectory line and boundary pixel distance of traffic cones using a traffic cone area detection model, the problems of missed detection and time-consuming post-processing in traffic cone detection are solved, and efficient detection of traffic cone placement areas is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JILUO TECH (SHANGHAI) CO LTD
- Filing Date
- 2022-12-22
- Publication Date
- 2026-06-30
AI Technical Summary
Existing technologies for traffic cone detection are prone to missed detections and require significant post-processing time, especially when traffic cones are placed in groups, making it difficult to accurately identify traffic cone areas.
The traffic cone area detection model predicts the center trajectory line and the pixel distance between the two side boundaries of the traffic cone. Image features are extracted using the feature extraction layer and the network output layer. The model is then trained using binary classification cross-entropy and regression loss function to determine the placement area of the traffic cone.
This avoids the problem of missed traffic cones, reduces post-processing time, and improves the accuracy and efficiency of traffic cone area identification.
Smart Images
Figure CN116012801B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of target detection technology, and in particular to a method and apparatus for detecting traffic cones. Background Technology
[0002] In autonomous driving, traffic cones need to be detected to avoid construction or accident areas. In existing technologies, traffic cones are treated as targets to be detected, and traffic cones in images are detected by target detection methods.
[0003] Traffic cones along roadsides are typically placed in groups, in straight or curved formations, with individual cones being less common. Existing object detection methods usually detect individual cones, outputting bounding boxes for each cone, and then post-processing these boxes (e.g., using non-maximum suppression) to determine the cone's placement area. However, detecting individual cones can lead to missed detections when they are close together, and post-processing requires processing numerous predicted bounding boxes, resulting in significant time consumption. Summary of the Invention
[0004] This invention provides a traffic cone detection method and apparatus to solve the problems of traffic cones being easily missed in the prior art and the post-processing being time-consuming.
[0005] This invention provides a traffic cone detection method, comprising:
[0006] Acquire an image of the road ahead as the vehicle is traveling;
[0007] The road image is input into the traffic cone area detection model to obtain the traffic cone center trajectory line predicted by the traffic cone area detection model, the first pixel distance from each pixel on one side boundary of the traffic cone placement area to the traffic cone center trajectory line, and the second pixel distance from each pixel on the other side boundary of the traffic cone placement area to the traffic cone center trajectory line.
[0008] The placement area of the traffic cone is determined based on the center trajectory line of the traffic cone, the first pixel distance, and the second pixel distance.
[0009] The traffic cone region detection model is trained based on road sample images and corresponding labels. The labels include: traffic cone center trajectory line label, first pixel distance label, and second pixel distance label.
[0010] According to the present invention, a traffic cone detection method is provided, wherein the traffic cone region detection model includes: a feature extraction layer and a network output layer.
[0011] The feature extraction layer is used to extract image features from the road image;
[0012] The network output layer is used to predict the center trajectory line of the traffic cone, the first pixel distance, and the second pixel distance based on the image features.
[0013] According to a traffic cone detection method provided by the present invention, the road image is input into a traffic cone region detection model to obtain the traffic cone center trajectory line predicted by the traffic cone region detection model, the first pixel distance from each pixel on one side boundary of the traffic cone placement area to the traffic cone center trajectory line, and the second pixel distance from each pixel on the other side boundary of the traffic cone placement area to the traffic cone center trajectory line, including:
[0014] The road image is input into the feature extraction layer to obtain the image features;
[0015] The image features are input into the network output layer to output the predicted trajectory line of the traffic cone center, the first pixel distance, and the second pixel distance.
[0016] According to a traffic cone detection method provided by the present invention, before inputting the road image into a traffic cone region detection model, the method further includes: training the traffic cone region detection model, specifically including:
[0017] The road sample image is input into the feature extraction layer;
[0018] The feature extraction layer extracts sample features from the road sample images;
[0019] The network output layer outputs training results based on the sample features. The training results include: training results of the traffic cone center trajectory line, training results of the first pixel distance, and training results of the second pixel distance.
[0020] The training results of the traffic cone center trajectory line, the first pixel distance training results, the second pixel distance training results, and the corresponding labels are input into the first loss function. Training is completed when the first loss function converges.
[0021] According to a traffic cone detection method provided by the present invention, the first loss function is obtained by weighted summation of the traffic cone center trajectory line loss function, the first pixel distance loss function, and the second pixel distance loss function. The traffic cone center trajectory line loss function is a binary classification cross-entropy loss function, and the first pixel distance loss function and the second pixel distance loss function are both regression loss functions.
[0022] According to a traffic cone detection method provided by the present invention, the label further includes: a traffic cone center point label; and before inputting the road image into the traffic cone region detection model, the method further includes: training the traffic cone region detection model, specifically including:
[0023] The road sample image is input into the feature extraction layer;
[0024] The feature extraction layer extracts sample features from the road sample images;
[0025] The network output layer outputs training results based on the sample features. The training results include: training results for the center point of the traffic cone, training results for the trajectory line of the center point of the traffic cone, training results for the first pixel distance, and training results for the second pixel distance.
[0026] The training results of the traffic cone center point, the traffic cone center trajectory line, the first pixel distance, and the second pixel distance, along with the corresponding labels, are input into the second loss function. Training is complete when the second loss function converges.
[0027] According to a traffic cone detection method provided by the present invention, the method determines the placement area of the traffic cone based on the center trajectory line of the traffic cone, a first pixel distance, and a second pixel distance, including:
[0028] The first boundary line is obtained by connecting the pixels on one side of the traffic cone center trajectory line that are a first pixel distance from the traffic cone center trajectory line.
[0029] The second boundary line is obtained by connecting the pixels on the other side of the traffic cone center trajectory line that are a second pixel distance from the traffic cone center trajectory line.
[0030] The area between the first boundary line and the second boundary line is defined as the traffic cone placement area.
[0031] The present invention also provides a traffic cone detection device, comprising:
[0032] The road image acquisition module is used to acquire images of the road ahead when the vehicle is driving.
[0033] The model running module is used to input the road image into the traffic cone area detection model to obtain the traffic cone center trajectory line predicted by the traffic cone area detection model, the first pixel distance from each pixel on one side boundary of the traffic cone placement area to the traffic cone center trajectory line, and the second pixel distance from each pixel on the other side boundary of the traffic cone placement area to the traffic cone center trajectory line.
[0034] The placement area determination module is used to determine the placement area of the traffic cone based on the center trajectory line of the traffic cone, the first pixel distance, and the second pixel distance.
[0035] The traffic cone region detection model is trained based on road sample images and corresponding labels. The labels include: traffic cone center trajectory line label, first pixel distance label, and second pixel distance label.
[0036] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the traffic cone detection method as described above.
[0037] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the traffic cone detection method as described above.
[0038] The traffic cone detection method and apparatus provided by this invention inputs the road image into a traffic cone region detection model to obtain the traffic cone center trajectory line predicted by the traffic cone region detection model, the first pixel distance from each pixel on one side boundary of the traffic cone placement area to the traffic cone center trajectory line, and the second pixel distance from each pixel on the other side boundary of the traffic cone placement area to the traffic cone center trajectory line. The traffic cone placement area is then determined based on the traffic cone center trajectory line, the first pixel distance, and the second pixel distance. Since this invention predicts the entire placement area of the traffic cone by predicting the traffic cone center trajectory line, the first pixel distance, and the second pixel distance, there is no problem of missing any individual traffic cone, and the inaccuracy of the traffic cone region due to the missed detection of individual traffic cones is avoided. Furthermore, the post-processing determines the traffic cone placement area based on the traffic cone center trajectory line, the first pixel distance, and the second pixel distance, eliminating the need to process the bounding box of each traffic cone, greatly reducing post-processing time. Attached Figure Description
[0039] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0040] Figure 1 This is a schematic diagram of the traffic cone detection method provided by the present invention;
[0041] Figure 2 This is a schematic diagram of the traffic cone region detection model structure in the traffic cone detection method provided by the present invention;
[0042] Figure 3 This is a schematic diagram of the traffic cone placement area in the traffic cone detection method provided by the present invention;
[0043] Figure 4 This is a schematic diagram of the traffic cone detection device provided by the present invention;
[0044] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation
[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0046] A traffic cone detection method according to an embodiment of the present invention, such as Figure 1 As shown, it includes:
[0047] Step S110: Acquire an image of the road ahead while the vehicle is in motion. Specifically, during vehicle travel, an image of the road ahead is captured using a forward-facing camera. If traffic cones are placed on the roadside in the captured image, the area where the traffic cones are located is detected according to the following steps.
[0048] Step S120: Input the road image into the traffic cone region detection model to obtain the traffic cone center trajectory line predicted by the traffic cone region detection model, the first pixel distance from each pixel on one side boundary of the traffic cone placement area to the traffic cone center trajectory line, and the second pixel distance from each pixel on the other side boundary of the traffic cone placement area to the traffic cone center trajectory line. The traffic cone region detection model is trained based on road sample images and corresponding labels. The labels include: a traffic cone center trajectory line label, a first pixel distance label, and a second pixel distance label. This traffic cone region detection model is used to predict the traffic cone center trajectory line, the first pixel distance, and the second pixel distance based on the road image.
[0049] Step S130: Determine the placement area of the traffic cone based on the center trajectory line of the traffic cone, the first pixel distance, and the second pixel distance.
[0050] This embodiment of the traffic cone detection method predicts the entire placement area of traffic cones using a traffic cone region detection model. It predicts the center trajectory line of the traffic cone, the first pixel distance from each pixel on one side of the traffic cone placement area to the center trajectory line, and the second pixel distance from each pixel on the other side of the traffic cone placement area to the center trajectory line. This eliminates the problem of missing any traffic cones and avoids inaccurate traffic cone regions due to the omission of individual traffic cones. Furthermore, the post-processing determines the traffic cone placement area based on the center trajectory line, the first pixel distance, and the second pixel distance, eliminating the need to process the bounding box of each traffic cone and greatly reducing post-processing time.
[0051] like Figure 2As shown, the traffic cone region detection model includes a feature extraction layer and a network output layer (head). The feature extraction layer is used to extract image features from the road image; the network output layer is used to predict the center trajectory line of the traffic cone, the first pixel distance, and the second pixel distance based on the image features. Specifically, the feature extraction layer includes a backbone network layer and a decoder layer. The backbone network layer performs convolution operations on the input road image to extract image features, and then inputs the extracted image features into the decoder layer. The decoder layer is mainly used to upsample and / or downsample the input image features, making the semantic information of the obtained image features richer and the final prediction result more accurate.
[0052] based on Figure 2 The traffic cone region detection model with a medium-sized structure includes step S120, which involves: inputting the road image into a feature extraction layer to obtain the image features, and inputting the image features into a network output layer to output the predicted center trajectory line of the traffic cone, the first pixel distance, and the second pixel distance. In this step, the traffic cone region detection model can accurately obtain the center line of the traffic cone placement area and the first and second pixel distances from the center line of both side boundaries, facilitating the determination of the traffic cone placement area during post-processing.
[0053] In this embodiment, before step S120, the method further includes: training the traffic cone region detection model, specifically including:
[0054] The road sample image is input into the feature extraction layer.
[0055] The feature extraction layer extracts sample features from the road sample image.
[0056] The network output layer outputs training results based on the sample features. The training results include: training results of the traffic cone center trajectory line, training results of the first pixel distance, and training results of the second pixel distance.
[0057] The training results of the traffic cone center trajectory line, the first pixel distance training results, the second pixel distance training results, and the corresponding labels are input into the first loss function. Training is completed when the first loss function converges.
[0058] Specifically, the first loss function Loss1 is obtained by weighted summation of the following three sub-loss functions:
[0059] The loss function `loss11` for the traffic cone center trajectory line can be calculated using the binary cross-entropy (BCE) loss function, i.e., `loss11 = Cross_Entropy(pred1, target1)`. `pred1` represents the training result of the traffic cone center trajectory line, and `target1` is the label for the traffic cone center trajectory line. In the road sample image, pixels located on the traffic cone center trajectory line are labeled with a value of 1, while pixels at other locations are labeled with a value of 0. `target1` can be viewed as a matrix, where each element represents a pixel in the road sample image; a value of 1 indicates that the corresponding pixel is located on the traffic cone center trajectory line. The training results `pred1` and `target1` are matrices of the same size. The value of each element in `pred1` represents the probability that the corresponding pixel is located on the traffic cone center trajectory line. `loss11` is considered converged when the difference between the element values at the same position in `pred1` and `target1` is less than a preset threshold (e.g., 0.1).
[0060] The first pixel distance loss function, loss12, can be calculated using the regression loss function smoothL1 loss, i.e., loss12 = L1(pred2, target2). pred2 represents the training result of the first pixel distance, and target2 is the label of the first pixel distance. In the road sample image, the label value of each pixel on one side boundary of the traffic cone placement area is set to the first pixel distance, and the label value of pixels at other locations is set to 0. Target2 can be viewed as a matrix, where each element represents a pixel in the road sample image. When an element's value is the first pixel distance, it indicates that the corresponding pixel is located on one side boundary of the traffic cone placement area. Of course, the training results pred2 and target2 are matrices of the same size. In pred2, when the difference between the distance value from each pixel located on one side boundary of the traffic cone placement area to the center trajectory line of the traffic cone and the corresponding element value in target2 is less than a preset distance threshold, loss12 converges.
[0061] The second pixel distance loss function, loss13, can be calculated using the regression loss function smoothL1loss, i.e., loss13 = L1(pred3, target3). pred3 represents the training result of the second pixel distance, and target3 is the label of the second pixel distance. The setting method of target3 is similar to that of target2, and will not be repeated here.
[0062] The final first loss function, Loss1, is:
[0063] Loss1=a·loss11+b·loss12+c·loss13
[0064] Where a, b, and c are the weights of loss11, loss12, and loss13, respectively. When each of the above pred values approaches its corresponding target, and the loss value Loss1 is less than a predetermined threshold, the first loss function is considered to have converged, and training is complete.
[0065] To improve the accuracy of the traffic cone region detection model, the labels also include: traffic cone center point labels. The following training method can also be used during the training of the traffic cone region detection model:
[0066] The road sample image is input into the feature extraction layer.
[0067] The feature extraction layer extracts sample features from the road sample image.
[0068] The network output layer outputs training results based on the sample features. The training results include: training results for the traffic cone center point, training results for the traffic cone center trajectory line, training results for the first pixel distance, and training results for the second pixel distance.
[0069] The training results of the traffic cone center point, the traffic cone center trajectory line, the first pixel distance, and the second pixel distance, along with the corresponding labels, are input into the second loss function. Training is complete when the second loss function converges.
[0070] Specifically, the second loss function Loss2 is:
[0071] Loss2=a·loss11+b·loss12+c·loss13+d·loss21
[0072] Here, `loss21` is the traffic cone center point loss function, which can be calculated using the focal loss function: `loss21 = focal(pred4, target4)`. `pred4` represents the training result of the traffic cone center points, `target4` represents the traffic cone center point label, and `d` represents the weight of `loss21`. In the road sample image, the pixel located at the center of the traffic cone is labeled with a value of 1, while the pixels at other locations are labeled with a value of 0. `target4` can be viewed as a matrix, where each element represents a pixel in the road sample image; a value of 1 indicates that the corresponding pixel is located at the center of the traffic cone. The training results `pred4` and `target4` are matrices of the same size. The value of each element in `pred4` represents the probability that the corresponding pixel is located at the center of the traffic cone. `loss21` converges when the difference between the element values at the same position in `pred4` and `target4` is less than a preset threshold (e.g., 0.1).
[0073] It should be noted that the predicted traffic cone center point is only used in the model training process, adding new information supervision during training, enhancing feature robustness and feature semantics, and making the prediction of the trained model more accurate.
[0074] In this embodiment, step S130 is the post-processing of the output results of the traffic cone area detection model, specifically including:
[0075] The first boundary line is obtained by connecting the pixels on one side of the traffic cone's center trajectory line that are a distance of a first pixel from the center trajectory line.
[0076] The second boundary line is obtained by connecting the pixels on the other side of the traffic cone center trajectory line that are a second pixel distance from the traffic cone center trajectory line.
[0077] The area between the first boundary line and the second boundary line is defined as the traffic cone placement area. For example... Figure 3 As shown, the circle represents a traffic cone 310 placed on one side of the road, the dashed line connecting the center of the traffic cone 310 is the center trajectory line 320 of the traffic cone, and the area between the first boundary line 330 and the second boundary line 340 on both sides of the traffic cone 310 is the traffic cone placement area. The method of this embodiment can accurately identify the entire traffic cone placement area through the above-described traffic cone area detection model and post-processing.
[0078] The traffic cone detection device provided by the present invention is described below. The traffic cone detection device described below can be referred to in correspondence with the traffic cone detection method described above.
[0079] like Figure 4 As shown, the traffic cone detection device of the present invention includes:
[0080] The road image acquisition module 410 is used to acquire images of the road ahead when the vehicle is driving.
[0081] The model running module 420 is used to input the road image into the traffic cone area detection model to obtain the traffic cone center trajectory line predicted by the traffic cone area detection model, the first pixel distance from each pixel on one side boundary of the traffic cone placement area to the traffic cone center trajectory line, and the second pixel distance from each pixel on the other side boundary of the traffic cone placement area to the traffic cone center trajectory line.
[0082] The placement area determination module 430 is used to determine the placement area of the traffic cone based on the center trajectory line of the traffic cone, the first pixel distance, and the second pixel distance.
[0083] The traffic cone region detection model is trained based on road sample images and corresponding labels. The labels include: traffic cone center trajectory line label, first pixel distance label, and second pixel distance label.
[0084] In the traffic cone detection device of the present invention, the entire placement area of the traffic cone is predicted by a traffic cone area detection model. The prediction includes the center trajectory line of the traffic cone, the first pixel distance from each pixel on one side boundary of the traffic cone placement area to the center trajectory line of the traffic cone, and the second pixel distance from each pixel on the other side boundary of the traffic cone placement area to the center trajectory line of the traffic cone. This eliminates the problem of missing a traffic cone and avoids the problem of inaccurate traffic cone area due to the missing detection of individual traffic cones. Moreover, the post-processing determines the placement area of the traffic cone based on the center trajectory line of the traffic cone, the first pixel distance, and the second pixel distance, without the need to process the annotation box of each traffic cone, which greatly reduces the post-processing time.
[0085] Optionally, the traffic cone region detection model includes a feature extraction layer and a network output layer.
[0086] The feature extraction layer is used to extract image features from the road image.
[0087] The network output layer is used to predict the center trajectory line of the traffic cone, the first pixel distance, and the second pixel distance based on the image features.
[0088] Optionally, the model running module 420 is specifically used to input the road image into the feature extraction layer to obtain the image features; and to input the image features into the network output layer to output the predicted traffic cone center trajectory line, the first pixel distance, and the second pixel distance.
[0089] Optionally, the traffic cone detection device of the present invention further includes: a model training module, used to train the traffic cone area detection model, specifically used for:
[0090] The road sample image is input into the feature extraction layer.
[0091] The feature extraction layer extracts sample features from the road sample image.
[0092] The network output layer outputs training results based on the sample features. The training results include: training results of the traffic cone center trajectory line, training results of the first pixel distance, and training results of the second pixel distance.
[0093] The training results of the traffic cone center trajectory line, the first pixel distance training results, the second pixel distance training results, and the corresponding labels are input into the first loss function. Training is completed when the first loss function converges.
[0094] Optionally, the first loss function is obtained by weighted summation of the traffic cone center trajectory line loss function, the first pixel distance loss function, and the second pixel distance loss function. The traffic cone center trajectory line loss function is a binary classification cross-entropy loss function, and the first pixel distance loss function and the second pixel distance loss function are both regression loss functions.
[0095] Optionally, the label further includes: a traffic cone center point label; the traffic cone detection device of the present invention further includes: a model training module for training the traffic cone region detection model, specifically for:
[0096] The road sample image is input into the feature extraction layer.
[0097] The feature extraction layer extracts sample features from the road sample image.
[0098] The network output layer outputs training results based on the sample features. The training results include: training results for the traffic cone center point, training results for the traffic cone center trajectory line, training results for the first pixel distance, and training results for the second pixel distance.
[0099] The training results of the traffic cone center point, the traffic cone center trajectory line, the first pixel distance, and the second pixel distance, along with the corresponding labels, are input into the second loss function. Training is complete when the second loss function converges.
[0100] Optionally, the placement area determination module 430 is used for:
[0101] The first boundary line is obtained by connecting the pixels on one side of the traffic cone's center trajectory line that are a distance of a first pixel from the center trajectory line.
[0102] The second boundary line is obtained by connecting the pixels on the other side of the traffic cone center trajectory line that are a second pixel distance from the traffic cone center trajectory line.
[0103] The area between the first boundary line and the second boundary line is defined as the traffic cone placement area.
[0104] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a traffic cone detection method, which includes:
[0105] Acquire an image of the road ahead as the vehicle is traveling.
[0106] The road image is input into the traffic cone area detection model to obtain the traffic cone center trajectory line predicted by the traffic cone area detection model, the first pixel distance from each pixel on one side boundary of the traffic cone placement area to the traffic cone center trajectory line, and the second pixel distance from each pixel on the other side boundary of the traffic cone placement area to the traffic cone center trajectory line.
[0107] The placement area of the traffic cone is determined based on the center trajectory line of the traffic cone, the first pixel distance, and the second pixel distance.
[0108] The traffic cone region detection model is trained based on road sample images and corresponding labels. The labels include: traffic cone center trajectory line label, first pixel distance label, and second pixel distance label.
[0109] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0110] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the traffic cone detection method provided by the above methods, the method comprising:
[0111] Acquire an image of the road ahead as the vehicle is traveling.
[0112] The road image is input into the traffic cone area detection model to obtain the traffic cone center trajectory line predicted by the traffic cone area detection model, the first pixel distance from each pixel on one side boundary of the traffic cone placement area to the traffic cone center trajectory line, and the second pixel distance from each pixel on the other side boundary of the traffic cone placement area to the traffic cone center trajectory line.
[0113] The placement area of the traffic cone is determined based on the center trajectory line of the traffic cone, the first pixel distance, and the second pixel distance.
[0114] The traffic cone region detection model is trained based on road sample images and corresponding labels. The labels include: traffic cone center trajectory line label, first pixel distance label, and second pixel distance label.
[0115] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the traffic cone detection method provided by the methods described above, the method comprising:
[0116] Acquire an image of the road ahead as the vehicle is traveling.
[0117] The road image is input into the traffic cone area detection model to obtain the traffic cone center trajectory line predicted by the traffic cone area detection model, the first pixel distance from each pixel on one side boundary of the traffic cone placement area to the traffic cone center trajectory line, and the second pixel distance from each pixel on the other side boundary of the traffic cone placement area to the traffic cone center trajectory line.
[0118] The placement area of the traffic cone is determined based on the center trajectory line of the traffic cone, the first pixel distance, and the second pixel distance.
[0119] The traffic cone region detection model is trained based on road sample images and corresponding labels. The labels include: traffic cone center trajectory line label, first pixel distance label, and second pixel distance label.
[0120] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0121] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A traffic cone detection method characterized by, include: Acquire an image of the road ahead as the vehicle is traveling; The road image is input into the traffic cone area detection model to obtain the traffic cone center trajectory line predicted by the traffic cone area detection model, the first pixel distance from each pixel on one side boundary of the traffic cone placement area to the traffic cone center trajectory line, and the second pixel distance from each pixel on the other side boundary of the traffic cone placement area to the traffic cone center trajectory line. The placement area of the traffic cone is determined based on the center trajectory line of the traffic cone, the first pixel distance, and the second pixel distance. The traffic cone region detection model is trained based on road sample images and corresponding labels. The labels include: traffic cone center trajectory line label, first pixel distance label, and second pixel distance label. The traffic cone region detection model includes: a feature extraction layer and a network output layer. The feature extraction layer is used to extract image features from the road image; The network output layer is used to predict the center trajectory line of the traffic cone, the first pixel distance, and the second pixel distance based on the image features. Before inputting the road image into the traffic cone region detection model, the method further includes: training the traffic cone region detection model, specifically including: The road sample image is input into the feature extraction layer; The feature extraction layer extracts sample features from the road sample images; The network output layer outputs training results based on the sample features. The training results include: training results of the traffic cone center trajectory line, training results of the first pixel distance, and training results of the second pixel distance. The training results of the traffic cone center trajectory line, the first pixel distance training results, the second pixel distance training results, and the corresponding labels are input into the first loss function. Training is completed when the first loss function converges. Among them, the training result of the traffic cone center trajectory line is the first result matrix, and the corresponding label is the first label matrix. Each element in the first label matrix represents a pixel in the road sample image. The label value of the pixel located on the traffic cone center trajectory line is set to 1, and the label value of the pixel in other positions is set to 0. The value of each element in the first result matrix represents the probability that the corresponding pixel is located on the traffic cone center trajectory line. The first pixel distance training result is the second result matrix, and the corresponding label is the second label matrix. Each element in the second label matrix represents a pixel in the road sample image. The label value of each pixel on one side boundary of the traffic cone placement area in the road sample image is set as the first pixel distance, and the label value of the pixels at other positions is set to 0. The element value of the second result matrix is the distance value from each pixel on one side boundary of the traffic cone placement area to the center trajectory line of the traffic cone. The training result of the second pixel distance is the third result matrix, and the corresponding label is the third label matrix. Each element in the third label matrix represents a pixel in the road sample image. The label value of each pixel on the other side boundary of the traffic cone placement area in the road sample image is set as the second pixel distance, and the label value of the pixels at other positions is set to 0. The element value in the third result matrix is the distance value from each pixel on the other side boundary of the traffic cone placement area to the center trajectory line of the traffic cone.
2. The traffic cone detection method of claim 1, wherein, The road image is input into a traffic cone region detection model to obtain the traffic cone center trajectory line predicted by the model, the first pixel distance from each pixel on one side boundary of the traffic cone placement area to the traffic cone center trajectory line, and the second pixel distance from each pixel on the other side boundary of the traffic cone placement area to the traffic cone center trajectory line, including: The road image is input into the feature extraction layer to obtain the image features; The image features are input into the network output layer to output the predicted trajectory line of the traffic cone center, the first pixel distance, and the second pixel distance.
3. The traffic cone detection method of claim 1, wherein, The first loss function is obtained by weighted summation of the traffic cone center trajectory line loss function, the first pixel distance loss function, and the second pixel distance loss function. The traffic cone center trajectory line loss function is a binary classification cross-entropy loss function, and the first pixel distance loss function and the second pixel distance loss function are both regression loss functions.
4. The traffic cone detection method of claim 1, wherein, The label also includes: a traffic cone center point label. Before inputting the road image into the traffic cone region detection model, the method further includes: training the traffic cone region detection model, specifically including: The road sample image is input into the feature extraction layer; The feature extraction layer extracts sample features from the road sample images; The network output layer outputs training results based on the sample features. The training results include: training results for the center point of the traffic cone, training results for the trajectory line of the center point of the traffic cone, training results for the first pixel distance, and training results for the second pixel distance. The training results of the traffic cone center point, the traffic cone center trajectory line, the first pixel distance, and the second pixel distance, along with the corresponding labels, are input into the second loss function. Training is complete when the second loss function converges.
5. The traffic cone detection method according to any one of claims 1 to 4, characterized in that, The traffic cone placement area is determined based on the center trajectory line of the traffic cone, the first pixel distance, and the second pixel distance, including: The first boundary line is obtained by connecting the pixels on one side of the traffic cone center trajectory line that are a first pixel distance from the traffic cone center trajectory line. The second boundary line is obtained by connecting the pixels on the other side of the traffic cone center trajectory line that are a second pixel distance from the traffic cone center trajectory line. The area between the first boundary line and the second boundary line is defined as the traffic cone placement area.
6. A traffic cone detection apparatus characterized by, include: The road image acquisition module is used to acquire images of the road ahead when the vehicle is driving. The model running module is used to input the road image into the traffic cone area detection model to obtain the traffic cone center trajectory line predicted by the traffic cone area detection model, the first pixel distance from each pixel on one side boundary of the traffic cone placement area to the traffic cone center trajectory line, and the second pixel distance from each pixel on the other side boundary of the traffic cone placement area to the traffic cone center trajectory line. The placement area determination module is used to determine the placement area of the traffic cone based on the center trajectory line of the traffic cone, the first pixel distance, and the second pixel distance. The traffic cone region detection model is trained based on road sample images and corresponding labels. The labels include: traffic cone center trajectory line label, first pixel distance label, and second pixel distance label. The traffic cone region detection model includes: a feature extraction layer and a network output layer. The feature extraction layer is used to extract image features from the road image; The network output layer is used to predict the center trajectory line of the traffic cone, the first pixel distance, and the second pixel distance based on the image features. It also includes: a model training module, used to train the traffic cone region detection model, specifically for: The road sample image is input into the feature extraction layer; The feature extraction layer extracts sample features from the road sample images; The network output layer outputs training results based on the sample features. The training results include: training results of the traffic cone center trajectory line, training results of the first pixel distance, and training results of the second pixel distance. The training results of the traffic cone center trajectory line, the first pixel distance training results, the second pixel distance training results, and the corresponding labels are input into the first loss function. Training is completed when the first loss function converges. Among them, the training result of the traffic cone center trajectory line is the first result matrix, and the corresponding label is the first label matrix. Each element in the first label matrix represents a pixel in the road sample image. The label value of the pixel located on the traffic cone center trajectory line is set to 1, and the label value of the pixel in other positions is set to 0. The value of each element in the first result matrix represents the probability that the corresponding pixel is located on the traffic cone center trajectory line. The first pixel distance training result is the second result matrix, and the corresponding label is the second label matrix. Each element in the second label matrix represents a pixel in the road sample image. The label value of each pixel on one side boundary of the traffic cone placement area in the road sample image is set as the first pixel distance, and the label value of the pixels at other positions is set to 0. The element value of the second result matrix is the distance value from each pixel on one side boundary of the traffic cone placement area to the center trajectory line of the traffic cone. The training result of the second pixel distance is the third result matrix, and the corresponding label is the third label matrix. Each element in the third label matrix represents a pixel in the road sample image. The label value of each pixel on the other side boundary of the traffic cone placement area in the road sample image is set as the second pixel distance, and the label value of the pixels at other positions is set to 0. The element value in the third result matrix is the distance value from each pixel on the other side boundary of the traffic cone placement area to the center trajectory line of the traffic cone.
7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the traffic cone detection method as described in any one of claims 1 to 5.
8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by the processor, it implements the traffic cone detection method as described in any one of claims 1 to 5.