Intelligent amplification method for sensing data set of special-shaped obstacles on manually-painted road surface

Abstract

The invention provides an intelligent amplification method for a manually-drawn pavement special-shaped obstacle perception data set, as shown in Figure 1, and the method specifically comprises the following steps: S1, collecting a large amount of pavement scene sample data, selecting basic scene sample images, and obtaining a blank pavement image material library; s2, classifying and selecting road surface special-shaped obstacles; s3, manually drawing special-shaped obstacles to obtain a special-shaped obstacle material library; s4, manually drawing and fusing the special-shaped obstacle road scene; and S5: data enhancement. Through the above steps, various special-shaped obstacles are added to the actually trained road scene image, and manual drawing fusion and data set amplification are carried out. According to the intelligent data amplification method for the road surface special-shaped obstacles, a large amount of road surface special-shaped obstacle sensing data can be intelligently generated in combination with a manual drawing fusion mode.

Description

Technical Field

[0001] The present invention relates to the fields of manual painting and artificial intelligence, and particularly to an intelligent amplification method for a perception dataset of abnormal-shaped obstacles on a road surface in manual painting. Background Art

[0002] Artificial intelligence object detection is a data-driven model. However, for abnormal-shaped obstacles on the road surface, such as stones, goods, etc., there are significant differences in shape, color, volume size, etc., and it is difficult to collect training samples of these abnormal-shaped obstacles, resulting in intelligent vehicles and smart city detection devices being unable to accurately identify these abnormal-shaped obstacles on the road surface. There are the following several existing solutions to address these problems.

[0003] The first solution is as described in Chinese invention patent CN202211659613.7. This solution discloses an obstacle detection method, terminal device, and storage medium. The method first divides the ground point cloud, positive obstacle point cloud, and negative obstacle point cloud in the three-dimensional point cloud through point cloud segmentation, then extracts the outer contour of the positive obstacle, the outer contour of the negative obstacle, the outer contour of the ground, and the inner contour of the ground, and further identifies cliff lines, potholes, and blank areas on this basis. It completes the tracking of obstacle targets based on the identified positive obstacle point cloud, negative obstacle point cloud, cliff lines, potholes, and blank areas, and outputs the matching obstacle coordinates. The present invention relates to the field of obstacle detection and can detect obstacles in various complex unstructured scenarios such as steep downhill slopes, downward steps, potholes, moving obstacles, steep uphill slopes, and uneven ground while ensuring the detection accuracy and stability, with strong adaptability, generalization, and versatility. This invention mainly solves the method of obstacle detection and still cannot solve the problem of obtaining samples of abnormal-shaped obstacles.

[0004] The second solution is as described in Chinese invention patent CN202310142885.8. This invention discloses an image data amplification method, device, computing device, and storage medium. The method includes: obtaining a reference picture containing a flexible target; determining a reference point selection area of the reference picture, and extracting a first number of first pixel points from the reference point selection area as reference points; for a reference point, generating a deformed point after offsetting the position of the reference point, and generating a pair of deformed points; using the thin plate spline interpolation algorithm to process the coordinates of the reference point and the deformed point in each pair of deformed points to obtain a mapping function from the amplified picture to the reference picture; for any second pixel point in the amplified picture, determining the first pixel point mapped by the second pixel point based on the mapping function, and generating the pixel value of the second pixel point according to the pixel value of the first pixel point to generate the amplified picture. This method mainly performs data amplification in units of pixels, cannot expand from the pixel level to the target level, and cannot achieve data amplification of abnormal-shaped obstacles with fixed categories.

[0005] The third solution is as described in Chinese Patent Invention CN202410493139.8. This invention discloses a data augmentation method, device, and computer-readable storage medium for a mine foreign object detection model, including collecting foreign object samples that appear on the coal conveyor belt; recording the foreign object samples from multiple angles and distances against a green screen background to generate videos; obtaining sample videos after artificially damaging the foreign object samples; collecting pictures of the underground conveyor belt in the real scene as a texture background; extracting one frame of image per second from the videos to generate video pictures; performing matte extraction on the video pictures; for the collected pictures with conveyor belts, saving the coordinate range where the conveyor belt is located as a txt file for texture use; performing batch processing and texture processing on the prepared pictures; and performing image processing on the generated pre-sample data. It solves the problem that the traditional foreign object detection algorithm in the prior art has insufficient samples, resulting in insufficient robustness and generalization ability of the detection model. This method is mainly aimed at the mine scene, which is essentially different from the background, foreground, and surrounding environment of abnormal obstacles on ordinary roads, and cannot enhance the generation of abnormal obstacles on the road surface.

[0006] The fourth solution is as described in Chinese Patent Invention CN202310765523.4. This invention discloses an electroencephalogram (EEG) signal data augmentation algorithm based on an improved generative adversarial network. The steps include: in the improved generative adversarial network model, inputting random noise into the generator to obtain a generated signal, inputting the generated signal and the real signal into the discriminator to judge the authenticity of the input signals and feedback to the generator, and the generator adjusts according to the feedback of the discriminator to continuously generate more realistic EEG signal data to augment the EEG signal dataset; the improved generative adversarial network model uses LSTM as the core component of the generator and CNN as the discriminator, and the improved generative adversarial network model uses gradient penalty to constrain the norm of the discriminator. This invention can augment the EEG signal dataset by generating more realistic EEG signal data, thereby improving the generalization ability and performance of the network model. The data type targeted by this invention is different from the image data type, and this data augmentation method for electrical signals cannot be used for augmenting the image abnormal obstacle perception dataset.

[0007] The fifth solution is as described in Chinese Patent Invention CN 119131757 A. The present invention discloses an obstacle recognition method and device, which relates to the field of big data technology. The specific implementation of this method includes: determining the background area in the road image; obtaining road boundary information, projecting the road boundary information onto the road image to obtain the target road boundary area; separating obstacles from the background area based on the target road boundary area. This implementation can improve the detection rate and accuracy of visually low obstacles, and can detect low obstacles spanning across the road boundary. This invention mainly solves the problem of roadside target detection, can extract irregular-shaped obstacles from the background, but still cannot solve the problem of lack of irregular-shaped data, nor does it propose a method for data augmentation. Summary of the Invention

[0008] To solve the above problems, the present invention designs an intelligent augmentation method for a manually drawn dataset of irregular-shaped obstacles on the road surface. Specifically, it includes the following steps:

[0009] S1: Selection of basic road surface scene images;

[0010] Collect a large number of road surface sample data images. The selection of images of the basic sample data scene is such that there are no other interfering obstacles, and the road surface in the image is clean and blank, where subsequent manual drawing can be carried out to add irregular-shaped obstacles.

[0011] S2: Classification and selection of irregular-shaped obstacles on the road surface;

[0012] Carry out typical obstacle classification, extract the features of the obstacles. For each type of irregular-shaped obstacle, extract basic appearance information such as color and shape. The information extraction relies on manual drawing for description.

[0013] S3: Manual drawing of irregular-shaped obstacles;

[0014] Obtain the features of different types of obstacles, and carry out manual drawing again. Combine these obstacle features, and generate multiple samples with different sizes and shapes for each type of irregular-shaped obstacle.

[0015] S4: Manual drawing fusion of the road surface scene with irregular-shaped obstacles;

[0016] Combine the obtained multiple samples with the candidate road surface images, and manually draw and edit the position and posture of the irregular-shaped obstacles on the road surface to obtain the distribution state of the irregular-shaped obstacles on the road surface that conforms to the actual visual sense.

[0017] S5: Data augmentation;

[0018] Annotate the images with added irregular-shaped obstacles, and through general image enhancement algorithms, perform sample amplification to obtain a large number of datasets for detecting irregular-shaped obstacles.

[0019] The beneficial effects of the present invention are as follows:

[0020] (1) The present invention can obtain a training data set of road surface special-shaped obstacles close to the real situation based on manual painting, and can obtain a data set that meets the detection of road surface special-shaped obstacles with high realism and various shapes to the greatest extent, with low cost and high repeatability of operations.

[0021] (2) The special-shaped obstacles generated by the present invention can have a variety of rich forms, and a data set of the distribution of special-shaped obstacles on the road surface that conforms to any real scene can be obtained by editing, improving the accuracy and scope of artificial intelligence detection of road surface special-shaped obstacles. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is the flowchart of the method of the present invention;

[0023] Figure 2 is the processing flow of the intelligent amplification database of the present invention;

[0024] Figure 3 is the first implementation example of the method of the present invention;

[0025] Figure 4 is the second implementation example of the method of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0026] The following are specific embodiments of the present invention and in combination with the attached Figure 1-2 , and the technical solutions of the present invention will be further described.

[0027] The present invention provides a method for intelligent amplification of a manually painted perception data set of road surface special-shaped obstacles, which specifically includes the following steps:

[0028] S1: Selection of basic road surface scene images;

[0029] Collect a large number of road surface scene sample data, and the sample data includes roads with many different forms and different locations.

[0030] The influence scenarios of special-shaped obstacles on roads and urban traffic are important references for candidate image selection. The basic scene images are selected from street road surfaces, main road surfaces, highway road surfaces, etc., and there are no other obstacles such as vehicles and pedestrians on the road surface.

[0031] Furthermore, in addition to the road surface part, other background parts of the image, such as roadside landscapes and sky backgrounds, have rich differences, increasing the number of comparison samples in the same image of the amplified data set, and forming an image material library with blank road surface images.

[0032] S2: Classification and selection of road surface special-shaped obstacles;

[0033] Conduct typical classification and selection of road surface irregular obstacles. According to the detection needs, include road surface foreign objects, etc. as data categories that need to be augmented. The targets of road surface irregular obstacles can all be used as the target categories of the irregular obstacle dataset that needs to be augmented.

[0034] Furthermore, for each category of irregular obstacles obtained, conduct feature analysis and manual painting extraction. Extract the features that can be basically shared by different targets of the same category of irregular obstacles, such as color, shape, etc.

[0035] S3: Manual painting of irregular obstacles;

[0036] For the obtained irregular obstacle targets, conduct manual painting copying to obtain multiple copied targets. Transform each category of irregular obstacles through basically shared features to obtain multi-category and multi-form road surface irregular obstacle samples with different sizes and shapes, and obtain a single irregular obstacle material library.

[0037] S4: Manual painting fusion of irregular obstacle road surface scenes;

[0038] Place different categories of irregular obstacles in the irregular obstacle material library into the blank road surface image through painting copying to obtain...

[0039] S5: Data augmentation;

[0040] Annotate and perform data augmentation on the samples with foreign objects added to the image. Use general data augmentation algorithms to increase the amount of road surface foreign object detection data.

[0041] Furthermore, the hand-drawn material library of irregular obstacles FODSampleLibrary and the road surface image material library RoadImageLibrary can be cross-combined. The same irregular obstacle can be manually painted and fused into multiple road surface materials to form a richer irregular obstacle detection data.

[0042] Furthermore, for the road surface irregular obstacle material library DataFusionLibrary obtained by fusion, conduct detection target annotation to obtain the labels of the targets.

[0043] Furthermore, synchronously enhance the labels and the images fused with irregular obstacles, and use traditional data augmentation algorithms for data amplification. An example of the data augmentation algorithm is: enhanced rotation, translation, cropping, and brightness enhancement. Finally, obtain a manually painted and fused road surface irregular obstacle perception dataset AugmentationDataset.

[0044] As can be seen from the above description, the hand-drawn irregular obstacles in the present invention can be the categories of irregular obstacles that need to be detected on the road surface, not limited to falling rocks, irregular goods, etc.

[0045] Furthermore, annotation is performed to obtain data labels for irregular obstacles. The types of labels are not limited to object detection labels, object semantic segmentation labels, object re-identification labels, etc. Labels for deep learning object perception can all be used as labels for manual painting image data augmentation.

[0046] Furthermore, the data augmentation algorithm is a general artificial intelligence data augmentation algorithm, DataAugmentationAlgotrithm. The augmentation algorithms in the examples are not limited to the image enhancement processing methods mentioned. Other general and common image enhancement processing algorithms, DataAugmentationAlgotrithm, are applicable to the data augmentation of the present invention.

[0047] In the first embodiment of the present invention, irregular obstacles are added to the road surface through manual painting, and through the data augmentation algorithm, a larger dataset of road surface irregular obstacles is obtained.

[0048] In the first embodiment of the present invention, the categories of irregular obstacles of the present invention can be freely selected, such as those related to traffic: broken-down vehicles, animals, debris, construction signs, temporary roadblocks, etc.; natural environment: trees, rocks, puddles, snow, fallen leaves, etc.; others: pedestrians, bicycles, electric vehicles, etc.

[0049] In the first embodiment of the present invention, various augmentation methods can be selected for the augmentation algorithm, such as: geometric transformation: rotation, translation, scaling, cropping, flipping, affine transformation, perspective transformation, etc.; color transformation: brightness, contrast, saturation, hue, color space conversion, etc.; texture synthesis: adding noise, blurring, sharpening, mosaics, style transfer, etc.; weather simulation: rain, snow, fog, haze, sand and dust, etc.; lighting change: shadows, reflections, glares, night mode, etc.

[0050] In the first embodiment of the present invention, the data augmentation algorithm is not limited to those mentioned in the examples of the present invention, and can also be combined with other advanced image processing technologies, such as: generative adversarial network (GAN): generating realistic irregular obstacles and road surface textures; image inpainting: repairing occluded or damaged road surface areas; image segmentation: precisely extracting irregular obstacles and road surface areas for more refined augmentation.

[0051] In the first embodiment of the present invention, the road surface morphologies fused with irregular obstacles are not limited to the examples and can be extended to other types of road surfaces, such as: urban roads: asphalt roads, cement roads, brick roads, etc.; highways: highways of different grades with different markings and signs; rural roads: gravel roads, dirt roads, muddy roads, etc.; special roads: bridges, tunnels, ramps, roundabouts, etc.

[0052] Furthermore, by selecting obstacles, data for different scenario requirements can be generated to meet the recognition and perception requirements for abnormal road surface obstacles.

[0053] In the second embodiment of the present invention, a database of abnormal obstacles and road surface forms is constructed by collecting and annotating various types of abnormal obstacles and road surface images for quick retrieval and invocation; a data augmentation strategy is designed: different data augmentation strategies are designed according to different training objectives and scenario requirements, such as random augmentation, combined augmentation, sequential augmentation, etc.; the data augmentation effect is evaluated: evaluation metrics are used to measure the impact of data augmentation on model training effects, such as accuracy, recall rate, F1 value, etc.; the augmented data is used to increase the diversity of the autonomous driving dataset and enhance the generalization ability of the model. The recognition and detection capabilities of the autonomous driving model for abnormal obstacles and complex road surfaces are improved, the data collection and annotation costs are reduced, and the data utilization efficiency is increased.

[0054] In the second embodiment of the present invention, the present invention can be applied to the training, testing, verification and other links of the autonomous driving system. However, it is not limited to the autonomous driving system and is also applicable to vehicles or robots operating on general roads for obstacle avoidance and intelligent path planning.

[0055] In the second embodiment of the present invention, the present invention can be used in combination with other data augmentation methods to obtain better results. Other data augmentation methods are not limited to those in the first and second embodiments.

[0056] In the second embodiment of the present invention, evaluation metrics can be used to evaluate and measure the impact of the augmented data of the present invention on the perception model. The evaluation metrics are not limited to those mentioned in this example and can be extended to the metric indicators of different task artificial intelligence models.

[0057] In the second embodiment of the present invention, the data augmentation algorithm adopted by the data augmentation strategy can be a supplement to the augmentation algorithm in the first embodiment, and the selection of the augmentation algorithm can be combined as needed.

[0058] It should be noted that in this article, categories and labels are not limited to those mentioned in the examples. Other forms of input of labels can be represented in multiple forms, and other elements of abnormal obstacles can be extended to more. Categories are not limited to the examples. Without more restrictions, other identical categories and elements are not excluded.

[0059] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred examples, for those of ordinary skill in the art, it should be understood that changes, modifications, substitutions and variations of the technical solutions of the present invention should be covered within the scope of the claims of the present invention without departing from the principles and spirit of the present invention.

Claims

1. An intelligent amplification method for a manually painted road surface special-shaped obstacle perception dataset, comprising the following steps: S1: Selection of basic road surface scene images; Collect a large number of sample data, obtain various road surface images, and screen images without vehicles, pedestrians, and other obstacles blocking on the road surface as candidate painted images. S2: Classification and selection of road surface special-shaped obstacles; Conduct classification and feature selection of typical road surface special-shaped obstacles. According to detection requirements, typical road surface rockfalls, special-shaped goods, etc. are used as data categories to be amplified. Extract the manual painting of typical obstacles, and manually paint multiple individuals of foreign objects on the road surface for each category. S3: Manual painting of special-shaped obstacles; Add various foreign objects to be detected in the blank area of the road surface in the candidate painted images to create a special-shaped obstacle material library. S4: Manual painting fusion of special-shaped obstacle road surface scenes; Closely fuse the edges of the foreign objects placed on the painted road surface with the background of the image to ensure that the foreign objects on the road surface are integrated with the background road surface. S5: Data augmentation; Annotate and augment the samples with foreign objects added in the image. Use general data augmentation algorithms to amplify the road surface foreign object detection data to obtain a road surface special-shaped obstacle detection amplified dataset.

2. The method for selecting the characteristics of the road surface special-shaped obstacle according to claim 1, characterized in that Copy the texture, shape, and color of the special-shaped obstacles by hand-drawing to obtain the characteristics of the special-shaped obstacles approaching the natural state.

3. The fusion of the road surface irregular obstacle and the road image according to claim 1 is characterized in that Place the special-shaped obstacles on the blank road surface by hand-drawing, adjust the position and posture, and obtain the distribution characteristics of the special-shaped obstacles approaching the natural state through manual painting.

4. The manual painting integration of the road surface with special-shaped obstacles according to claim 1, characterized in that Manually process and fuse the edges between the special-shaped obstacles placed on the blank road surface and the road surface by hand-drawing to ensure that the special-shaped obstacles are integrated with the road surface and the surrounding environment.

5. The manual painting fusion method according to claim 4, wherein the feature lies in Manually adjust the painting color and texture of the special-shaped obstacles, add colors to the painting, and achieve a natural transition between the special-shaped obstacles and the background edge.

6. The amplified data set obtained by data augmentation according to claim 1, wherein The amplified dataset is a large sample size dataset. Different numbers of enhanced data can be obtained through the amplification factor selected by the data augmentation algorithm, which can meet the training needs of the artificial intelligence perception model.

7. According to claim 1, the sampling data enhancement algorithm performs intelligent amplification, characterized in that The data augmentation algorithm automatically amplifies each fused sample by a multiple and automatically generates multiple enhanced samples.

Images