Image segmentation and target detection integrated method for cross-country road

Abstract

The invention belongs to the technical field of image recognition, and provides an image segmentation and target detection integrated method for cross-country roads, which comprises the following steps: step 1, preprocessing an image: adjusting the input resolution of the image to 640 * 640, adjusting the pixel mean value and variance of the detection image to enable the data sample distribution of the input image to be close to the data sample during training; adjusting an input image channel into a BGR sequence format; and step 2, combining the yolov8 model and the Yolcat model to realize the integrated design of target detection and segmentation. According to the technical scheme, two tasks of image segmentation and target detection can be completed at the same time through one detection model, calculation resources can be reduced, and calculation power consumption can be reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition, and in particular relates to an integrated method of image segmentation and target detection for off-road roads. Background Art

[0002] With the addition of artificial intelligence technology, the functions of unmanned driving technology have become increasingly perfect. In terms of perception, cameras can provide vehicles with high-resolution, information-rich image data. Therefore, the environmental perception method for images has become one of the important links in environmental perception. Today, unmanned vehicles have many commercial applications in various industries, but they are all limited to urban roads with strong regularity and logic. In the process of unmanned transformation of special vehicles, data-driven artificial intelligence technology often has unclear perception effects during scene conversion due to the complexity and diversity of off-road roads. Therefore, most of the original image perception technologies cannot identify off-road roads well. At the same time, the task-driven perception method separates image segmentation and target detection tasks, resulting in a waste of computing resources and cutting off the connection between data. Summary of the Invention

[0003] (1) Technical issues to be resolved

[0004] The technical problem to be solved by the present invention is that the existing image perception technology cannot effectively identify off-road roads, and the task-driven perception method separates image segmentation and target detection tasks, resulting in a waste of computing resources.

[0005] (2) Technical solution

[0006] To solve the above technical problems, the present invention provides an integrated method for image segmentation and target detection for off-road roads, comprising the following steps:

[0007] Step 1: Preprocess the image:

[0008] Adjust the input image resolution to 640×640, adjust the pixel mean and variance of the test image so that the data sample distribution of the input image is close to the data sample during training; adjust the input image channel to BGR sequential format;

[0009] Step 2: Combine yolov8 and Yolcat models to realize integrated design of target detection and segmentation:

[0010] ① Feature extraction stage:

[0011] The backbone network and Neck structure of yolov8 are used to extract features of the input image to obtain feature maps of different scales, which contain semantic information and spatial information at different levels in the image.

[0012] ②Instance segmentation module

[0013] The above feature map is introduced into the instance segmentation module of Yolcat, and fused with the features extracted by Yolcat's own feature extraction network to obtain the fused feature map;

[0014] ③Task execution phase

[0015] The integrated model fuses the feature maps and performs both target detection and segmentation tasks. For target detection, the detection head structure of yolov8 is used to directly predict the target category and bounding box. For segmentation, the ProtoNet and Prediction Head structure of Yolcat are used to generate instance masks.

[0016] Step 3: The algorithm model calculates the input image:

[0017] The algorithm model adds an image segmentation branch module based on the yolov8 target detection model. The model outputs three types of high-dimensional matrices, namely, the type of object in the specified area of ​​the image, the center point and width and height information of the object in the image, and the label mask value in the specified area of ​​the image.

[0018] The types of objects in the designated area of ​​the corresponding image include passable areas, impassable areas, unreachable areas, pedestrians, vehicles, and irregular obstacles;

[0019] The center point and width and height information of the target in the corresponding image include the position and size information of the target at the resolution of the preprocessed image;

[0020] The label mask value in the specified area of ​​the corresponding image contains the category to which the corresponding pixel in the specified area belongs;

[0021] Step 4: Post-processing of results:

[0022] By solving and analyzing the three types of high-dimensional matrix information output in step 3, the target detection task is completed using the category matrix in the corresponding image specified area and the center point and width and height information matrix of the target in the corresponding image; the image segmentation task is completed using the category matrix in the corresponding image specified area and the label mask value in the corresponding image specified area; the inverse operation is performed according to the image resolution scaling value in step 1, and the solution result is restored to the original image size;

[0023] Step 5. Results:

[0024] The results of image segmentation and target detection are displayed uniformly in the original image. The target detection task only displays the results of non-road condition information, including pedestrians, vehicles, and irregular obstacles.

[0025] Among them, the algorithm model in step 3 introduces a convolution module based on the attention mechanism on the basis of yolov8 and Yolcat. This convolution module incorporates channel attention and spatial attention mechanisms on the basis of traditional convolution.

[0026] Among them, the mechanism of channel attention is to weight the channel dimension of the feature map.

[0027] The algorithm model adopts a hierarchical and progressive design approach: in the shallow network, it focuses on the detailed information of the image, extracting rich low-level features through smaller convolution kernels and more convolution layers; in the intermediate network, dilated convolution is used to expand the field of view, enabling the model to obtain a wider range of contextual information, thereby better handling occluded targets and targets in complex scenes.

[0028] (3) Beneficial effects

[0029] Compared with the existing technology, the present invention has the following beneficial effects: it can simultaneously complete two tasks of image segmentation and target detection through one detection model, which can reduce computing resources and reduce computing power consumption. At the same time, it is trained on its own off-road road dataset, which contains off-road terrain data in multiple scenes and all-day conditions. The trained model can complete image segmentation and target detection tasks on complex off-road roads. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a flow chart of the integrated method for image segmentation and target detection for off-road roads of the present invention;

[0031] Figure 2 Schematic diagram of the detection model. DETAILED DESCRIPTION

[0032] In order to make the purpose, content, and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below with reference to the accompanying drawings and examples.

[0033] like Figure 1 As shown, the integrated image segmentation and target detection method for off-road roads proposed in this embodiment includes the following steps:

[0034] Step 1: Preprocess the image, which mainly includes adjusting the image resolution, the mean and variance of the image pixel values, and the image pass sequence.

[0035] Image resolution adjustment: The present invention has a fixed requirement for the input resolution of the image, which is 640×640. Therefore, the resolution of the input image needs to be adjusted to ensure that the output result of the algorithm model is a uniform fixed value.

[0036] Image pixel mean and variance adjustment: By adjusting the pixel mean and variance of the detection image, the data sample distribution of the input image is made close to the data samples during training, which can be more conducive to the algorithm model's recognition and segmentation of the image.

[0037] Image channel order adjustment: Usually the input image is in RGB sequential format, but the algorithm model usually requires the input image channel order to be BGR. Therefore, to ensure the applicability of the model, the image channel order will be judged in the preprocessing stage.

[0038] Step 2: Model improvement: Combine yolov8 with Yolcat to achieve integrated design of target detection and segmentation.

[0039] The core idea of ​​combining yolov8 and Yolcat to achieve integrated target detection and segmentation is to make full use of the advantages of both in feature extraction, detection and segmentation capabilities. Through reasonable structural design and task coordination, the model can complete target detection and segmentation tasks simultaneously in one forward propagation.

[0040] In principle, yolov8 has powerful target detection capabilities. Its efficient backbone network and multi-scale feature fusion structure can quickly and accurately locate and identify target objects, while Yolcat performs well in instance segmentation, especially in small target segmentation and mask generation.

[0041] ① Feature extraction stage:

[0042] The backbone network and Neck structure of yolov8 are used to extract features of the input image to obtain rich feature maps of different scales, which contain semantic information and spatial information at different levels in the image.

[0043] ②Instance segmentation module

[0044] The above feature maps are introduced into Yolcat’s instance segmentation module and fused with the features extracted by Yolcat’s own feature extraction network (ResNet-101+FPN), combining the advantages of yolov8 in target detection with the advantages of Yolcat in instance segmentation.

[0045] ③Task execution phase

[0046] The integrated model fuses the feature maps and performs target detection and segmentation tasks simultaneously. For target detection tasks, the detection head structure of yolov8 is used to directly predict target categories and bounding boxes. For segmentation tasks, instance masks are generated with the help of Yolcat's ProtoNet and Prediction Head structures. By sharing features and executing detection and segmentation tasks in parallel, the error accumulation and inefficiency caused by the traditional method of first detecting and then segmenting are avoided.

[0047] Step 3: The algorithm model calculates the input image. The model algorithm is an improved yolov8 algorithm model. The model outputs three types of high-dimensional matrices, namely the type in the specified area of ​​the corresponding image, the center point and width and height information of the target in the image, and the label mask value in the specified area of ​​the corresponding image.

[0048] The categories within the specified area of ​​the corresponding image: The matrix contains the categories of different targets detected in the image, including passable areas (roads), impassable areas (fixed obstacles such as bushes, walls, cliffs, etc.), inaccessible areas (such as the sky), pedestrians, vehicles, and irregular obstacles.

[0049] The center point, width, and height information of the target in the corresponding image: Contains the position and size information of the target at the resolution of the preprocessed image.

[0050] The label mask value corresponding to the specified area of ​​the image: contains the category to which the corresponding pixels in the specified area belong, which is used for image segmentation.

[0051] Step 4: Post-processing of results:

[0052] Post-processing of the results involves solving and analyzing the three types of matrix information output in step 2 above. The target detection task is performed using the category matrix within the specified image region and the matrix containing the center point, width, and height information of the target within the image. Image segmentation is performed using the category matrix within the specified image region and the label mask values ​​within the specified image region. An inverse operation is performed based on the image resolution scaling value from step 1, restoring the solution to the original image size.

[0053] Step 5: Result display: The results of image segmentation and target detection are displayed uniformly in the original image. The target detection task only displays the results of non-road condition information, including pedestrians, vehicles, and irregular obstacles.

[0054] The key points of this embodiment are:

[0055] 1. This paper combines target detection and image segmentation tasks, improves the algorithm model, and tests the yolo series target detection model. The yolov8 target detection model is optimized and improved based on the poor support effect of new target detection models such as v9 and v10 on edge computing devices and the development level of related models. Figure 2 As shown in the red dotted box, an image segmentation branch module is added to the yolov8 target detection model, so that the model can output target detection and image segmentation results at the same time.

[0056] 2. A new backbone network design, based on yolov8 and Yolcat, introduces an attention-based convolution module (ACM). On the basis of traditional convolution, it incorporates channel attention and spatial attention mechanisms. Specifically, the channel attention mechanism weights the channel dimension of the feature map, enabling the model to pay more attention to important channel information, highlighting target-related features to help more accurately locate the target.

[0057] 3. A hierarchical and progressive design approach is adopted in the backbone network structure. In the shallow network, the focus is on image details, extracting rich low-level features through smaller convolution kernels and more convolution layers. In the intermediate network, dilated convolution is used to expand the field of view, enabling the model to obtain a wider range of contextual information, thereby better handling occluded objects and objects in complex scenes.

[0058] 4. In deep networks, residual connections and skip connections are introduced. This connection method helps solve the gradient vanishing problem and enables the model to fuse features at different levels and enhance the expressiveness of features.

[0059] 5. The present invention integrates the image segmentation branch into the yolov8 target detection algorithm model, adopts the same category loss function as yolov8 during training, and performs weighted processing on different loss functions during training calculation to balance the training directionality of the model.

[0060] 6. The present invention is trained on a local off-road dataset, which is all-terrain, all-weather, and has a large amount of data. This enables the trained model to have good generalization and adapt to a variety of complex off-road terrain environment perception tasks.

[0061] The present invention can achieve a detection rate of 60FPS in a GTX1660super ordinary graphics card and output image segmentation and target detection results at the same time.

[0062] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An integrated method for image segmentation and target detection for off-road roads, characterized in that: The following steps are involved: Step 1: Preprocess the image: The input image resolution is adjusted to 640×640, and the pixel mean and variance of the test image are adjusted so that the data sample distribution of the input image is close to the data sample during training; Adjust the input image channels to BGR sequential format; Step 2: Combine yolov8 and Yolcat models to realize integrated design of target detection and segmentation: ① Feature extraction stage: The backbone network and Neck structure of yolov8 are used to extract features of the input image to obtain feature maps of different scales, which contain semantic information and spatial information at different levels in the image. ②Instance segmentation module The above feature map is introduced into the instance segmentation module of Yolcat, and fused with the features extracted by Yolcat's own feature extraction network to obtain the fused feature map; ③Task execution phase The integrated model fuses the feature maps and performs both target detection and segmentation tasks. For target detection, the detection head structure of yolov8 is used to directly predict the target category and bounding box. For segmentation, the ProtoNet and Prediction Head structure of Yolcat are used to generate instance masks. Step 3: The algorithm model calculates the input image: The algorithm model adds an image segmentation branch module based on the yolov8 target detection model. The model outputs three types of high-dimensional matrices, namely, the type of object in the specified area of ​​the image, the center point and width and height information of the object in the image, and the label mask value in the specified area of ​​the image. The types of objects in the designated area of ​​the corresponding image include passable areas, impassable areas, unreachable areas, pedestrians, vehicles, and irregular obstacles; The center point and width and height information of the target in the corresponding image include the position and size information of the target at the resolution of the preprocessed image; The label mask value in the specified area of ​​the corresponding image contains the category to which the corresponding pixel in the specified area belongs; Step 4: Post-processing of results: By solving and analyzing the three types of high-dimensional matrix information output in step 3, the target detection task is completed using the category matrix in the corresponding image specified area and the center point and width and height information matrix of the target in the corresponding image; the image segmentation task is completed using the category matrix in the corresponding image specified area and the label mask value in the corresponding image specified area; the inverse operation is performed according to the image resolution scaling value in step 1, and the solution result is restored to the original image size; Step 5. Results: The results of image segmentation and target detection are displayed uniformly in the original image. The target detection task only displays the results of non-road condition information, including pedestrians, vehicles, and irregular obstacles.

2. The integrated method for off-road image segmentation and target detection according to claim 1, wherein: The algorithm model in step 3 introduces a convolution module based on the attention mechanism based on yolov8 and Yolcat. This convolution module incorporates channel attention and spatial attention mechanisms on the basis of traditional convolution.

3. The integrated method for off-road image segmentation and target detection according to claim 2, wherein: The mechanism of channel attention is to weight the channel dimension of the feature map.

4. The integrated method for off-road image segmentation and target detection according to claim 1, wherein: The algorithm model adopts a hierarchical and progressive design approach: in the shallow network, it focuses on the detailed information of the image and extracts rich low-level features through smaller convolution kernels and more convolution layers; in the intermediate network, dilated convolution is used to expand the field of view, enabling the model to obtain a wider range of contextual information, thereby better handling occluded targets and targets in complex scenes.

Images