Obstacle recognition method based on machine vision

Abstract

The application discloses a kind of obstacle identification methods based on machine vision, it is related to image processing method technical field.The method includes the following steps: the region of warning line is determined to the image collected, then the mask of warning line region is generated, and the warning line region is extracted from the whole image using the mask;Edge extraction is carried out on the warning line region, and the edge of non-warning line is filtered out after filtering the extracted edge;The warning line is extracted according to color characteristics by converting the warning line region to hsv space, and the complete warning line is obtained by edge filling to the extracted warning line;Finally, the profile of warning line is extracted.The method can more fully utilize the color characteristics and edge characteristics of warning line, more accurately and completely identify and mark the warning line, while reducing labor cost and improving efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing method, and particularly relates to a barrier identification method based on machine vision. BACKGROUND

[0002] With the rapid development of machine vision and artificial intelligence technology, the recognition and processing ability of specific environmental characteristics has become the key of the system. As an important marking line, the warning line is widely used in the environment such as roads, factories, public places and the like, and is used for indicating the safe operation area of personnel or machine equipment. The present barrier identification method mostly depends on image processing, manual inspection and simple sensor detection. These methods currently face not only low efficiency but also low recognition accuracy, and even misjudgment and omission problems. Among them, the method of using image processing to identify the warning line usually involves binarization and edge detection processing of the image captured by the camera to identify the position of the warning line. However, this method does not fully utilize the characteristics of the warning line and cannot accurately and completely identify the warning line, especially when the edge and contour of the warning line are affected under complex conditions such as wear of the warning line, the recognition ability is greatly reduced.

[0003] In summary, in the identification of the warning line, the identification of the warning line based on machine vision currently faces the following problems:

[0004] 1) In addition to the warning line, there are many objects in the collected image, the distribution of these objects is irregular and complex, and it has a great influence on the subsequent extraction of the warning line.

[0005] 2) When extracting the edge of the image, in order to obtain more edge information, the threshold value in edge extraction can be reduced, but at the same time, many non-warning line edges will also be extracted, which will make the subsequent operation more complex, and under these influences, it is difficult to finally extract only the warning line and other objects may be extracted together.

[0006] 3) The present warning line identification mostly depends on edge detection processing, but when the edge of the warning line is damaged, it will seriously affect the recognition effect of the warning line, and it is difficult to fully utilize the characteristics of the warning line by using this method. The above problems will seriously affect the accurate identification of the warning line. SUMMARY

[0007] The technical problem to be solved by the present application is how to provide a method capable of accurately and efficiently identifying the warning line.

[0008] To solve the above technical problems, the technical scheme adopted by the present application is: a barrier identification method based on machine vision, comprising the following steps:

[0009] S1: determining and extracting a warning line region from the collected image;

[0010] S2: performing edge extraction on the warning line region;

[0011] S3: extracting a corresponding warning line region according to color features;

[0012] S4: filling the edge-extracted image with the extracted warning line region;

[0013] S5: extracting a warning line contour and identifying the warning line.

[0014] Further, the method comprises the following steps:

[0015] S1: extracting a warning line region from the collected image. First, the region where the warning line is located is determined according to color features, then a mask is generated for the region, and finally the region where the warning line is located is extracted. This reduces the interference of other contents in the image and facilitates further operations.

[0016] S2: performing edge extraction on the warning line region. The Canny operator is used to perform edge extraction on the extracted warning line region, and then the extracted edges are filtered to filter out horizontal and vertical line segments that are not warning line edges, so that the warning line edges are more accurate.

[0017] S3: extracting a warning line according to color features. The color features of the warning line are analyzed, the warning line region is converted to the hsv space, and the warning line is extracted according to the color.

[0018] S4: filling the warning line edge extracted in step S2 with the warning line extracted in step S3.

[0019] S5: extracting a warning line contour. The warning line image filled in step S4 is subjected to contour extraction, the extracted contour is sorted according to the contour length, the warning line is extracted, and finally the warning line is identified and marked.

[0020] The beneficial effects produced by the above technical solutions are: 1) The method introduces region extraction with "determining a warning line region-generating a mask-extracting a warning line region" as the main line, and extracts the warning line region. This eliminates the influence of regions other than the warning line on subsequent operations, making the subsequent operation of the image more focused on the target object, greatly improving the usability.

[0021] 2) The method of the present application introduces a filtering method for the image after edge extraction, which is mainly based on "constructing convolution kernel-extracting horizontal and vertical line segments-generating mask-filtering out line segments of non-warning line edges". By constructing a specific convolution kernel according to the size of the image height and width and using morphological operation to extract the horizontal and vertical line segments of the non-warning line edges, the position information of the line segments is obtained by Hough transform, then the mask is generated according to the position information, and finally the line segments of the non-warning line edges are filtered out. The problem of the influence of redundant information on subsequent operation in ordinary edge extraction is overcome.

[0022] 3) The method of the present application introduces a method of combining color features with edge features to extract the warning line contour. The edge extraction is used to determine the approximate contour of the warning line, and the color feature extraction is used to supplement the details of the warning line. The warning line extracted by color feature extraction is used to fill the warning line contour obtained by edge feature extraction, and finally a relatively complete contour is obtained. The problem of incomplete extraction of warning line contour is effectively solved. In summary, the method can accurately and efficiently complete the recognition of warning line. BRIEF DESCRIPTION OF DRAWINGS

[0023] The present application will be further described in detail below in combination with the drawings and specific embodiments.

[0024] Figure 1 The flowchart of the method of the embodiment of the present application;

[0025] Figure 2 The initial warning line img image in the method of the embodiment of the present application;

[0026] Figure 3 The hsv_img image converted to hsv space in the method of the embodiment of the present application;

[0027] Figure 4 The mask1 image after threshold processing of the hsv space image in the method of the embodiment of the present application;

[0028] Figure 5 The canny_img1 image after edge extraction in the method of the embodiment of the present application;

[0029] Figure 6 The same_img image after initialization in the method of the embodiment of the present application;

[0030] Figure 7 The bit_img image obtained after the operation in the method of the embodiment of the present application;

[0031] Figure 8 The canny_bit image after edge extraction in the method of the embodiment of the present application;

[0032] Figure 9 mask_reh image after the etching operation in the method described in the embodiments of the present application;

[0033] Figure 10 mask_rev image after the etching operation in the method described in the embodiments of the present application;

[0034] Figure 11 canny_img image after the operation in the method described in the embodiments of the present application;

[0035] Figure 12 mask_b image after the threshold processing in the method described in the embodiments of the present application;

[0036] Figure 13 mask_black image after the operation in the method described in the embodiments of the present application;

[0037] Figure 14 dilation_mask image after the dilation operation in the method described in the embodiments of the present application;

[0038] Figure 15 same_img2 image after the initialization in the method described in the embodiments of the present application;

[0039] Figure 16 img_end image after the outline drawing in the method described in the embodiments of the present application. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0041] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be practiced according to other embodiments that do not require the specific details described herein. Therefore, the present application is not limited to the embodiments described herein but rather encompasses all embodiments that are within the scope of the present application.

[0042] As shown in FIG. 1, the embodiments of the present application disclose a method for obstacle recognition based on machine vision, which comprises the following steps: Figure 1

[0043] ​S1: Extract the warning line region from the collected image. First, determine the location of the warning line region according to color features, then generate a mask for the region, and finally extract the region where the warning line is located. This reduces the interference of other content in the image and facilitates further operations.

[0044] S2: Edge extraction on the warning line region. Use the Canny operator to perform edge extraction on the extracted warning line region, and then filter the extracted edges to remove horizontal and vertical line segments that are not warning line edges, making the warning line edges more accurate.

[0045] S3: Extract the warning line according to the color feature. Analyze the color feature of the warning line, convert the warning line region to the hsv space, and extract the warning line according to the color.

[0046] S4: Fill the warning line edge extracted in step S2 with the warning line extracted in step S3.

[0047] S5: Extract the warning line contour. Perform contour extraction on the warning line image filled in step S4, sort the extracted contours according to their perimeter, extract the warning line, and finally identify and mark the warning line.

[0048] Further, the step S1 specifically comprises:

[0049] S11: Preprocess the collected image.

[0050] S12: Determine the region of the warning line according to the color feature.

[0051] S13: Generate a mask for the determined warning line region.

[0052] S14: Extract the warning line region according to the mask generated in step S13.

[0053] In the step S11, the preprocessing of the image includes first performing median filtering on the collected image, and then performing bilateral filtering to remove noise.

[0054] The step S12 specifically comprises: first, converting the preprocessed warning line image to the hsv space; then generating a threshold value according to the color feature of the warning line, and performing binaryzation on the warning line image in the hsv space according to the threshold value; and then performing Hough transform on the binaryzation image to find straight lines, sorting all found straight lines according to the vertical coordinate to obtain the straight line with the smallest vertical coordinate, and obtaining the position information of the straight line in the image to determine the warning line region.

[0055] The step S13 specifically includes: first generating a matrix area containing the warning line according to the position information of the straight line found in the step S12, then constructing an image with the same width and height as the warning line image, and initializing the constructed image to set the matrix area as white and other areas as black. Finally, a black and white image is obtained as a mask. The formula for generating the mask is as follows:

[0056]

[0057] wherein the image same_img is the constructed image with the same width and height as the warning line image, y1 is the ordinate of the straight line mentioned above, t1 and t2 are floating values used to generate the matrix area containing the warning line, and h_con is the height of the warning line area.

[0058] The step S14 specifically includes performing an AND operation between the mask generated in the step S13 and the collected warning line image, so as to finally achieve the effect of extracting the warning line area.

[0059] Further, the step S2 specifically includes:

[0060] S21: performing edge extraction on the extracted warning line area;

[0061] S22: obtaining a horizontal line segment of a non-warning line edge in the image after edge extraction;

[0062] S23: obtaining a vertical line segment of a non-warning line edge in the image after edge extraction;

[0063] S24: generating a mask by using the horizontal and vertical line segments extracted in the steps 22 and 23;

[0064] S25: filtering out the line segment of the non-warning line edge by using the mask generated in the step 24;

[0065] The step S21 specifically includes: first converting the warning line area extracted in the step S14 into a gray-scale image, and then performing edge extraction on the gray-scale image by using a Canny operator.

[0066] The step S22 specifically includes: first constructing a convolution kernel kernel_hors with a size of hors_k*1, and then performing an erosion operation on the image after edge extraction by using the convolution kernel to obtain the horizontal line segment of the non-warning line edge.

[0067]

[0068] wherein hors_k is the size of the constructed convolution kernel, width is the width of the warning line image, and kernel_hors is the specific convolution kernel used.

[0069] The step S23 specifically includes, first constructing a 1*vert_k convolution kernel kernel_vert, and then using the convolution kernel to perform erosion operation on the edge extracted image to obtain the vertical line segment of the non-warning line edge.

[0070]

[0071] Wherein vert_k is the size of the constructed convolution kernel, height is the width of the warning line image, and kernel_vert is the specific used convolution kernel.

[0072] The step S24 specifically includes, performing Hough transform on the horizontal and vertical line segments obtained in the step S22 and the step S23 to find straight lines to obtain the position information of each line segment, and finally making a mask according to the position information of each line segment, constructing an image with the same width and height as the warning line image, and initializing the constructed image, setting the area of the line segment as black and the other areas as white.

[0073] The step S25 specifically includes performing AND operation on the mask obtained in the step S24 and the edge extracted image in the step S21 to filter out the line segment of the non-warning line edge.

[0074] Further, the method will be described in detail in combination with specific data and the accompanying drawings as follows:

[0075] The step S1 specifically includes the following steps:

[0076] S11: performing median filtering on the collected image img to obtain an image middle_img. Figure 2 The median filtering is performed to obtain the image middle_img.

[0077] S12: performing bilateral filtering on the image middle_img to obtain an image bilater_img.

[0078] S13: converting the image bilater_img into an hsv space image to obtain an hsv_img. Figure 3

[0079] S14: setting a threshold according to the value of yellow in the hsv, wherein the lower limit of the threshold is low(min_h, min_s, min_v), and the upper limit of the threshold is high(max_h, max_s, max_v).

[0080]

[0081] ​Wherein, hsv yellow is the hsv value of yellow in hsv space, hsv min is the minimum hsv value that can detect yellow in hsv space, hsv max is the maximum hsv value that can detect yellow in hsv space, (min_h, min_s, min_v) is the specific value of low, which is set to (15, 57, 200) here, (max_h, max_s, max_v) is the specific value of high, which is set to (30, 255, 255) here.

[0082] S15: Threshold processing is performed on the image hsv_img to obtain the image mask1, as shown in the following formula: Figure 4 Wherein, the lower limit of the threshold is low (min_h, min_s, min_v), and the upper limit of the threshold is high (max_h, max_s, max_v). The image within the range is changed to white, and the image outside the range is changed to black. The threshold formula is as follows:

[0083]

[0084] S16: The mask1 is dilated using a 3*3 convolution kernel, and the iteration number is iterated once to obtain the image dilation_img. The iteration number determination formula is as follows:

[0085]

[0086] Wherein, iterated is the iteration number of dilation, s_w is the area of the white region, and size_kmax is the size of the convolution kernel, which is set to 10 here.

[0087] S17: The Canny operator is used on the dilation_img to perform edge operation to obtain the image canny_img1, as shown in the following formula: Figure 5

[0088] S18: The Hough transform is performed on the image canny_img1 to find straight lines to obtain all straight lines lines, and save them to the lines_list list.

[0089] S19: The lines_list list is sorted according to the y coordinate to obtain the straight line where the point dot (x1, y1) with the minimum y coordinate value is located.

[0090] S110: A black and white image same_img with the same width and height as the image img is initialized, as shown in the following formula: Figure 6

[0091]

[0092] ​​Wherein, same_img(x, y) is the value of x, y coordinate of image same_img, y1 is the vertical coordinate of dot, w_con is the height of the warning line area, t1, t2 are the floating range, t1 is 30 and t2 is 270.

[0093] Further, the step S2 specifically comprises the following steps:

[0094] S21: converting the image img into a gray image gray;

[0095] S22: performing AND operation on the gray image gray by taking the image same_img as a mask to obtain an image bit_img, as shown in the following formula: Figure 7

[0096]

[0097] Wherein, bit_img(x, y) is the value of x, y coordinate of image bit_img, same_img(x, y) is the value of x, y coordinate of image same_img, and dilate_img(x, y) is the value of x, y coordinate of image dilate_img.

[0098] S23: performing edge operation on the image bit_img by using Canny operator to obtain an image canny_bit, as shown in the following formula: Figure 8

[0099] S24: performing Hough transform on the image canny_bit to find longer straight lines, and storing the coordinates of the starting point and the ending point into a list lines.

[0100] S25: generating a mask for the coordinates of each line segment in lines to obtain an image mask_reline, as shown in the following formula:

[0101]

[0102] Wherein, mask_reline(x, y) is the value of x, y coordinate of image mask_reline, and y1 is the vertical coordinate of the starting point of each line segment.

[0103] S26: constructing a convolution kernel kernel_hors of hors_k*1 to perform itera times of erosion operation on mask_reline to obtain a horizontal short line segment, and obtaining an image mask_reh, as shown in the following formula: Figure 9

[0104]

[0105] ​​​Wherein, width is the width of the image img, kernel_hors is the used convolution kernel, itera is the iteration number, s_d is the area of the black region, size_kmin is the size of the convolution kernel, which is set to 1 here.

[0106] S27: Perform Hough transform on the image mask_reh, and store the found straight lines into the list list_line1.

[0107] S28: Generate the mask mask_h according to the straight lines in list_line1, and the formula is as follows:

[0108]

[0109] Wherein, mask_h(x, y) is the value of the horizontal and vertical coordinates of the image mask_h, width is the width of the image, and y1 is the vertical coordinate of the line segment in list_line1.

[0110] S29: Construct the convolution kernel kernel_vert of 1*vert_k and perform erosion operation with the image mask_reline to obtain the shorter vertical line segment, and the iteration number is itera2, to obtain the image mask_rev, as shown in Figure 10 .

[0111] Wherein, itera2 is the iteration number 1 of the expansion, s_d is the area of the black region, size_kmin is the size of the convolution kernel, which is set to 1 here, height is the height of the image img, and kernel_vert is the specific convolution kernel.

[0112] S210: Perform Hough transform on the image mask_rev, and store the found straight lines into the list list_line2.

[0113] S211: Generate the mask mask_v according to the straight lines in list_line2, and the formula is as follows:

[0114]

[0115] Wherein, mask_v(x, y) is the value of the horizontal and vertical coordinates of the image mask_v, height is the width of the image, and x1 is the vertical coordinate of the line segment in list_line2.

[0116] S212: Perform AND operation on the image mask_h and the image mask_v to obtain the image canny_img, as shown in Figure 11 .

[0117] Further, the step S3 specifically comprises the following steps:

[0118] S31: Set the threshold according to the value of black in hsv, wherein the lower limit of the threshold is blacklow(min2_h, min2_s, min2_v), and the upper limit of the threshold is blackhigh(max2_h, max2_s, max2_v)

[0119]

[0120] wherein hsv_black is the hsv value of black in hsv space, hsvblack_min is the minimum hsv value in hsv space that can detect black, hsvblack_max is the maximum hsv value in hsv space that can detect black, (min2_h, min2_s, min2_v) is the specific value of blacklow, which is set to (58, 1, 46) here, and (max2_h, max2_s, max2_v) is the specific value of blackhigh, which is set to (174, 40, 136) here.

[0121] S32: Threshold processing is performed on the image hsv_img to obtain the image mask_b, as shown in the following formula: Figure 12

[0122]

[0123] wherein the lower limit of the threshold is low1(min2_h, min2_s, min2_v), and the upper limit of the threshold is high1(max2_h, max2_s, max2_v), the image within the range is changed to white, and the image outside the range is changed to black

[0124] S33: The image mask_b and the image same_img are subjected to AND operation to obtain the image mask_black, as shown in the following formula: Figure 13

[0125]

[0126] wherein mask_black(x, y) is the value of the x, y coordinate of the mask_black image, same_img(x, y) is the value of the x, y coordinate of the same_img image, and mask_b(x, y) is the value of the x, y coordinate of the mask_b image.

[0127] Further, the step S4 specifically comprises the following steps:

[0128] S41: The image mask1 and the image canny_img are subjected to AND operation to obtain the image bit_yellow.​​

[0129]

[0130] bit yellow(x, y) is the value of the x, y coordinates of the bit yellow image, canny img(x, y) is the value of the x, y coordinates of the canny img image, and mask 1(x, y) is the value of the x, y coordinates of the mask 1 image.

[0131] S42: Perform an AND operation on the image mask b and the image bit yellow to obtain an image bit b.

[0132]

[0133] bit b(x, y) is the value of the x, y coordinates of the image bit b, bit yellow(x, y) is the value of the x, y coordinates of the bit yellow image, and mask b(x, y) is the value of the x, y coordinates of the mask b image.

[0134] S43: Perform an inflation operation on bit b using a 7*7 convolution kernel, with the number of iterations being itera3 times, to obtain an image dilation mask, as shown in the following formula: Figure 14

[0135]

[0136] itera3 is the number of iterations of inflation, s w is the area of the white region, and size kmin is the size of the convolution kernel, which is set to 2 here.

[0137] Further, the step S5 extracts the alert line contour, and specifically includes the following steps:

[0138] S51: Find the contour of the image dilation mask to obtain all contours saved in a contours list.

[0139] S52: Sort the contours list according to the perimeter of the contour to obtain a contour with the largest perimeter.

[0140] S53: Initialize a black and white image same img2 with the same width and height as the image img, as shown in the following formula: Figure 15

[0141]

[0142] ​​Wherein, same_img2(x, y) is the value of x, y coordinate of image same_img2, contour(x, y) is the coordinate value of x, y of each point in contour.

[0143] S54: finding contour of same_img2 and drawing contour to obtain final image img_end as Figure 16 .

[0144] The method can more fully utilize the color characteristics and edge characteristics of the warning line, more accurately and completely identify and mark the warning line, and reduce the labor cost and improve the efficiency.

Claims

1. A machine vision-based obstacle recognition method, characterized in that The steps include: S1: Determine and extract the warning line area from the collected image; The step S1 specifically includes: Preprocessing the collected images includes first performing median filtering on the collected images and then performing bilateral filtering to remove noise; First, the pre-processed warning line image is converted to HSV space. Then, a threshold is generated according to the color characteristics of the warning line, and the warning line image in HSV space is binarized according to the threshold. Then, the binarized image is subjected to Hough transform to find a straight line. All the found straight lines are sorted according to the ordinate to obtain the line with the smallest ordinate. The position information of the line in the image is obtained to determine the warning line area. First, a matrix area containing the warning line is generated based on the position information of the found straight line. Then, an image with the same width and height as the warning line image is constructed. The constructed image is initialized to set the matrix area to white and the other areas to black. Finally, a black and white image is obtained as a mask. The generated mask is then ANDed with the collected warning line image to extract the warning line area. S2: Extract the edge of the warning line area; The step S2 specifically includes: First, the extracted warning line area is converted into a grayscale image, and then the Canny operator is used to extract the edge of the grayscale image; Obtaining the horizontal line segments of the non-warning line edges in the image after edge extraction; Obtaining the longitudinal line segments of the non-warning line edges in the image after edge extraction; Perform Hough transform on the obtained horizontal and vertical line segments to find the straight line and obtain the position information of each line segment. Finally, create a mask based on the position information of each line segment to construct an image with the same width and height as the warning line image. Initialize the constructed image, set the line segment area to black, and the other areas to white. The obtained mask is then ANDed with the edge-extracted image to filter out the line segments that are not on the edge of the warning line. S3: Extract the corresponding warning line area based on color features; S4: Filling the edge-extracted image with the extracted warning line area; S5: Perform contour extraction on the warning line image filled in step S4 to identify the warning line.

2. The obstacle recognition method based on machine vision according to claim 1, characterized in that: The S1 specifically includes the following steps: S11: Filter the collected image img to obtain the image middle_img; S12: Filter and denoise the filtered image middle_img to obtain the image bilater_img; S13: convert the image bilater_img into an hsv space image to obtain an image hsv_img; S14: Set a threshold value based on the value of yellow in hsv, where the lower limit of the threshold is low (min_h, min_s, min_v) and the upper limit of the threshold is high (max_h, max_s, max_v) Among them, hsv_yellow is the HSV value of yellow in the HSV space, hsv_min is the minimum HSV value that can detect yellow in the HSV space, hsv_max is the maximum HSV value that can detect yellow in the HSV space, (min_h, min_s, min_v) are the specific values ​​of low, and (max_h, max_s, max_v) are the specific values ​​of high; S15: Perform threshold processing on the image hsv_img to obtain the image mask1, where the lower limit of the threshold is low (min_h, min_s, min_v) and the upper limit of the threshold is high (max_h, max_s, max_v). The image within the threshold range is turned white, and the image outside the threshold range is turned black. The threshold formula here is: S16: Use a 3*3 convolution kernel to dilate mask1, the number of iterations is itera1, and the image dilation_img is obtained. The formula for determining the number of iterations is: Where itera1 is the number of dilation iterations, s_w is the area of ​​the white region, and size_kmax is the size of the convolution kernel; S17: Use the Canny operator to perform edge processing on the image dilation_img to obtain the image canny_img1; S18: Perform Hough transform on the image canny_img1 to find all the straight lines and save them in the lines_list list; S19: Sort the lines_list list according to the y coordinate to obtain the line where the point dot(x1, y1) with the smallest y coordinate value is located; S110: Initialize a black and white image same_img with the same width and height as the image img. The initialization formula is: Among them, same_img(x,y) is the x and y coordinate values ​​of the same_img image, y1 is the ordinate of the dot point, w_con is the height of the warning line area, and t1 and t2 are floating ranges; S111 , using the black and white image same_img as a mask to perform an AND operation on the collected warning line image img to extract the warning line area.

3. The obstacle recognition method based on machine vision according to claim 1, characterized in that: The S2 specifically includes the following steps: S21: Convert the image img into a grayscale image gray; S22: Use image same_img as a mask to perform AND operation on the grayscale image gray to obtain image bit_img; Among them, bit_img(x,y) is the value of the x,y coordinates of the bit_img image, same_img(x,y) is the value of the x,y coordinates of the same_img image, dilate_img(x,y) is the value of the x,y coordinates of the dilate_img image, and gray(x,y) is the x,y coordinates of the grayscale image gray; S23: Use the Canny operator to perform edge operation on the image bit_img to obtain the image canny_bit; S24: Perform Hough transform on the image canny_bit to find a longer straight line and store the coordinates of the starting point and the end point in the lines list; S25: Generate a mask for the coordinates of each line segment in the lines list to obtain the image mask_reline formula as follows: Among them, mask_reline(x,y) is the x and y coordinate values ​​of the image mask_reline, and y2 is the ordinate of the starting point of each line segment; S26: Construct the convolution kernel kernel_hors of hors_k*1 and perform itera corrosion operations with mask_reline to obtain a shorter horizontal line segment to obtain the image mask_reh, where: Where width is the width of the image img, kernel_hors is the convolution kernel used, itera is the number of iterations, s_d is the area of ​​the black area, and size_kmin is the size of the convolution kernel; S27: Perform Hough transform on the image mask_reh, and store the found straight line in the list list_line1; S28: The formula for generating mask mask_h based on the straight line in list_line1 is as follows: Among them, mask_h(x,y) is the value of the horizontal and vertical coordinates of the image mask_h, width is the width of the image img, and y3 is the vertical coordinate of the line segment in list_line1; S29: Construct the convolution kernel kernel_vert of 1*vert_k and perform corrosion operation with mask_reline to obtain shorter vertical line segments. The number of iterations is itera2 to obtain the image mask_rev. Among them, itera2 is the number of dilation iterations, s_d is the area of ​​the black area, size_kmin is the size of the convolution kernel, height is the height of the image img, and kernel_vert is the specific convolution kernel; S210: Perform Hough transform on the image mask_rev and store the found straight line in list list_line2 S211: The formula for generating mask mask_v based on the straight line in list_line2 is as follows: Among them, mask_v(x,y) is the value of the horizontal and vertical coordinates of the image mask_v, height is the height of the image img, and x1 is the horizontal coordinate of the line segment in list_line2; S212: Perform an AND operation on the image mask_h and the image mask_v to obtain the image canny_img.

4. The obstacle recognition method based on machine vision according to claim 1, characterized in that: The S3 specifically includes the following steps: S31: Set the threshold value according to the value of black in hsv, where the lower limit of the threshold value is blacklow (min2_h, min2_s, min2_v) and the upper limit of the threshold value is blackhigh (max2_h, max2_s, max2_v) Among them, hsv_black is the HSV value of black in the HSV space, hsvblack_min is the minimum HSV value that can detect black in the HSV space, hsvblack_max is the maximum HSV value that can detect black in the HSV space, (min2_h, min2_s, min2_v) are the specific values ​​of blacklow, and (max2_h, max2_s, max2_v) are the specific values ​​of blackhigh; S32: Perform threshold processing on the image hsv_img to obtain the image mask_b. The threshold formula here is: Among them, the lower limit of the threshold is low1 (min2_h, min2_s, min2_v), and the upper limit of the threshold is high1 (max2_h, max2_s, max2_v). The image within the threshold range will be turned white, and the image outside the range will be turned black; S33: Perform an AND operation on the image mask_b and the image same_img to obtain the image mask_black; Among them, mask_black(x,y) is the value of the x,y coordinates of the mask_black image, same_img(x,y) is the value of the x,y coordinates of the same_img image, and mask_b(x,y) is the value of the x,y coordinates of the mask_b image.

5. The obstacle recognition method based on machine vision according to claim 1, characterized in that: The S4 specifically includes the following steps: S41: performing an AND operation on the image mask1 and the image canny_img to obtain the image bit_yellow; Among them, bit_yellow(x,y) is the value of the x,y coordinates of the bit_yellow image, canny_img(x,y) is the value of the x,y coordinates of the canny_img image, and mask_1(x,y) is the value of the x,y coordinates of the mask_1 image; S42: performing an AND operation on the image mask_black and the image bit_yellow to obtain the image bit_b; Among them, bit_b(x,y) is the value of the x,y coordinates of the bit_b image, bit_yellow(x,y) is the value of the x,y coordinates of the bit_yellow image, and mask_black(x,y) is the value of the x,y coordinates of the mask_black image; S43: Use a 7*7 convolution kernel to dilate bit_b, and the number of iterations is itera3 to obtain the image dilation_mask: Among them, itera3 is the number of dilation iterations, s_w is the area of ​​the white area, and size_kmin is the size of the convolution kernel.

6. The obstacle recognition method based on machine vision according to claim 1, characterized in that: The S5 specifically includes the following steps: S51: Search for contours in the image dilation_mask and save all the contours in the contours_list list; S52: Sort the contours_list list according to the perimeter of the contours to obtain the contour with the largest perimeter; S53: Initialize a black and white image same_img2 with the same width and height as the image img. The initialization formula is: Among them, same_img2(x,y) is the x, y coordinate value of the same_img2 image, and contour(x,y) is the x, y coordinate value of each point in the contour; S54: Find the contour of the image same_img2 and draw the contour to obtain the final image img_end marked with a warning line.

Images