Obstacle contour detection method, device, terminal equipment and storage medium

Abstract

The embodiments of the present invention disclose an obstacle contour detection method, apparatus, terminal device and storage medium. The obstacle contour detection method obtains an actual image of an actual scene through a monocular camera, obtains an actual bird's-eye view through perspective transformation, calculates the actual feature value of each pixel in the actual bird's-eye view, obtains a binary image based on the open space feature value and the actual feature value of each pixel in the open space bird's-eye view, performs denoising on abnormal points in the binary image to obtain a first image, and performs contour detection on the first image to determine the contour of the obstacle in the actual scene. The detection method of the present application can, on the one hand, more accurately identify the size and shape features of the obstacle, and on the other hand, remove the influence of illumination on image feature recognition. It can be applied to most obstacle feature recognition, avoids the use of a large amount of algorithm training and testing, and thus can reduce production costs and labor costs.

Description

Technical Field

[0001] The present invention relates to the field of obstacle detection, and in particular to an obstacle contour detection method, device, terminal equipment and storage medium. Background Art

[0002] Conventional technology typically uses binocular cameras for obstacle detection, coupled with AI (Artificial Intelligence) recognition algorithms. However, binocular cameras are complex, expensive, and require high computing power. Using AI recognition algorithms requires pre-training and high processor memory and computing power. Furthermore, the need to adapt to varying lighting conditions and obstacles creates complex practical requirements, requiring extensive algorithm training and testing in numerous scenarios, making them complex to use. Summary of the Invention

[0003] In view of this, the present application provides an obstacle contour detection method, apparatus, terminal device and storage medium.

[0004] In a first aspect, the present application proposes a method for detecting obstacle contours using a monocular camera, the method comprising:

[0005] Acquire an actual image of an actual scene through a monocular camera, and perform perspective transformation on the actual image to obtain an actual bird's-eye view;

[0006] Calculate the actual feature value of each pixel in the actual bird's-eye view;

[0007] Performing outlier recognition processing based on the vacant land feature value of each pixel point in the vacant land bird's-eye view image and the actual feature value to obtain a binary image containing the outlier point;

[0008] Performing denoising on abnormal points in the binary image to obtain a first image;

[0009] Perform contour detection on the first image to determine the contours of obstacles in the actual scene.

[0010] In some embodiments, performing contour detection on the first image to determine the contour of the obstacle in the actual scene includes:

[0011] extracting at least one contour in the first image based on a preset algorithm;

[0012] Calculate the center point of each of the contours and the side length and area of ​​the minimum oblique rectangle;

[0013] If the center point of the current contour is not at the boundary of the first image, detecting whether the side length is less than a preset side length threshold, and whether the area is less than a preset area threshold;

[0014] If the side length is less than the side length threshold and the area is less than the area threshold, the current contour is determined to be an interference contour and discarded; otherwise, the current contour is retained;

[0015] If the center point of the current contour is at the boundary of the first image, retaining the current contour;

[0016] The retained contour is used as the obstacle contour.

[0017] In some embodiments, the method of acquiring an actual image of an actual scene through a monocular camera includes:

[0018] Acquire an open space image of an open space scene using a monocular camera, and measure the actual size of the open space scene to determine the aspect ratio of the open space scene;

[0019] Determining the open space coordinates based on the aspect ratio and the resolution of the open space bird's-eye view of the open space scene to be generated;

[0020] Setting corner points at the boundaries of the open space scene, and determining the corresponding corner point coordinates of the corner points in the open space image;

[0021] Calculating parameters of a perspective transformation matrix based on the corner point coordinates and the open space coordinates to determine the perspective transformation matrix;

[0022] The open space image is perspective transformed using the perspective transformation matrix to obtain an open space bird's-eye view of the open space scene.

[0023] In some embodiments, the feature value of each pixel in the corresponding bird's-eye view image is calculated by:

[0024] Perform grayscale processing on the corresponding bird's-eye view image to obtain a grayscale image;

[0025] Calculate the corresponding central brightness and ambient brightness based on each pixel point of the grayscale image as the center;

[0026] A ratio of the central brightness of each pixel to the ambient brightness is used as a characteristic value of the corresponding pixel.

[0027] In some embodiments, the outlier identification process based on the vacant land feature value of each pixel point in the vacant land bird's-eye view image and the actual feature value includes:

[0028] Compare the clearing feature value and the actual feature value of the pixel point at the same position to obtain a corresponding ratio;

[0029] If the ratio is less than or equal to a preset ratio threshold, the pixel point is determined to be the abnormal point; otherwise, the pixel point is determined to be a normal point.

[0030] In some embodiments, performing denoising on the outliers in the binary image to obtain the first image includes:

[0031] performing dilation processing on the binary image at a first pixel size to obtain a second image;

[0032] performing an erosion process of a second pixel size on the second image to obtain a third image;

[0033] performing dilation processing on the third image at a third pixel size to obtain a fourth image;

[0034] performing an erosion process of a fourth pixel size on the fourth image to obtain a first image;

[0035] Among them, the relationship between the first pixel size, the second pixel size, the third pixel size and the fourth pixel size is as follows: the sum of the first pixel size and the third pixel size is equal to the sum of the second pixel size and the fourth pixel size, and the first pixel size is smaller than the second pixel size, which is smaller than the third pixel size.

[0036] In some embodiments, the corrosion treatment includes:

[0037] Aligning the center point of a preset structural element with each abnormal point in the binary image in sequence, and determining whether all pixels in the structural element are abnormal points;

[0038] If all the pixels are abnormal points, the pixel corresponding to the center point is retained;

[0039] Otherwise, the pixel point corresponding to the center point is modified to a normal point.

[0040] In a second aspect, an embodiment of the present application further provides an obstacle contour determination device, comprising:

[0041] A bird's-eye view acquisition module acquires an actual image of an actual scene through a monocular camera and performs perspective transformation on the actual image to obtain an actual bird's-eye view;

[0042] An eigenvalue calculation module calculates the actual eigenvalue of each pixel in the actual bird's-eye view;

[0043] An outlier recognition module performs outlier recognition processing based on the vacant land feature value of each pixel point in the vacant land bird's-eye view image and the actual feature value to obtain a binary image containing the outlier point;

[0044] a denoising processing module, performing denoising on abnormal points in the binary image to obtain a first image;

[0045] The contour determination module performs contour detection on the first image to determine the contours of obstacles in the actual scene.

[0046] In a third aspect, an embodiment of the present application further provides a terminal device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program runs on the processor, it executes any one of the above-mentioned obstacle contour detection methods.

[0047] In a fourth aspect, an embodiment of the present application further provides a readable storage medium storing a computer program, which executes any of the above-mentioned obstacle contour detection methods when running on a processor.

[0048] The embodiments of the present application have the following beneficial effects:

[0049] The embodiments of the present application provide an obstacle contour detection method, apparatus, terminal device, and storage medium. The method uses a monocular camera to obtain an actual image of an actual scene, performs perspective transformation on the actual image to obtain an actual bird's-eye view, calculates the actual feature value of each pixel in the actual bird's-eye view, performs outlier recognition processing based on the open space feature value and the actual feature value of each pixel in the open space bird's-eye view, obtains a binary image containing outliers, performs denoising processing on the outliers in the binary image to obtain a first image, performs contour detection on the first image, and determines the contour of the obstacle in the actual scene. The detection method of the present application can, on the one hand, more accurately determine the size, shape features, etc. of the identified obstacle, and on the other hand, remove the influence of illumination on image feature recognition by denoising the outliers. The method can be applied to most obstacle feature recognition, avoiding the use of a large amount of algorithm training and testing, thereby reducing production costs and labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope of protection of the present invention. In each of the drawings, similar components are numbered similarly.

[0051] Figure 1 A schematic diagram of a flow chart of an obstacle contour detection method proposed in an embodiment of the present application is shown;

[0052] Figure 2 A schematic diagram of the process of generating a bird's-eye view of an open space in an obstacle contour detection method proposed in an embodiment of the present application is shown;

[0053] Figure 3 A schematic diagram of a process for calculating characteristic values ​​in an obstacle contour detection method proposed in an embodiment of the present application is shown;

[0054] Figure 4 A schematic diagram of the process of outlier identification processing in an obstacle contour detection method proposed in an embodiment of the present application is shown;

[0055] Figure 5 A schematic diagram of the process of corrosion processing in an obstacle contour detection method proposed in an embodiment of the present application is shown;

[0056] Figure 6 A schematic diagram of the process of determining an obstacle contour in an obstacle contour detection method proposed in an embodiment of the present application is shown;

[0057] Figure 7 A schematic diagram of an obstacle in an obstacle contour detection method proposed in an embodiment of the present application is shown;

[0058] Figure 8 A schematic structural diagram of an obstacle contour judgment device provided in an embodiment of the present application is shown.

[0059] Description of main component symbols:

[0060] 10-Obstacle contour judgment device; 11-Bird's-eye view acquisition module; 12-Eigenvalue calculation module; 13-Outlier identification module; 14-Noise removal processing module; 15-Contour determination module. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0062] The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the figures is not intended to limit the scope of the claimed invention, but rather merely represents selected embodiments of the present invention. All other embodiments derived by those skilled in the art based on the embodiments of the present invention without inventive effort are intended to be within the scope of protection of the present invention.

[0063] Hereinafter, the terms "including", "having" and their cognates, which may be used in various embodiments of the present invention, are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or the possibility of adding one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items.

[0064] Furthermore, the terms “first,” “second,” “third,” etc., are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.

[0065] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by those skilled in the art to which the various embodiments of the present invention pertain. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meaning as in the context of the relevant technical field and will not be interpreted as having an idealized meaning or an overly formal meaning unless clearly defined in the various embodiments of the present invention.

[0066] Example 1

[0067] One embodiment of the present application, such as Figure 1 As shown, the present application provides a monocular camera obstacle contour detection method, including steps S110 to S150:

[0068] S110: Acquire an actual image of an actual scene through a monocular camera, and perform perspective transformation on the actual image to obtain an actual bird's-eye view.

[0069] In this embodiment, the object is re-projected onto a new imaging plane by performing a transformation according to the object imaging projection law. In other words, a perspective transformation is performed on the photo obtained from the perspective of the monocular camera to obtain a top-down plan view of the open space scene, i.e., a bird's-eye view, so that the target features can be detected based on a unified benchmark, thereby realizing the detection of obstacles.

[0070] Among them, such as Figure 2 As shown, before step S100, steps S210 to S250 are also included:

[0071] S210: Acquire an open space image of an open space scene through a monocular camera, and measure an actual size of the open space scene to determine an aspect ratio of the open space scene.

[0072] In this embodiment, an open space image of the open space scene is acquired by a monocular camera, and the actual size of the open space scene is measured, and the aspect ratio of the actual site is determined by measuring the actual size of the open space scene.

[0073] S220: Determine the coordinates of the open space based on the aspect ratio and the resolution of the bird's-eye view of the open space scene to be generated.

[0074] Based on the aspect ratio of the open space scene and the maximum resolution of the bird's-eye view of the open space to be generated, the length and width of the open space scene at the maximum resolution can be calculated, and thus the coordinates of the open space corresponding to the open space scene can be determined.

[0075] Exemplarily, if the measured actual aspect ratio of the open space scene is m:n, and the length of the maximum resolution of the bird's-eye view of the open space to be generated is L and the width is W, the length L' and width W' of the open space at the maximum resolution can be calculated. The coordinates of the open space obtained after perspective transformation of the open space scene can be determined through the actual aspect ratio m:n and the length L and width W of the open space image. The above coordinates of the open space include: (0, 0), (0, L'), (W', 0), (W', L'). Among them, when W*m / n < L, the length L' and width W' are respectively L' = W*m / n and W' = W; otherwise, the length L' and width W' are respectively L' = L and W' = L*n / m.

[0076] In this embodiment, the cost of obtaining images through a monocular camera is lower, and in actual use, a large number of scenes are not required for algorithm training and testing, which can reduce the workload of staff.

[0077] S230: Set corner points at the boundaries of the open space scene and determine the coordinates of the corresponding corner points in the open space image.

[0078] It can be understood that the corner points are extreme points, that is, points with particularly prominent attributes in some aspects of the image. In this embodiment, multiple corner points can be set at the boundaries of the open space scene, or the intersection of two lines can be used as a corner point, or points located on two adjacent things with different main directions can be used as corner points. The coordinates of the corresponding corner points in the open space image can be determined through a preset detection algorithm. Among them, the above detection algorithms include but are not limited to detection algorithms such as the Harris corner point algorithm and the Shi-Tomas corner point algorithm.

[0079] Exemplarily, at least four corner points can be set at the boundaries of the open space scene. For example, if a solid black block with a size of 2cm * 2cm is placed at the boundary, then the solid black block can be a corner point in the image. The coordinates of the four corner points automatically identified through the Harris corner point algorithm are respectively (x1, y1), (x2, y1), (x1, y2), (x2, y2), where x represents the length direction and y represents the width direction.

[0080] S240: Calculate the parameters of the perspective transformation matrix based on the corner point coordinates and the open space coordinates to determine the perspective transformation matrix.

[0081] Perspective transformation is the process of projecting a two-dimensional image onto a three-dimensional viewing plane and then converting it into two-dimensional coordinates, which is also called projection mapping.

[0082] Specifically, the perspective transformation matrix is ​​as follows:

[0083]

[0084] Among them, X, Y and Z represent the three-dimensional coordinates after perspective transformation, and x and y represent the two-dimensional coordinates before perspective transformation. The above perspective transformation matrix formula can be obtained:

[0085]

[0086] Because (X, Y, Z) is a three-dimensional coordinate, it is necessary to convert the obtained three-dimensional coordinate into a two-dimensional coordinate (x', y', 1), thus obtaining the following formula:

[0087]

[0088] Therefore, x' and y' are the final calculation results of the two-dimensional perspective transformation, where c3 = 1. Substitute the above-mentioned empty field coordinates and corner coordinates as the coordinate points before and after the perspective transformation into the above formula, that is, (0, 0), (0, L'), (W', 0), (W', L') and (x1, y1), (x2, y1), (x1, y2), (x2, y2) as four groups of (x, y), (x', y') into the above formula to calculate the parameters a1, a2, a3, b1, b2, b3 and c1 of the perspective transformation matrix.

[0089] S250: Performing perspective transformation on the open space image using the perspective transformation matrix to obtain an open space bird's-eye view of the open space scene.

[0090] Using the perspective transformation formula above, any point (x, y) in the image captured by the monocular camera can be substituted to calculate the coordinates of the point (x', y') after perspective transformation. In other words, the captured open space image is transformed using the perspective transformation matrix above to obtain a bird's-eye view of the open space scene.

[0091] By transforming the object according to the object's imaging projection rules, the object is reprojected onto a new imaging plane. In other words, a perspective transformation is performed on the photo obtained from the monocular camera's perspective to obtain a top-down plan view of the open space scene, i.e., a bird's-eye view, so that target features can be detected based on a unified benchmark, i.e., obstacle detection.

[0092] For the above-mentioned step S110, exemplarily, after obtaining the open space image of the open space scene, the installation position of the monocular camera is kept unchanged, the actual image of the actual scene is obtained through the monocular camera, and the actual image is perspective transformed through the above-mentioned perspective transformation matrix to obtain a top-down plan view of the actual scene, that is, an actual bird's-eye view.

[0093] S120: Calculate the actual feature value of each pixel in the actual bird's-eye view.

[0094] It can be understood that after obtaining the actual bird's-eye view, the actual feature value of each pixel in the actual bird's-eye view corresponding to the actual scene is determined by calculation. The above-mentioned actual feature value and the open space feature value of each pixel in the open space bird's-eye view corresponding to the open space scene can be calculated using the same calculation method. Figure 3 As shown in the figure, the feature values ​​of each pixel in the corresponding bird's-eye view images of the actual scene and the open space scene can be calculated by the following sub-steps:

[0095] Sub-step S121: performing grayscale processing on the corresponding bird's-eye view image to obtain a grayscale image.

[0096] It can be understood that after perspective transformation, the bird's-eye view corresponding to the open space scene and the actual scene is obtained, and the bird's-eye view is converted into a grayscale image through grayscale processing, wherein the grayscale value corresponding to each pixel in the grayscale image ranges from 0 to 255. The methods for grayscale processing of the corresponding bird's-eye view include: component method, maximum value method, average value method and weighted average method.

[0097] Sub-step S122: Calculating corresponding central brightness and ambient brightness based on each pixel point of the grayscale image as the center.

[0098] After obtaining the grayscale image, take each pixel in the grayscale image as the center and take the first range of pixels surrounding the pixel as the core area of ​​the pixel. The average brightness of the core area of ​​each pixel is calculated as the central brightness of the pixel. That is, with the pixel as the core, take the average grayscale value of all pixels in the core area. Take the second range of pixels surrounding the pixel as the ambient area of ​​the pixel, and calculate the average brightness of the ambient area of ​​each pixel as the ambient brightness of the pixel. That is, calculate the average grayscale value of all the grayscale values ​​in the ambient area around each pixel. The pixels in the first range are smaller than those in the second range.

[0099] For example, when pixel (30, 30) is the core point and the range of the core area is 3*3, the central brightness of the pixel is the average of the grayscale values ​​of the nine pixels (29, 29), (29, 30), (29, 31), (30, 29), (30, 30), (30, 31), (31, 29), (31, 30), and (31, 31). When the range of the ambient area is 9*9, the average of the grayscale values ​​of the 81 pixels in the surrounding area is calculated as the ambient brightness of the pixel.

[0100] Sub-step S123: taking the ratio of the central brightness of each pixel to the ambient brightness as the characteristic value of the corresponding pixel.

[0101] After obtaining the central brightness and ambient brightness of each pixel, the ratio of the central brightness to the ambient brightness for each pixel is calculated as the eigenvalue of each pixel. The number of eigenvalues ​​in a grayscale image is equal to the number of pixels in the image. After calculating the eigenvalue for each pixel in the bird's-eye view of the open space scene, the eigenvalue for each pixel can be pre-stored in memory. By comparing the central brightness of each pixel with the ambient brightness, interference from changes in the lighting environment on obstacle recognition can be eliminated.

[0102] S130: performing outlier identification processing based on the vacant land feature value of each pixel point in the vacant land bird's-eye view image and the actual feature value to obtain a binary image containing the outlier.

[0103] It can be understood that when there are no obstacles in the actual scene, even if the overall light brightness changes, the brightness ratio of the core area of ​​each pixel and the surrounding environment area will not change significantly. When there are obstacles in the actual scene, the characteristics of the core area and the environment area will change, and the brightness ratio of each pixel in the obstacle and the surrounding environment will change significantly. By comparing the vacant land feature values ​​of each pixel in the vacant land bird's-eye view with the actual feature values ​​of each pixel in the actual bird's-eye view, the abnormal points in the actual bird's-eye view can be determined, that is, the pixel points when there are obstacles on the site. Figure 4 As shown, step S130 includes the following sub-steps:

[0104] Sub-step S131: Compare the clearing feature value and the actual feature value of the pixel point at the same position to obtain a corresponding ratio.

[0105] The number of pixels in the bird's-eye view images corresponding to the open space scene and the actual scene is the same, and the positions of each pixel in the open space bird's-eye view correspond one-to-one with each pixel in the actual bird's-eye view. By comparing the actual eigenvalue corresponding to each pixel in the actual bird's-eye view with the corresponding open space eigenvalue in the open space bird's-eye view, we can obtain the ratio of the eigenvalues ​​corresponding to each pixel.

[0106] Sub-step S132: Determine whether the ratio is less than or equal to a preset ratio threshold.

[0107] According to the ratio threshold pre-set according to the actual situation, determine whether the ratio corresponding to the above-mentioned each pixel point is less than or equal to the preset ratio threshold. If the ratio calculated by the pixel point is less than or equal to the preset ratio threshold, execute sub-step S133, otherwise execute sub-step S134.

[0108] Sub-step S133: Determine that the pixel point is the abnormal point.

[0109] Sub-step S134: Determine whether the pixel point is a normal point.

[0110] If the calculated ratio of a pixel is greater than the preset ratio threshold, the pixel is considered a match and is considered a normal point. If the pixel's feature value is less than or equal to the preset ratio threshold, the pixel is determined to be mismatched and marked as an outlier. For example, when the preset ratio threshold is 80%, the ratio of the actual feature value of a pixel to the feature value of the open space is calculated to be 70%. At this time, the ratio is less than the ratio threshold, and the pixel is determined to be an outlier.

[0111] After determining whether each pixel in the actual bird's-eye view image is an outlier, that is, marking each outlier in the image, a binary image containing the outliers can be obtained. For example, the marking result for each pixel can be represented by 0 and 1. The actual bird's-eye view image is marked to obtain a binary image containing the outliers, and this binary image can be stored. Here, 1 represents an outlier and 0 represents a normal point. In this case, black represents 0 and white represents 1.

[0112] S140: Performing denoising on abnormal points in the binary image to obtain a first image.

[0113] It can be understood that after obtaining the binary image, the judgment result corresponding to each pixel is a normal point or an abnormal point. When there are obstacles in the actual scene, significant abnormal points can be identified, that is, pixel points where the central brightness and ambient brightness change significantly. However, because the binary image itself contains noise, a large number of discrete abnormal points will appear. These abnormal points need to be further screened to determine whether they are pixel points corresponding to obstacles. In this embodiment, the pixel points marked as abnormal points in the above binary image will be denoised, that is, expansion and corrosion will be performed to obtain the first image. Among them, the abnormal points in the above binary image are denoised in the following way:

[0114] First, the binary image is dilated by a first pixel size to obtain a second image, which can expand the connected area of ​​the image and prevent noise from drowning out actual obstacles. Then, the second image is eroded by a second pixel size to obtain a third image, which can clear some pixels in the second image and remove some interference noise. Then, the third image is dilated by a third pixel size to obtain a fourth image, which can further expand the connected area in the third image. Finally, the fourth image is eroded by a fourth pixel size to obtain the first image, which can restore the feature points of the binary image and avoid changing the size of the feature points due to erosion and dilation. The relationship between the above pixel sizes is as follows: the sum of the first pixel size and the third pixel size is equal to the sum of the second pixel size and the fourth pixel size, and the first pixel size is smaller than the second pixel size and smaller than the third pixel size.

[0115] Exemplarily, when the first pixel size is n1, the second pixel size is m1, the third pixel size is n2, and the fourth pixel size is m2, m1>n1, n2>m1, m2=n1+n2-m1, that is, n1+n2=m2+m2.

[0116] In this embodiment, the sizes of the dilation process and the erosion process are the same. Through the above four-step denoising process, not only can the noise interference be removed, but the size characteristics of the binary image can also be retained.

[0117] Among them, such as Figure 5 As shown, the image corrosion process includes the following sub-steps:

[0118] Sub-step S141: sequentially aligning the center point of a preset structural element with each abnormal point in the binary image.

[0119] In this embodiment, a template matrix for the corrosion operation is first defined. In other words, a structural element for the corrosion process is pre-set as a first structural element, that is, a structure for controlling the operation. The center point of the first structural element is sequentially placed in each pixel point marked as an abnormal point in the above binary image.

[0120] Sub-step S142: Determine whether all pixels in the structural element are abnormal points.

[0121] If all pixels are outliers, the pixel corresponding to the center point is retained. Otherwise, step S143 is executed.

[0122] Sub-step S143: If not all the pixels in the structural element are abnormal points, the pixel corresponding to the center point is modified to a normal point.

[0123] It is determined whether all pixels included in the first structuring element are outliers. If all image pixels covered by the first structuring element are outliers, the pixel corresponding to the center point of the first structuring element is retained. Otherwise, the pixel corresponding to the center point of the first structuring element is deleted, that is, the pixel corresponding to the center point is changed from an outlier to a normal point. By sequentially aligning the center point of the first structuring element with each pixel of the binary image to determine whether there are pixels that need to be deleted, the main area of ​​the image can be reduced, thereby removing small connected domains caused by noise.

[0124] For example, the determination result for each pixel is represented by 0 and 1, with 0 representing a normal pixel and 1 representing an abnormal pixel. The first structuring element can be defined as 3*3 pixels in size and rectangular in shape. The center point of this 3*3 pixel first structuring element is then placed sequentially at each pixel in the binary image where the element is 1. If the result for all pixels covered by the first structuring element is 1, the pixel corresponding to the center point of the first structuring element is retained, i.e., the determination result for this pixel is kept marked as 1. Otherwise, the determination result for the pixel corresponding to the center point is changed from 1 to 0.

[0125] Image dilation processing includes the following steps:

[0126] First, define the template matrix for the dilation operation. In other words, pre-set the structuring element used for the dilation process as the second structuring element, which is the structure used to control the operation. For example, the second structuring element can be defined as 3*3 pixels in size and rectangular in shape. Align the center point of the second structuring element with each pixel marked as an outlier in the binary image. Determine whether all pixels covered by the second structuring element are outliers. If any are normal, modify the normal pixels to be outliers, thus completing the dilation operation. This dilation process expands the area of ​​each region, thereby filling the holes caused by noise.

[0127] S150: Perform contour detection on the first image to determine the contours of obstacles in the actual scene.

[0128] After obtaining a first image including multiple connected regions, the shape features of each connected region can be identified through contour detection processing. The image contour refers to the boundary of the image, that is, the external features of the target image. Figure 6 As shown, step S150 includes the following sub-steps:

[0129] Sub-step S151: extracting at least one contour in the first image based on a preset algorithm.

[0130] In this embodiment, the image contour is obtained from the first image after the denoising process by a preset algorithm, wherein the preset algorithm may be a Satoshi Suzuki algorithm. Each image may have multiple contours, and each contour is composed of multiple points.

[0131] Sub-step S152: Calculate the center point of each of the contours and the side length and area of ​​the minimum oblique rectangle.

[0132] It is understandable that by determining the center point and the minimum oblique rectangle corresponding to each of the above contours, the side length and area of ​​the minimum oblique rectangle corresponding to each contour can be calculated using Python, OpenCV, etc. Figure 7 As shown in the figure, the gray part is a suspected obstacle. First, determine the minimum outline size of the obstacle. In other words, first determine the minimum resolution of the obstacle, such as Figure 7 The box shows the effect when four obstacles are recognized. The center point of each outline is the white dot in the box. The center point coordinates and pixel size of each outline can be converted to actual size according to the ratio.

[0133] Sub-step S153: Determine whether the center point of the current contour is at the boundary of the first image.

[0134] Determine whether the center point of each contour is at the boundary of the first image. If the center point of the current contour is not at the boundary of the first image, execute sub-step S154. Otherwise, execute sub-step S156 and retain the current contour as a questionable contour for merging with the boundary image obtained by the adjacent monocular camera.

[0135] S154: Detect whether the side length is smaller than a preset side length threshold, and whether the area is smaller than a preset area threshold.

[0136] If the side length is smaller than the side length threshold and the area is smaller than the area threshold, sub-step S155 is executed; otherwise, sub-step S156 is executed.

[0137] Sub-step S155: Determine that the current contour is an interference contour and discard it.

[0138] Sub-step S156: retaining the current outline.

[0139] If the side length of the current contour is less than the preset side length threshold and the area is less than the preset area threshold, the current contour is determined to be an interference contour and will be discarded. Otherwise, the contour is determined to be an obstacle contour because it can more accurately reflect the characteristics of the obstacle.

[0140] Exemplarily, the preset side length threshold size is 5 pixels, and the preset area threshold size is 30 pixels. If the side lengths of the current contour are less than 5 pixels and the area is less than 30 pixels, the current contour is considered to be an interference contour.

[0141] Sub-step S157: taking the retained outline as the outline of the obstacle.

[0142] The above-mentioned retained contours are outputted, and the outputted contours can more accurately reflect the characteristics of the obstacle, that is, the above-mentioned retained contours can be used as the contours of the obstacle.

[0143] In this embodiment, the influence of the installation angle of the monocular camera can be eliminated through perspective transformation processing, and calculations are performed on a unified top-down plan view, which can more accurately determine the size, appearance characteristics, etc. of the identified obstacles, and can also remove the influence of different lighting on the recognition of image features. It can be applied to most obstacle feature recognition, avoiding the use of a large amount of pre-algorithm training and testing, thereby reducing production costs and labor costs.

[0144] Based on the obstacle contour detection method of the above embodiment, Figure 8 A schematic structural diagram of an obstacle contour determination device 10 provided in an embodiment of the present application is shown.

[0145] The obstacle contour judgment device 10 includes:

[0146] The bird's-eye view acquisition module 11 acquires an actual image of the actual scene through a monocular camera, and performs perspective transformation on the actual image to obtain an actual bird's-eye view;

[0147] The eigenvalue calculation module 12 calculates the actual eigenvalue of each pixel in the actual bird's-eye view;

[0148] An outlier recognition module 13 performs outlier recognition processing based on the vacant land feature value of each pixel point in the vacant land bird's-eye view image and the actual feature value to obtain a binary image containing the outlier point;

[0149] a denoising module 14 for performing denoising on abnormal points in the binary image to obtain a first image;

[0150] The contour determination module 15 performs contour detection on the first image to determine the contours of obstacles in the actual scene.

[0151] This embodiment provides an obstacle contour judgment device 10, which is used to execute the obstacle contour detection method described in the above embodiment through the coordinated use of a bird's-eye view image acquisition module 11, a feature value calculation module 12, an outlier identification module 13, a denoising processing module 14 and a contour determination module 15. The implementation scheme and beneficial effects involved in the above embodiment are also applicable to this embodiment and will not be repeated here.

[0152] In addition, the present application also proposes a terminal device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program runs on the processor, it executes the obstacle contour detection method described in the above embodiment.

[0153] This embodiment further provides a computer storage medium for storing the computer program used in the above-mentioned terminal device.

[0154] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or flowchart, and the combination of boxes in the structure diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0155] In addition, the functional modules or units in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0156] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0157] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be covered by the scope of protection of the present invention.

Claims

1. A method for detecting obstacle contours, characterized in that: include: Acquire an actual image of an actual scene through a monocular camera, and perform perspective transformation on the actual image to obtain an actual bird's-eye view; Calculate the actual feature value of each pixel in the actual bird's-eye view; Performing outlier recognition processing based on the vacant land feature value of each pixel point in the vacant land bird's-eye view image and the actual feature value to obtain a binary image containing the outlier point; Performing denoising on abnormal points in the binary image to obtain a first image; Performing contour detection on the first image to determine the contours of obstacles in the actual scene; The characteristic value of each pixel in the actual bird's-eye view is calculated by: Performing grayscale processing on the actual bird's-eye view image to obtain a grayscale image; Calculate the corresponding central brightness and ambient brightness based on each pixel point of the grayscale image as the center; The ratio of the central brightness of each pixel to the ambient brightness is used as the actual characteristic value of the corresponding pixel.

2. The obstacle contour detection method according to claim 1, characterized in that: The performing contour detection on the first image to determine the contour of the obstacle in the actual scene includes: extracting at least one contour in the first image based on a preset algorithm; Calculate the center point of each of the contours and the side length and area of ​​the minimum oblique rectangle; If the center point of the current contour is not at the boundary of the first image, detecting whether the side length is less than a preset side length threshold, and whether the area is less than a preset area threshold; If the side length is less than the side length threshold and the area is less than the area threshold, the current contour is determined to be an interference contour and discarded; otherwise, the current contour is retained; If the center point of the current contour is at the boundary of the first image, retaining the current contour; The retained contour is used as the obstacle contour.

3. The obstacle contour detection method according to claim 1 or 2, characterized in that: The method of obtaining an actual image of an actual scene by a monocular camera includes: Acquire an open space image of an open space scene using a monocular camera, and measure the actual size of the open space scene to determine the aspect ratio of the open space scene; Determining the open space coordinates based on the aspect ratio and the resolution of the open space bird's-eye view of the open space scene to be generated; Setting corner points at the boundaries of the open space scene, and determining the corresponding corner point coordinates of the corner points in the open space image; Calculating parameters of a perspective transformation matrix based on the corner point coordinates and the open space coordinates to determine the perspective transformation matrix; The open space image is perspective transformed using the perspective transformation matrix to obtain an open space bird's-eye view of the open space scene.

4. The obstacle contour detection method according to claim 1, characterized in that: The outlier identification process based on the vacant land feature value of each pixel point in the vacant land bird's-eye view image and the actual feature value includes: Compare the clearing feature value and the actual feature value of the pixel point at the same position to obtain a corresponding ratio; If the ratio is less than or equal to a preset ratio threshold, the pixel point is determined to be the abnormal point; otherwise, the pixel point is determined to be a normal point.

5. The obstacle contour detection method according to claim 1, characterized in that: The performing denoising on the abnormal points in the binary image to obtain the first image includes: performing dilation processing on the binary image at a first pixel size to obtain a second image; performing an erosion process of a second pixel size on the second image to obtain a third image; performing dilation processing on the third image at a third pixel size to obtain a fourth image; performing an erosion process of a fourth pixel size on the fourth image to obtain a first image; Among them, the relationship between the first pixel size, the second pixel size, the third pixel size and the fourth pixel size is as follows: the sum of the first pixel size and the third pixel size is equal to the sum of the second pixel size and the fourth pixel size, and the first pixel size is smaller than the second pixel size, which is smaller than the third pixel size.

6. The obstacle contour detection method according to claim 5, characterized in that: The corrosion treatment includes: Aligning the center point of a preset structural element with each abnormal point in the binary image in sequence, and determining whether all pixels in the structural element are abnormal points; If all the pixels are abnormal points, the pixel corresponding to the center point is retained; Otherwise, the pixel point corresponding to the center point is modified to a normal point.

7. An obstacle contour judgment device, characterized in that: include: A bird's-eye view acquisition module acquires an actual image of an actual scene through a monocular camera and performs perspective transformation on the actual image to obtain an actual bird's-eye view; An eigenvalue calculation module calculates the actual eigenvalue of each pixel in the actual bird's-eye view; An outlier recognition module performs outlier recognition processing based on the vacant land feature value of each pixel point in the vacant land bird's-eye view image and the actual feature value to obtain a binary image containing the outlier point; a denoising processing module, performing denoising on abnormal points in the binary image to obtain a first image; a contour determination module, performing contour detection on the first image to determine the contours of obstacles in the actual scene; Wherein, the eigenvalue calculation module is further used to perform grayscale processing on the actual bird's-eye view image to obtain a grayscale image; Calculate the corresponding central brightness and ambient brightness based on each pixel point of the grayscale image as the center; The ratio of the central brightness of each pixel to the ambient brightness is used as the actual characteristic value of the corresponding pixel.

8. A terminal device, characterized in that: The system comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is run on the processor, the obstacle contour detection method according to any one of claims 1 to 6 is executed.

9. A readable storage medium, characterized in that: The computer program is stored therein, and when the computer program is run on a processor, the obstacle contour detection method according to any one of claims 1 to 6 is executed.

Images