An obstacle recognition method for self-propelled devices and the self-propelled device itself.

Abstract

The present application relates to an obstacle recognition method and device for a self-propelled device, which converts the collected image in front of the self-propelled device from RGB space to HSV space, extracts the chromaticity component and the brightness component; performs edge extraction on the brightness component after preprocessing, obtains the chromaticity component histogram, and counts the maximum value maxH of the number of pixels within a preset chromaticity interval; performs image segmentation on the collected image in front of the self-propelled device to obtain a segmented image; performs contour detection on the segmented image to determine the obstacle recognition parameters of a single contour image; and determines whether there is an obstacle in front of the self-propelled device in combination with the pre-set obstacle recognition judgment conditions. The method solves the problem that the anti-collision method for self-propelled devices in the prior art requires the construction of a boundary system, which is time-consuming, labor-intensive, and costly.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and in particular, relates to an obstacle recognition method applied to a self-propelled device and the self-propelled device. Background Art

[0002] With the gradual acceleration of urban greening and the rapid development of the lawn industry, urban lawns, golf courses, airports, as well as private gardens, parks, playgrounds and other places need to be trimmed from time to time to ensure beauty, and annual trimming consumes a lot of time, money and manpower.

[0003] At present, self-propelled equipment is usually used to replace manual trimming. Unmanned lawn mowers can free labor from highly repetitive and boring lawn mowing operations, which has become an inevitable trend of social development. The cutting blades of unmanned lawn mowers are sharp, and their safety is uncontrollable. When unmanned, they may encounter stationary or moving obstacles, which may damage the blades or the body of the vehicle, and accidents are likely to occur, causing unnecessary losses.

[0004] There are three traditional methods for self-propelled equipment to avoid collisions:

[0005] The first method is to manually wire around obstacles before mowing. Once the mower detects an electromagnetic signal during its movement, it changes direction. However, the wires are prone to aging and require regular inspection and maintenance.

[0006] The second method is to use a combination of multiple sensors, such as ultrasonic and infrared sensors, which is very expensive and easily affected by the environment.

[0007] The third method is GPS positioning, which locates the coordinates of obstacles in advance. The signal of this method is easily affected, and the internal components are prone to problems.

[0008] The above methods are prone to missed detection, and existing anti-collision methods all require the construction of a boundary system, which is time-consuming, labor-intensive and costly. Summary of the invention

[0009] The present invention aims to solve the problem that the anti-collision method of a self-propelled device in the prior art needs to build a boundary system, which is time-consuming, labor-intensive and costly, and provides an obstacle recognition method and a self-propelled device applied to the self-propelled device.

[0010] A first aspect of an embodiment of the present invention provides an obstacle recognition method applied to a self-propelled device, comprising:

[0011] Acquire an image of the environment of the self-propelled device in the direction of travel;

[0012] Separating a chromaticity channel image and a luminance channel image according to the image;

[0013] Preprocessing and edge processing the brightness channel image to obtain an edge image;

[0014] Perform histogram statistics on the chromaticity channel image to obtain the number of pixels with the largest color share within a preset chromaticity interval, recorded as maxH;

[0015] Performing segmentation processing and contour processing on the chromaticity channel image to obtain a chromaticity segmentation threshold and a contour image;

[0016] Performing contour detection on the contour image to obtain a contour block;

[0017] Counting the feature values ​​corresponding to the contour block in the edge image and the contour image;

[0018] The maxH, the characteristic value and the preset obstacle recognition determination condition are compared to obtain a recognition result.

[0019] The obstacle avoidance method in this scheme separates the images of the chromaticity channel and the luminance channel, processes the luminance channel and the chromaticity channel separately, obtains the contour block through contour detection, and effectively identifies the non-lawn area based on the statistical feature values ​​of the contour block and the pre-set obstacle recognition and judgment conditions, thereby minimizing missed judgments.

[0020] A second aspect of an embodiment of the present invention provides a self-propelled device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the obstacle recognition method applied to a self-propelled device described in the first aspect of the embodiment of the present invention are implemented.

[0021] The self-propelled device in this scheme separates the chromaticity channel and the brightness channel of the collected image of the environment in front of the walking direction, processes the brightness channel and the chromaticity channel respectively, obtains the contour block through contour detection, and can effectively identify the non-lawn area based on the statistical feature values ​​of the contour block and the pre-set obstacle recognition and judgment conditions, thereby minimizing missed judgments.

[0022] A third aspect of an embodiment of the present invention provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the obstacle recognition method applied to a self-propelled device described in the first aspect of the embodiment of the present invention are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The technical solution of the present application is further described below in conjunction with the accompanying drawings and embodiments.

[0024] Figure 1 is a method flow chart of an embodiment of the present application;

[0025] Figure 2(a) to Figure 2(c) The RGB image, the image after segmentation processing, and the image after contour processing in front of the self-propelled device corresponding to a specific implementation method;

[0026] Figure 3(a) to Figure 3(c) RGB image, segmented image, and contour processed image in front of the self-propelled device corresponding to another specific implementation manner. DETAILED DESCRIPTION

[0027] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application may be combined with each other.

[0028] The technical solution of the present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0029] Example 1

[0030] Embodiment 1 of the present invention provides an obstacle recognition method for a self-propelled device, such as Figure 1 As shown, including:

[0031] S1: Acquire an image of the environment of the self-propelled device in its travel direction;

[0032] S2: Separating a chromaticity channel image and a luminance channel image according to the image;

[0033] S3: Preprocessing and edge processing the brightness channel image to obtain an edge image;

[0034] S4: Performing histogram statistics on the chromaticity channel image to obtain the number of pixels with the largest color proportion within a preset chromaticity interval, recorded as maxH;

[0035] S5: performing segmentation processing and contour processing on the chromaticity channel image to obtain a chromaticity segmentation threshold and a contour image;

[0036] S6: Performing contour detection on the contour image to obtain a contour block;

[0037] S7: counting the feature values ​​corresponding to the contour block in the edge image and the contour image;

[0038] S8: Compare the maxH, the characteristic value and the preset obstacle recognition judgment condition to obtain the recognition result.

[0039] The self-propelled device in the embodiment of the present invention may be a sweeping robot, a lawn mowing robot, etc. The obstacle recognition method in the embodiment of the present invention uses image recognition to enable the self-propelled device to effectively identify non-lawn areas, identify the distance information of obstacles ahead, and take reasonable obstacle avoidance measures during operation.

[0040] In this embodiment, a camera is arranged in front of the self-propelled device. During the operation of the self-propelled device, the camera collects images of the environment in front of the device in real time, which are recorded as orgMat. The collected image orgMat is an RGB image. The acquired image orgMat is converted from the RGB (red, green, blue) space to the HSV (hue, saturation, brightness) space, and the chromaticity component (H) and the brightness component (V) are extracted. The conversion of images from the RGB space to the HSV space is a well-known technology for those skilled in the art and will not be repeated here.

[0041] For the brightness channel (V) image, preprocessing is first performed, and the preprocessing includes filtering and normalization processing, so as to obtain a preprocessed image, which is recorded as normVMat. The edge extraction of the preprocessed image normVMat can be performed using, but not limited to, the canny operator edge detection algorithm, so as to obtain an edge image, which is recorded as cannyMat. The roughness of the image can be well reflected by edge extraction.

[0042] Perform histogram statistics on the chromaticity channel (H) image to obtain a chromaticity component histogram, denoted as orgLabelsMat, where the horizontal axis of the chromaticity component histogram represents the chromaticity component of the image in front of the self-propelled device, and the vertical axis represents the number of pixels. Perform filtering on the obtained chromaticity component histogram orgLabelsMat to obtain a denoised and smoothed histogram LabelsMat. For the denoised and smoothed histogram LabelsMat, count the number of pixels with the largest color proportion within the preset chromaticity interval, denoted as maxH. In this embodiment, the preset chromaticity interval range can be 15 to 180.

[0043] For the chroma channel (H) image, the dynamic segmentation method can be used to perform image segmentation processing to obtain the chroma segmentation threshold [lowValue, highValue]. Of course, the chroma segmentation threshold [lowValue, highValue] can also be obtained by a fixed threshold segmentation method, such as the Otsu threshold method.

[0044] The segmented image obtained after the image segmentation processing is recorded as dstMat, and the dstMat is inverted and opened and closed, and the obtained image is recorded as obstacleMat.

[0045] Perform contour processing on the image obstacleMat to obtain a contour image, and perform contour detection on the contour image to obtain a contour block.

[0046] The feature values ​​corresponding to the contour blocks in the edge image and the contour image are counted, and the features, maxH and the preset obstacle recognition judgment conditions are compared and processed to determine whether there is a non-lawn area in the environmental image in front of the self-propelled device, that is, whether there is an obstacle in front of the self-propelled device.

[0047] In a feasible implementation of this embodiment, the feature value includes a contour size feature value AContours i .X, average roughness value HContours i BContours is the percentage of black pixels in the contour area i ;or,

[0048] The eigenvalues ​​include contour size eigenvalues ​​AContours i .X, average roughness value HContours i , the percentage of pixels within the chromaticity segmentation threshold range SContours i BContours is the percentage of black pixels in the contour area i . Where i is the number of the contour block.

[0049] Specifically, for the contour size eigenvalue AContours i .X, where X can be expressed as area, diagonal, width, height, number of pixels, etc., and i represents the contour number.

[0050] In this embodiment, the image is segmented by a chromaticity segmentation threshold [lowValue, highValue] to generate a first image region and a second image region. The pixel proportion SContours within the range of the chromaticity segmentation threshold [lowValue, highValue] in this embodiment is i , refers to the pixel ratio of the first image area to the second image area.

[0051] Among them, the chromaticity value corresponding to the first image region is within the range of a preset chromaticity interval and within the range of the segmentation threshold [lowValue, highValue]; the chromaticity value corresponding to the second image region is within the range of the preset chromaticity interval and the chromaticity value is not within the range of the segmentation threshold [lowValue, highValue]. For example, if the preset chromaticity interval is [15, 95] and the chromaticity segmentation threshold is [15, li], then the first image region is the region with chromaticity values in [15, li], and the second image region is the region with chromaticity values in [li, 95].

[0052] In a feasible implementation manner of this embodiment, the preset obstacle recognition determination conditions include multiple different preset obstacle recognition determination conditions. The comparing the maxH, the eigenvalue with the preset obstacle recognition determination conditions to obtain a recognition result includes:

[0053] Compare the maxH, the eigenvalue with the preset obstacle recognition determination conditions. If the comparison results of the maxH and the eigenvalue meet one or more of the multiple different preset obstacle recognition determination conditions, then recognize that there is an obstacle region in the image; if the comparison results of the maxH and the eigenvalue do not meet any one of the multiple different preset obstacle recognition determination conditions, then recognize the image as an image to be screened.

[0054] Specifically, based on the selected eigenvalue above, the preset obstacle recognition determination conditions in this embodiment can be:

[0055] If AContours i .diagonal>105 or AContours i .height>80, and satisfy maxH>300 and HContours i <0.24, and at the same time satisfy one of the two conditions a and b, then it is determined that there is an obstacle in the image:

[0056] a. 300 < maxH ≤ 400 and AContours i .pixels>3500 and BContours i <0.01;

[0057] b. 400 < maxH < 550 and AContours i .pixels>4000 and BContours i <0.03;

[0058] Taking the work of a certain lawn mowing robot as an example, such as Figure 2(a) to Figure 2(c)As shown, Figure 2(a) is the RGB image collected in front of the self-propelled device, Figure 2(b) is the segmented image obtained after image segmentation, and Figure 2(c) is the contour image numbered 8. maxH = 467, and the eigenvalues ​​corresponding to the contour block numbered 8 are as follows:

[0059] AContours8.diagonal=92.135, AContours8.height=83, BContours8=0.012, AContours8.pixels=7060, HContours8=0.176.

[0060] It can be seen from the judgment that the above conditions meet AContours i .height>80, and maxH>300 and HContours i <0.24, and condition b is satisfied, and there is a non-lawn area in the image.

[0061] Specifically, the obstacle recognition determination condition preset in this embodiment may also be:

[0062] If AContours i .diagonal>105 or AContours i .height>80, and YContours i >75, and maxH>300 and HContours i <0.24, and one of the two conditions c and d is met at the same time, it is determined that the distance between the obstacle in the image and the self-propelled device is within the preset anti-collision range, and the self-propelled device performs obstacle avoidance measures:

[0063] c.maxH≥550 and SContours i <0.4 and BContours i <0.11;

[0064] d.maxH≥550 and SContours i >0.4 and BContours i <0.06.

[0065] The obstacle recognition determination condition preset in the above implementation manner is only a preferred embodiment listed in this embodiment, and in other implementation manners, it can be adjusted according to actual conditions.

[0066] In a feasible implementation of this embodiment, in S7, the counting of feature values ​​corresponding to the contour block in the edge image and the contour image includes:

[0067] S71: Obtain the position information of the contour block;

[0068] S72: Compare the position information of the contour block with a preset position threshold to obtain a target contour block;

[0069] S73: Statistically analyze the feature values corresponding to the target contour block in the edge image and the contour image.

[0070] As a feasible embodiment, further optionally, S7 further includes statistically analyzing the y-axis coordinate value YContoursi of the lower right corner of the contour of the contour block, and reasonably selecting an obstacle avoidance timing according to the magnitude of YContoursi. Wherein, YContoursi represents the positional relationship between the contour position in the image in front of the self-driving device and the position of the self-driving device.

[0071] Specifically, this embodiment can also set the distance parameter between the image contour position and the self-driving device, that is, the y-axis coordinate value YContours of the lower right corner of the contour i , and the positional relationship between the contour position obtained when performing image contour detection on the image in front of the self-driving device and the position of the self-driving device can be used as an auxiliary judgment factor, and the obstacle avoidance timing can be reasonably selected.

[0072] If an obstacle is detected in the image, and an obstacle image or a partial obstacle image is included in a certain image contour block, then determine the positional relationship between the corresponding contour block and the self-driving device, and thus take reasonable obstacle avoidance measures.

[0073] In the case of including the feature value YContours i , the obstacle recognition and determination conditions preset in this embodiment can be:

[0074] If AContours i .diagonal > 105 or AContours i .height > 80, and YContours i > 75, and satisfy maxH > 300 and HContours i < 0.24, and at the same time satisfy one of a, b or both satisfy one of c, d, then it is determined that the distance between the obstacle in the image and the self-driving device is within the preset anti-collision range, and the self-driving device executes an obstacle avoidance measure:

[0075] a. 300 < maxH ≤ 400 and AContours i .pixels > 3500 and BContours i < 0.01;

[0076] b. 400 < maxH < 550 and AContours i .pixels > 4000 and BContours i < 0.03;

[0077] c. maxH ≥ 550 and SContours i < 0.4 and BContours i < 0.11;

[0078] d. maxH ≥ 550 and SContours i > 0.4 and BContours i < 0.06.

[0079] If in the obstacle recognition parameters, the y-axis coordinate value YContoursi of the lower right corner of the contour is set, then after recognizing the obstacle based on the full map information, it is also necessary to determine whether the self-driving device performs obstacle avoidance measures according to the size of YContoursi.

[0080] In a specific implementation, as Figure 3(a) to Figure 3(c) shown, Fig. 3(a) is the RGB image in front of the self-driving device collected, Fig. 3(b) is the segmented image obtained after image segmentation, and Fig. 3(c) is the contour image numbered 18. maxH = 576, and the obstacle recognition parameters corresponding to the contour image numbered 18 are as follows:

[0081] AContours 18 .diagonal = 164.07, YContours 18 = 114, SContours 18 = 0.28, HContours 18 = 0.218, BContours 18 = 0.

[0082] Judging from this, the above conditions satisfy AContours 18 .height > 80, and satisfy maxH > 300 and HContours i < 0.24, and at the same time satisfy condition c. Therefore, it can be determined that there is a non-lawn area in the image. Also, because YContours 18 = 114, which satisfies YContours 18 > 75. Therefore, it can be determined that the obstacle is within the distance range that requires obstacle avoidance from the self-driving device, and obstacle avoidance measures need to be taken.

[0083] Example 2:

[0084] This embodiment provides a self-propelled device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the obstacle recognition method applied to the self-propelled device described in Embodiment 1 are implemented.

[0085] Specifically, the self-propelled device of this embodiment may be a sweeping robot, a lawn mowing robot, etc. During the walking process, the self-propelled device can effectively identify obstacles and take reasonable obstacle avoidance measures.

[0086] Embodiment 3:

[0087] This embodiment provides a readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the obstacle recognition method applied to a self-propelled device described in Embodiment 1 are implemented.

[0088] Based on the above ideal embodiments of this application, the relevant staff can make various changes and modifications without departing from the technical concept of this application through the above description. The technical scope of this application is not limited to the content in the specification, and its technical scope must be determined according to the scope of the claims.

[0089] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0090] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0091] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0092] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

Claims

1. An obstacle recognition method applied to a self-propelled device, wherein the self-propelled device is a lawn mowing robot, characterized in that: include: Acquire an image of the environment of the self-propelled device in the direction of travel; Separating a chromaticity channel image and a luminance channel image according to the image; Preprocessing and edge processing the brightness channel image to obtain an edge image; Perform histogram statistics on the chromaticity channel image to obtain the number of pixels with the largest color share within a preset chromaticity interval, recorded as maxH; Performing segmentation processing and contour processing on the chromaticity channel image to obtain a chromaticity segmentation threshold and a contour image; Performing contour detection on the contour image to obtain a contour block; Counting the feature values ​​corresponding to the contour block in the edge image and the contour image; Compare the maxH, the characteristic value and a preset obstacle recognition determination condition to obtain a recognition result; The characteristic values ​​include the contour size characteristic value, the average roughness value and the proportion of black pixels in the contour block; or, The characteristic values ​​include contour size characteristic value, average roughness value, pixel ratio within the chromaticity segmentation threshold range, and black pixel ratio within the contour block; The counting of the feature values ​​corresponding to the contour blocks in the edge image and the contour image comprises: Obtaining position information of the contour block; Compare the position information of the contour block with a preset position threshold to obtain a target contour block; The feature values ​​corresponding to the target contour block in the edge image and the contour image are counted.

2. The obstacle recognition method for a self-propelled device according to claim 1, characterized in that: The contour size characteristic values ​​include contour area, contour diagonal length, contour width, contour height or the number of pixels of the to-be-identified area included in the contour block.

3. The obstacle recognition method for a self-propelled device according to claim 1, characterized in that: The preset obstacle recognition determination condition includes a plurality of different preset obstacle recognition determination conditions, and the comparing and processing the maxH, the characteristic value and the preset obstacle recognition determination condition to obtain the recognition result includes: The maxH and the eigenvalue are compared with pre-set obstacle recognition and determination conditions. If the comparison result obtained shows that the maxH and the eigenvalue meet one or more of the multiple different pre-set obstacle recognition and determination conditions, it is identified that there is an obstacle area in the image; if the comparison result obtained shows that the maxH and the eigenvalue do not meet any of the multiple different pre-set obstacle recognition and determination conditions, the image is identified as an image to be screened.

4. The obstacle recognition method for a self-propelled device according to claim 1, characterized in that: The step of performing histogram statistics on the chromaticity channel image to obtain the number of pixels with the largest color proportion within a preset chromaticity interval specifically includes: Perform histogram statistics on the chromaticity channel image to obtain the chromaticity component histogram; Performing filtering on the chrominance component histogram to obtain a denoised and smoothed histogram; For the denoised and smoothed histogram, the pixel values ​​with the largest color proportion within the preset chromaticity interval are counted.

5. The obstacle recognition method for a self-propelled device according to claim 4, characterized in that: The preset chromaticity interval ranges from 15 to 180.

6. The obstacle recognition method for a self-propelled device according to claim 1, characterized in that: It also includes counting the y-axis coordinate value of the lower right corner of the contour block, and selecting the obstacle avoidance time according to the size of the y-axis coordinate value, wherein the y-axis coordinate value represents the positional relationship between the corresponding contour block and the unmanned lawn mower self-propelled device.

7. A self-propelled device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the obstacle recognition method applied to a self-propelled device described in any one of claims 1 to 6 are implemented.

8. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the obstacle recognition method applied to a self-propelled device described in any one of claims 1 to 6 are implemented.

Images