Image recognition method and system for self-propelled equipment, self-propelled equipment and readable storage medium

Abstract

The present invention discloses an image recognition method, system, self-propelled device and readable storage medium applied to a self-propelled device; the method comprises the following steps: acquiring an image of the environment of the self-propelled device in its traveling direction; performing contour processing on the image to obtain a contour image with contour information; performing contour detection on the contour image to obtain a contour block; counting the characteristic value of the contour block; and comparing the characteristic value with a preset characteristic value threshold to obtain a recognition result. The present invention performs contour detection on the contour image to obtain a contour block, and can effectively identify the abnormality of the contour block according to the characteristic value of the contour block, thereby reducing the misjudgment of the image.

Description

Technical Field

[0001] The present invention relates to an image recognition method and system for self-propelled equipment, a self-propelled equipment and a readable storage medium, and in particular to an image recognition method and system for self-propelled equipment, a self-propelled equipment and a readable storage medium capable of reducing misjudgment. Background Art

[0002] When performing image recognition, existing lawn mowers can identify whether an image includes a non-lawn area based on the chromaticity component of the image. The method of identifying images by chromaticity component works better when the color difference between the lawn and the background is large, but it is easy to cause misjudgment when the color difference between the lawn and the background is small. Summary of the invention

[0003] The present invention provides an image recognition method and system for a self-propelled device, a self-propelled device and a readable storage medium capable of reducing misjudgment.

[0004] The present invention provides an image recognition method applied to a self-propelled device; the method comprises the following steps:

[0005] Acquire an image of the environment of the self-propelled device in the direction of travel;

[0006] Performing contour processing on the image to obtain a contour image with contour information;

[0007] Performing contour detection on the contour image to obtain a contour block;

[0008] Counting the characteristic values ​​of the contour blocks;

[0009] The feature value is compared with a preset feature value threshold to obtain a recognition result.

[0010] Optionally, the characteristic value includes an average brightness value, an average roughness value and an outline size characteristic value, or the characteristic value includes an average brightness value and an average roughness value.

[0011] Optionally, the contour size characteristic value includes contour area, contour diagonal length, contour width, contour height or the number of pixels of the area to be identified included in the contour block.

[0012] Optionally, the counting of characteristic values ​​of the contour block includes:

[0013] Obtaining position information of the contour block;

[0014] Compare the position information of the contour block with a preset position threshold to obtain a target contour block;

[0015] The characteristic values ​​of the target contour block are counted.

[0016] Optionally, the preset position threshold includes a first preset position threshold and a second preset position threshold, and the distance between the area corresponding to the first preset position threshold in the image and the self-propelled device is not equal to the distance between the area corresponding to the second preset position threshold in the image and the self-propelled device, and the preset feature value threshold includes a first preset feature value threshold and a second preset feature value threshold, wherein the first preset position threshold corresponds to the first preset feature value threshold, the second preset position threshold corresponds to the second preset feature value threshold, and the first preset feature value threshold is different from the second preset position threshold.

[0017] Optionally, when the preset position threshold is a first preset position threshold, the comparing the position information of the contour block with the preset position threshold to obtain a target contour block is comparing the position information of the contour block with the first preset position threshold to obtain a first target contour block; the counting of the characteristic value of the target contour block is counting the first characteristic value of the first target contour block, the first characteristic value including an average brightness value, an average roughness value and a contour size characteristic value;

[0018] When the preset position threshold is the second preset position threshold, the comparing the position information of the contour block with the preset position threshold to obtain the target contour block is the comparing the position information of the contour block with the second preset position threshold to obtain the second target contour block; and the counting of the characteristic value of the target contour block is the counting of the second characteristic value of the second target contour block, and the second characteristic value includes the average brightness value and the average roughness value.

[0019] Optionally, performing contour processing on the image to obtain a contour image with contour information comprises:

[0020] Separating the image to obtain a v channel image;

[0021] Performing filtering and normalization processing on the v channel image to obtain a preprocessed image;

[0022] Performing edge extraction on the preprocessed image to obtain an edge image;

[0023] The edge image is expanded, inverted, and eroded to obtain a contour image.

[0024] The present invention also provides an image recognition system for a self-propelled device, the system comprising:

[0025] An image acquisition module, which is used to acquire an image of the environment of the self-propelled device in its travel direction;

[0026] An image processing module, used for performing contour processing on the image to obtain a contour image with contour information;

[0027] A contour detection module, which is used to perform contour detection on the contour image to obtain a contour block;

[0028] A feature processing module, which is used to count the feature values ​​of the contour block;

[0029] The image recognition module compares the feature value with a preset feature value threshold to obtain a recognition result.

[0030] The present invention also provides a self-propelled device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the image recognition method applied to the self-propelled device when executing the computer program.

[0031] The present invention also provides a readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the image recognition method applied to a self-propelled device are implemented.

[0032] Compared with the prior art, the present invention obtains contour blocks by performing contour detection on contour images, and can effectively identify the abnormality of contour blocks according to the characteristic values ​​of contour blocks, thereby reducing the misjudgment of images. The present invention obtains the target contour block according to the position information of the contour block, and the target contour block is located in the area close to the self-propelled device in the image, so that the self-propelled device performs obstacle avoidance identification when it is close to the abnormality (that is, the position information of the contour block is within the range of the preset position threshold), and the self-propelled device does not need to perform obstacle avoidance identification when it is far away from the abnormality (that is, the position information of the contour block exceeds the range of the preset position threshold), so as to reasonably select the obstacle avoidance time. The present invention sets a first preset position threshold and a second preset position threshold, and sets a first preset characteristic value threshold and a second preset characteristic value threshold corresponding thereto, so that the target contour blocks at different positions are compared with the corresponding preset characteristic value threshold, effectively reducing the misjudgment caused by a single preset characteristic value threshold. The present invention obtains a contour image after performing separation processing, filtering processing, normalization processing, edge extraction, expansion, negation, and corrosion processing on the image to improve the recognition rate of the contour image. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A flowchart of an image recognition method applied to a self-propelled device according to the present invention;

[0034] Figure 2 for Figure 1 Flowchart of step S40;

[0035] Figure 3A schematic diagram of an image recognition method for a self-propelled device to obtain image position information;

[0036] Figure 4 for Figure 1 Flowchart of step S20;

[0037] Figure 5 To pass Figure 4 Obtain the effect diagram of the contour image;

[0038] Figure 6 The figure is a principle block diagram of the image recognition system applied to a self-propelled device according to the present invention. DETAILED DESCRIPTION

[0039] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0040] See also Figure 1-6 As shown, the present invention provides an image recognition method applied to a self-propelled device; the method comprises the following steps:

[0041] Step S10: Acquire an image orgMat of the environment of the self-propelled device in its traveling direction;

[0042] Step S20: performing contour processing on the image to obtain a contour image dstMat having contour information;

[0043] Step S30: performing contour detection on the contour image dstMat to obtain a contour block;

[0044] Step S40: Counting the characteristic values ​​of the contour block;

[0045] Step S50: Compare the feature value with a preset feature value threshold to obtain a recognition result.

[0046] In another embodiment of the present invention, the self-propelled device in step S10 is a lawn mower or a sweeper, and the image in step S10 is an image obtained by the lawn mower or the sweeper in its traveling direction. When the self-propelled device is a lawn mower, the preset characteristic value threshold is a threshold related to the lawn, and the recognition result includes that the image has a non-lawn area, the image is an image to be screened, the image to be screened may be a lawn image, and the non-lawn area may be an abnormal area such as an obstacle or a boundary. The image recognition method applied to a self-propelled device by the present invention can filter out images that may be lawns, so as to facilitate subsequent image processing. When the self-propelled device is a sweeper, the preset characteristic value threshold is a threshold related to the road surface, and the recognition result includes that the image has a non-road surface area, the image is an image to be screened, the image to be screened may be a road surface image, and the non-road surface area may be an abnormal area such as an obstacle or a boundary.

[0047] In another embodiment of the present invention, the contour detection in step S30 is to detect the outermost contour block, and the inner contour block contained in the outermost contour block is ignored. Step S30 can detect the number of contour blocks and the continuous contour points on each contour block through the findContours() function in the openCV library.

[0048] In another embodiment of the present invention, the characteristic values ​​in step S40 include average brightness value, average roughness value and contour size characteristic value, or the characteristic values ​​include average brightness value and average roughness value.

[0049] In another embodiment of the present invention, the contour size characteristic value includes contour area, contour diagonal length, contour width, contour height or the number of pixels of the area to be identified included in the contour block. For example, the contour image has white pixels and black pixels, the white pixels represent possible anomalies, and the area to be identified is a collection of white pixels in a single contour block.

[0050] In another embodiment of the present invention, the step S40 includes:

[0051] Step S410: obtaining the position information of the contour block;

[0052] Step S420: Compare the position information of the contour block with a preset position threshold to obtain a target contour block;

[0053] Step S430: Counting the characteristic values ​​of the target contour block.

[0054] The position information of the contour block in step S410 is determined by the preset coordinate system of the image, and the preset coordinate system includes an X-coordinate axis and a Y-coordinate axis, and the X-coordinate axis and the Y-coordinate axis are respectively parallel to the boundary of the image, and the position information of the contour block is the Y-coordinate value of the contour block, and the unit of the Y-coordinate value is pixel, and the Y-coordinate value of the contour block indicates the distance between the contour block and the self-propelled device. Assuming that the image is a rectangle, one of the boundary intersections of the image is used as the coordinate origin of the preset coordinate system, and the X-coordinate axis and the Y-coordinate axis intersect at the coordinate origin. The first preset position threshold and the second preset position threshold are set according to the extension direction of the Y-coordinate axis. Different preset coordinate systems correspond to the Y-coordinate value of the contour block and the first preset position threshold and the second preset position threshold, and the unit of the first preset position threshold and the second preset position threshold is pixel.

[0055] The step S420 of comparing the position information of the contour block with a preset position threshold to obtain a target contour block includes:

[0056] Obtaining position information of the contour block through the actual boundary or virtual boundary of the contour block;

[0057] Determining whether the contour block is at least partially located within a preset position threshold according to the position information;

[0058] When at least a portion of the contour block is located within a preset position threshold, the contour block is a target contour block; otherwise, the contour block is not a target contour block.

[0059] The virtual boundary is generated after the actual boundary of the contour block is virtualized. For example, the actual boundary is an irregular shape, the virtual boundary is a rectangle, and the sides of the virtual boundary are parallel to the X-coordinate axis and the Y-coordinate axis respectively. A figure of a specific shape can also be generated according to other rules of virtual processing, and the position information obtained through the virtual boundary is convenient for comparison processing.

[0060] In another embodiment of the present invention, the preset position threshold includes a first preset position threshold and a second preset position threshold, and the distance between the area corresponding to the first preset position threshold in the image and the self-propelled device is not equal to the distance between the area corresponding to the second preset position threshold in the image and the self-propelled device, and the preset feature value threshold includes a first preset feature value threshold and a second preset feature value threshold, wherein the first preset position threshold corresponds to the first preset feature value threshold, the second preset position threshold corresponds to the second preset feature value threshold, and the first preset feature value threshold is different from the second preset position threshold.

[0061] In another embodiment of the present invention, when the preset position threshold is a first preset position threshold, the step of comparing the position information of the contour block with the preset position threshold to obtain a target contour block is to compare the position information of the contour block with the first preset position threshold to obtain a first target contour block; the step of counting the characteristic value of the target contour block is to count the first characteristic value of the first target contour block, wherein the first characteristic value includes an average brightness value, an average roughness value and a contour size characteristic value;

[0062] When the preset position threshold is the second preset position threshold, the comparing the position information of the contour block with the preset position threshold to obtain the target contour block is the comparing the position information of the contour block with the second preset position threshold to obtain the second target contour block; and the counting of the characteristic value of the target contour block is the counting of the second characteristic value of the second target contour block, and the second characteristic value includes the average brightness value and the average roughness value.

[0063] In another embodiment of the present invention, step S20 includes:

[0064] Step S201: Separate the image orgMat to obtain a v channel image;

[0065] Step S202: filtering and normalizing the v channel image to obtain a pre-processed image;

[0066] Step S203: extracting edges from the preprocessed image to obtain an edge image cannyMat; the edge extraction may use a canny operator or the like;

[0067] Step S204: dilate, negate and erode the edge image cannyMat to obtain a contour image dstMat.

[0068] Assume the contour size eigenvalue AContours i is the number of pixels in the area to be identified included in the contour block, and the average brightness value BContours i = pixel value / contour size feature value AContours i , where the pixel value range of a single pixel is 0-255, that is, 256 levels of grayscale. The pixel value for a single-channel image can also be called a grayscale value. The larger the grayscale value, the brighter it is. Therefore, the v-channel image is filtered and normalized to obtain a preprocessed image; the preprocessed image can be used to calculate the brightness, and the pixel value is the sum of the pixel values ​​in the preprocessed image corresponding to the white pixel position in the contour block in the contour image. Average roughness value HContours i= Number of pixels with pixel values ​​> 100 in the edge image / contour size feature value AContours i The i is the number of the contour block, and different contour blocks are represented by different numbers.

[0069] Assume that the first preset position threshold is set to 60, and the second preset position threshold is set to 80.

[0070] The first comparison process corresponding to the first preset position threshold is as follows:

[0071] First pre-order judgment: 50 <BContours i <200 and YContours i >60

[0072] First post-order judgment:

[0073] 1-a.AContours i >520 and HContours i <0.02

[0074] 1-b.AContours i >1000 and HContours i <0.04

[0075] 1-c.AContours i >3000 and HContours i <0.07

[0076] 1-d.AContours i >5000 and HContours i <0.09

[0077] 1-e.AContours i >10000 and HContours i <0.15

[0078] The second comparison process corresponding to the second preset position threshold is as follows:

[0079] Second pre-order judgment: 50 <BContours i <200 and YContours i >80

[0080] Second post-order judgment: HContours i <0.05

[0081] In the process of identifying whether there is a non-lawn area in an image, if the image meets the first pre-order judgment and meets any one of the first post-order judgments, then the image has a non-lawn area; if the image does not meet the first pre-order judgment, or meets the first pre-order judgment but does not meet any one of the first post-order judgments, then the image is an image to be screened, and the image to be screened may be a lawn image, and subsequent image processing (judgment of other feature values ​​in the image to be screened) is required to further identify whether there is a non-lawn area in the image.

[0082] If the image meets the second pre-order judgment and the second post-order judgment, then the image has a non-lawn area; if the image does not meet the second pre-order judgment, or meets the second pre-order judgment but does not meet the second post-order judgment, then the image is an image to be screened, and the image to be screened may be a lawn image, and subsequent image processing (judgment of other feature values ​​in the image to be screened) is required to further identify whether the image has a non-lawn area.

[0083] The present invention also provides an image recognition system 1 applied to a self-propelled device, the system 1 comprising:

[0084] An image acquisition module 10, which is used to acquire an image of the environment of the self-propelled device in its travel direction;

[0085] An image processing module 20, which is used to perform contour processing on the image to obtain a contour image with contour information;

[0086] A contour detection module 30, which is used to perform contour detection on the contour image to obtain a contour block;

[0087] A feature processing module 40, which is used to count the feature values ​​of the contour block;

[0088] The image recognition module 50 compares the feature value with a preset feature value threshold to obtain a recognition result.

[0089] The present invention also provides a self-propelled device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the image recognition method applied to the self-propelled device when executing the computer program.

[0090] The present invention also provides a readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the image recognition method applied to a self-propelled device are implemented.

[0091] In summary, the present invention obtains contour blocks by performing contour detection on contour images, and can effectively identify the abnormality of contour blocks according to the characteristic values ​​of contour blocks, thereby reducing the misjudgment of images. The present invention obtains the target contour block according to the position information of the contour block, and the target contour block is located in the area closer to the self-propelled device in the image, so that the self-propelled device performs obstacle avoidance identification when it is close to the abnormality (that is, the position information of the contour block is within the range of the preset position threshold), and the self-propelled device does not need to perform obstacle avoidance identification when it is far away from the abnormality (that is, the position information of the contour block exceeds the range of the preset position threshold), so as to reasonably select the obstacle avoidance time. The present invention sets a first preset position threshold and a second preset position threshold, and sets a first preset characteristic value threshold and a second preset characteristic value threshold corresponding thereto, so that the target contour blocks at different positions are compared with the corresponding preset characteristic value threshold, effectively reducing the misjudgment caused by a single preset characteristic value threshold. The present invention obtains a contour image after performing separation processing, filtering processing, normalization processing, edge extraction, expansion, negation, and corrosion processing on the image to improve the recognition rate of the contour image.

[0092] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each implementation mode may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

[0093] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. All equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

Claims

1. An image recognition method applied to a self-propelled device; characterized in that: The method comprises the following steps: Acquire an image of the environment of the self-propelled device in the direction of travel; Performing contour processing on the image to obtain a contour image with contour information; Performing contour detection on the contour image to obtain a contour block; Counting the characteristic values ​​of the contour block, the characteristic values ​​include average brightness value, average roughness value and contour size characteristic value, or the characteristic values ​​include average brightness value and average roughness value; the contour size characteristic value includes contour area, contour diagonal length, contour width, contour height or the number of pixels of the area to be identified included in the contour block; Compare the feature value with a preset feature value threshold to obtain a recognition result, wherein the recognition result is used to indicate whether there is an abnormal area in the image, and the abnormal area includes an obstacle or a boundary; The counting of the characteristic values ​​of the contour block includes: Obtaining position information of the contour block; Compare the position information of the contour block with a preset position threshold to obtain a target contour block, the preset position threshold includes a first preset position threshold and a second preset position threshold, and the distance between the area corresponding to the first preset position threshold in the image and the self-propelled device is not equal to the distance between the area corresponding to the second preset position threshold in the image and the self-propelled device, the preset feature value threshold includes a first preset feature value threshold and a second preset feature value threshold, the first preset position threshold corresponds to the first preset feature value threshold, the second preset position threshold corresponds to the second preset feature value threshold, and the first preset feature value threshold is different from the second preset position threshold; The characteristic values ​​of the target contour block are counted.

2. The image recognition method applied to a self-propelled device according to claim 1, characterized in that: When the preset position threshold is the first preset position threshold, the comparing the position information of the contour block with the preset position threshold to obtain the target contour block is to compare the position information of the contour block with the first preset position threshold to obtain the first target contour block; the counting of the characteristic value of the target contour block is to count the first characteristic value of the first target contour block, and the first characteristic value includes an average brightness value, an average roughness value and a contour size characteristic value; When the preset position threshold is the second preset position threshold, the comparing the position information of the contour block with the preset position threshold to obtain the target contour block is the comparing the position information of the contour block with the second preset position threshold to obtain the second target contour block; and the counting of the characteristic value of the target contour block is the counting of the second characteristic value of the second target contour block, and the second characteristic value includes the average brightness value and the average roughness value.

3. The image recognition method applied to a self-propelled device according to claim 1, characterized in that: The performing contour processing on the image to obtain a contour image with contour information comprises: Separating the image to obtain a v channel image; Performing filtering and normalization processing on the v channel image to obtain a preprocessed image; Performing edge extraction on the preprocessed image to obtain an edge image; The edge image is expanded, inverted, and eroded to obtain a contour image.

4. An image recognition system applied to a self-propelled device, characterized in that: The system comprises: An image acquisition module, which is used to acquire an image of the environment of the self-propelled device in its travel direction; An image processing module, used for performing contour processing on the image to obtain a contour image with contour information; A contour detection module, which is used to perform contour detection on the contour image to obtain a contour block; A feature processing module, which is used to count the feature values ​​of the contour block; An image recognition module compares the feature value with a preset feature value threshold to obtain a recognition result; The feature processing module is also used to obtain the position information of the contour block; compare the position information of the contour block with a preset position threshold to obtain a target contour block, the preset position threshold includes a first preset position threshold and a second preset position threshold, and the distance between the area corresponding to the first preset position threshold in the image and the self-propelled device is not equal to the distance between the area corresponding to the second preset position threshold in the image and the self-propelled device, the preset feature value threshold includes a first preset feature value threshold and a second preset feature value threshold, the first preset position threshold corresponds to the first preset feature value threshold, the second preset position threshold corresponds to the second preset feature value threshold, and the first preset feature value threshold is different from the second preset position threshold; and count the feature values ​​of the target contour block.

5. A self-propelled device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the image recognition method applied to a self-propelled device described in any one of claims 1 to 3 are implemented.

6. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image recognition method applied to a self-propelled device described in any one of claims 1 to 3 are implemented.

Images