Edge control method, device, medium and self-moving device of a self-moving device

Abstract

The present application discloses a method, device, medium and self - moving device for edge - following control of a self - moving device. The method includes: when it is detected that the self - moving device moves to a specified area, acquiring an environmental image; performing image segmentation processing on the environmental image to obtain a segmented image; the image segmentation processing includes multiple serial target feature extraction operations, and each target feature extraction operation includes multiple parallel convolution operations and a fusion operation on the results of the multiple convolution operations; the segmented image is used to indicate the working area and the non - working area in the environmental image; extracting a plurality of boundary pixel points between the working area and the non - working area in the segmented image to obtain a boundary image; and controlling the self - moving device to move along the edge according to the boundary image. The technical solution of the present application improves the recognition accuracy of the boundary pixel points in the boundary image, so that the self - moving device can more accurately locate the boundary of the working area during the edge - following process, effectively improving the edge - following effect of the self - moving device.

Description

Technical Field

[0001] This application belongs to the technical field of artificial intelligence, and particularly relates to a method, device, medium and self-mobile device for edge control of a self-mobile device. Background Art

[0002] In recent years, self-mobile devices have been increasingly widely used in people's daily work and life. For example, self-mobile devices are used for lawn maintenance, environmental cleaning, goods handling, etc. Self-mobile devices usually move within a specified working area. When moving to the edge of the working area, the self-mobile device needs to move along the edge. In related technologies, by setting the working map of the self-mobile device, and then based on the positioning of the self-mobile device in the working map, the self-mobile device is enabled to move along the edge. However, in some cases, the positioning accuracy of the self-mobile device is low, resulting in the self-mobile device being unable to accurately identify the edge of the working area, thus leading to poor edge-following effects.

[0003] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of this application, and thus may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, medium and self-mobile device for edge control of a self-mobile device, so as to solve the problem in related technologies that the boundary of the edge following of the self-mobile device cannot be accurately identified.

[0005] Other features and advantages of this application will become apparent through the following detailed description, or will be partially learned through the practice of this application.

[0006] According to one aspect of the embodiments of this application, a method for edge control of a self-mobile device is provided, including:

[0007] When it is detected that the self-mobile device moves to a specified area, obtain an environmental image;

[0008] Perform image segmentation processing on the environmental image to obtain a segmented image; the image segmentation processing includes multiple serial target feature extraction operations, and each target feature extraction operation includes multiple parallel convolution operations and a fusion operation on the results of the multiple convolution operations; the segmented image is used to indicate the working area and the non-working area in the environmental image;

[0009] Extract multiple boundary pixel points between the working area and the non-working area in the segmented image to obtain a boundary image;

[0010] Control the self-mobile device to move along the edge according to the boundary image.

[0011] According to one aspect of the embodiments of the present application, there is provided a device for controlling the movement along the edge of a self - moving device, including:

[0012] An environmental image acquisition module, configured to acquire an environmental image when it is detected that the self - moving device moves to a specified area;

[0013] An image segmentation module, configured to perform image segmentation processing on the environmental image to obtain a segmented image; the image segmentation processing includes multiple serial target feature extraction operations, and each target feature extraction operation includes multiple parallel convolution operations and a fusion operation on the results of the multiple convolution operations; the segmented image is used to indicate the working area and the non - working area in the environmental image;

[0014] A boundary image acquisition module, configured to extract multiple boundary pixel points between the working area and the non - working area in the segmented image to obtain a boundary image;

[0015] An edge - following module, configured to control the self - moving device to move along the edge according to the boundary image.

[0016] In an embodiment of the present application, the image segmentation module is specifically configured to:

[0017] Use the output data of the (i - 1)th target feature extraction as the input data of the ith target feature extraction; where 2 ≤ i ≤ K, and K is the preset number of feature extraction times; the input data of the first target feature extraction is the feature map obtained by performing a convolution operation on the environmental image;

[0018] Perform multiple parallel convolution operations on the input data of the ith target feature extraction to obtain multiple convolution results; where one convolution operation corresponds to obtaining one convolution result;

[0019] Fuse the multiple convolution results to obtain a fused feature, and perform activation processing on the fused feature to obtain the output data of the ith target feature extraction.

[0020] In an embodiment of the present application, the device further includes:

[0021] A moving direction determination module, configured to determine the perpendicular bisector of the boundary image and the image edge of the boundary image; use the intersection point of the perpendicular bisector and the image edge as the projection pixel point of the self - moving device; determine the target boundary pixel point closest to the projection pixel point from the multiple boundary pixel points in the boundary image; and determine the moving direction of the self - moving device according to the positions of the projection pixel point and the target boundary pixel point.

[0022] In an embodiment of the present application, the device further includes:

[0023] A turning point detection module, configured to detect whether the self - moving device reaches a turning point during the process of the self - moving device moving along the edge;

[0024] A moving direction adjustment module, configured to adjust the moving direction of the self-moving device according to the currently detected working area when the self-moving device reaches a turning point.

[0025] In an embodiment of the present application, the turning point detection module is specifically configured to:

[0026] Calculate the area of the currently detected working area;

[0027] When the area of the currently detected working area is less than a preset area threshold, it is determined that the self-moving device has reached a turning point.

[0028] In an embodiment of the present application, the moving direction adjustment module is specifically configured to:

[0029] Divide the currently detected working area into a first working area and a second working area according to a preset dividing line;

[0030] When the area of the first working area is greater than the area of the second working area, control the self-moving device to adjust a first preset angle in a first direction; wherein, the first direction is the direction towards the first working area;

[0031] When the area of the second working area is greater than the area of the first working area, control the self-moving device to adjust a second preset angle in a second direction; wherein, the second direction is the direction towards the second working area.

[0032] In an embodiment of the present application, the device further includes:

[0033] A detection module, configured to obtain a working map of the self-moving device; wherein, the working map includes the boundaries of each working area; obtain the positioning information of the self-moving device; determine the distance from the self-moving device to the boundary according to the positioning information; when the distance is within a preset distance range, determine that the self-moving device has moved to a specified area in the working area.

[0034] According to one aspect of the embodiments of the present application, there is provided a computer-readable medium, on which a computer program is stored, and when the computer program is executed by a processor, the edge control method of the self-moving device in the above technical solution is implemented.

[0035] According to one aspect of the embodiments of the present application, there is provided an electronic device, which includes: a processor; and a memory for storing executable instructions of the processor; wherein, when the processor executes the executable instructions, the electronic device executes the edge control method of the self-moving device in the above technical solution.

[0036] According to one aspect of the embodiments of the present application, there is provided a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the edge control method of the self-moving device in the above technical solution.

[0037] In the technical solution provided by the embodiments of the present application, when the self-moving device moves to a specified area in the working area, an environmental image is acquired and the environmental image is subjected to image segmentation processing to obtain a segmentation image that distinguishes the working area and the non-working area, and then a boundary image is obtained from the segmentation image; wherein, the image segmentation processing includes multiple serial target feature extractions, and each target feature extraction includes multiple parallel convolution operations and a result fusion operation of multiple parallel convolution operations; after obtaining the boundary image, the self-moving device is controlled to move along the edge according to the boundary image. On the one hand, the movement of the automatic moving device to the specified area is equivalent to a rough positioning of the self-moving device. In this specified area, processing the environmental image to obtain a boundary image is a fine positioning of the boundary where the self-moving device is located. Thus, a method of combining rough positioning and fine positioning is implemented to control the self-moving device to move along the edge, improving the accuracy of boundary positioning. On the other hand, in the segmentation processing of the environmental image, multiple serial feature extractions deepen the depth of feature extraction, thereby obtaining deep features of the environmental image; at the same time, multiple parallel convolution operations in each feature extraction process retain the shallow features of the environmental image. Then, the final result fusion operation fuses the deep features and shallow features of the environmental image, which is beneficial to improving the accuracy of image segmentation, and further improves the recognition accuracy of boundary pixel points in the boundary image, so that the self-moving device can more accurately locate the working area boundary during the edge movement process, effectively improving the edge following effect of the self-moving device.

[0038] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings here are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0040] Figure 1 The structural block diagram of the self-moving device applying the technical solution of the present application is schematically shown.

[0041] Figure 2 The flowchart of the edge control method for a self - moving device provided by an embodiment of the present application is schematically shown.

[0042] Figure 3A The flowchart of the image segmentation process provided by an embodiment of the present application is schematically shown.

[0043] Figure 3B The schematic diagram of the target feature extraction process provided by an embodiment of the present application is schematically shown.

[0044] Figure 3C The schematic diagram of the target feature extraction process provided by an embodiment of the present application is schematically shown.

[0045] Figure 4 The schematic diagram of the boundary image provided by an embodiment of the present application is schematically shown.

[0046] Figure 5 The schematic diagram of the boundary image provided by an embodiment of the present application is schematically shown.

[0047] Figure 6 The structural block diagram of the edge control device for a self - moving device provided by an embodiment of the present application is schematically shown.

[0048] Figure 7 The structural block diagram of a self - moving device suitable for implementing the embodiments of the present application is schematically shown. Detailed implementation manners

[0049] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more complete and comprehensive, and will fully convey the concept of the example embodiments to those skilled in the art.

[0050] In addition, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present application. However, those skilled in the art will realize that the technical solutions of the present application can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, well - known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present application.

[0051] The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0052] The flowcharts shown in the drawings are only exemplary descriptions, and do not necessarily include all contents and operations / steps, nor are they necessarily executed in the described order. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined. Therefore, the actual execution order may be changed according to the actual situation.

[0053] Figure 1 The structural block diagram of the self - moving device applying the technical solution of the present application is schematically shown.

[0054] As Figure 1 shown, the self - moving device includes a vehicle body 110 and a control module 120. The vehicle body 110 includes a vehicle body 111 and wheels 112. The control module 120 is disposed on the vehicle body 110. Generally, the control module 120 is disposed on the vehicle body 111. The control module 120 is used to receive the control instructions of the self - moving device or generate various control instructions for the self - moving device. The self - moving device in the embodiment of the present application can be a device with a self - moving assistance function. Among them, the self - moving assistance function can be realized by an in - vehicle terminal, and the corresponding self - moving device can be a vehicle with the in - vehicle terminal. The self - moving device can also be a semi - self - moving device or a fully autonomous moving device, such as robots like floor - cleaning robots, floor - mopping robots, vegetable - delivering robots, transportation robots, lawn - mowing robots, etc. The specific types and functions of the self - moving device in the embodiment of the present application are not limited. It can be understood that the self - moving device in this embodiment can also include other devices with self - moving functions.

[0055] In the embodiment of the present application, the control module 120 is used to implement the edge - following control method of the self - moving device provided in any embodiment of the present application. A camera device 130 can be provided on the self - moving device, and the camera device 130 is connected to the control module 120 inside the self - moving device.

[0056] First, when the control module 120 detects that the self - moving device moves to a specified area, it acquires an environmental image. The specific process can be as follows: when the control module 120 detects that the self - moving device moves to a specified area, it sends a photographing instruction to the imaging device 130, so as to acquire the environmental image through the imaging device 130. The imaging device 130 can be fixed or non - fixed and rotatable. The embodiments of the present application do not make any limitation in this regard. The environmental image captured by the imaging device 130 can be a color image, a black - and - white image, an infrared image, etc. The embodiments of the present application do not make any limitation in this regard. Exemplarily, the imaging device 110 is an RGB camera, and the RGB camera captures the environment in the forward direction of the self - moving device to obtain the environmental image.

[0057] Next, the control module 120 performs image segmentation processing on the environmental image to obtain a segmented image. Among them, the image segmentation processing includes multiple serial target feature extraction operations. Each target feature extraction operation includes multiple parallel convolution operations and a fusion operation on the results of the multiple convolution operations. The segmented image is used to indicate the working area and the non - working area in the environmental image.

[0058] Then, the control module 120 extracts multiple boundary pixel points between the working area and the non - working area in the segmented image to obtain a boundary image.

[0059] Finally, the control module 120 controls the self - moving device to move along the edge according to the boundary image. The control module 120 is also connected to the driving components of the self - moving device, such as the steering shaft, steering wheel, motor, etc. of the self - moving device, for controlling the movement, steering, etc. of the self - moving device, and further controlling the self - moving device to move along the edge.

[0060] The following makes a detailed description of the edge - following control method of the self - moving device provided by the present application in combination with specific embodiments.

[0061] Figure 2 Schematically shows a flowchart of the edge - following control method of the self - moving device provided by an embodiment of the present application. As Figure 2 shown, the method includes steps 210 to 240, which are specifically as follows:

[0062] Step 210: When it is detected that the self - moving device moves to a specified area, acquire an environmental image.

[0063] Specifically, the specified area is an area preset to be closer to the boundary of the working area of the self - moving device. For example, the specified area is an area where the distance from the working area boundary is less than a preset threshold. The environmental image refers to an image of the current physical environment in the forward direction of the self - moving device, and this environmental image can be obtained by photographing with an imaging device installed on the self - moving device. The environmental image can be an RGB image or a depth image, etc., and no limitation is made here.

[0064] In an embodiment of the present application, the process of detecting whether a self - moving device moves to a specified area includes: obtaining a working map of the self - moving device; obtaining the positioning information of the self - moving device; determining the distance from the self - moving device to the boundary according to the positioning information; and when the distance is within a preset distance range, determining that the self - moving device moves to the specified area in the working area.

[0065] Specifically, the self - moving device usually moves within the working area planned by the working map. The working map includes the boundaries of each working area of the self - moving device, which means that the working map has the positioning information of the boundaries of each working area. During the movement of the self - moving device, the positioning information of the self - moving device is obtained. For example, it can be obtained through the Global Navigation Satellite System (GNSS), and GNSS includes but is not limited to the Global Positioning System (GPS), BeiDou Navigation Satellite System (BDS), Global Navigation Satellite System (GLONASS), Galileo satellite positioning system and other methods to determine the positioning information of the self - moving device. Then, according to the positioning information of the self - moving device and the positioning information of the boundaries of each working area, the distance from the self - moving device to the boundary of the working area is calculated. When the distance is within the preset distance range, it means that the self - moving device has moved near the boundary of the working area, and it can be determined that the self - moving device moves to the specified area in the working area. For example, when the distance from the self - moving device to the boundary of the working area is less than 2 meters of the set distance range, it is considered that the self - moving device moves to the specified area.

[0066] Although methods such as GPS positioning and BDS positioning can be used to determine whether the self - moving device reaches near the boundary of the working area, the positioning accuracy of this positioning method is not high, and it is difficult to accurately determine whether the self - moving device reaches the boundary of the working area. In the embodiment of the present application, first, the self - moving device is determined to reach near the boundary of the working area through rough positioning, and then the boundary of the working area is accurately positioned by combining image processing in subsequent steps, that is, the accurate positioning of the boundary of the working area is realized, which can effectively improve the accuracy of the edge control of the self - moving device.

[0067] Step 220: Perform image segmentation processing on the environmental image to obtain a segmented image; the image segmentation processing includes multiple serial target feature extraction operations, and each target feature extraction operation includes multiple parallel convolution operations and a fusion operation on the results of the multiple convolution operations; the segmented image is used to indicate the working area and the non - working area in the environmental image.

[0068] Specifically, image segmentation is to identify the working area and non-working area in the environmental image, and obtain a segmentation image that can distinguish the working area and the non-working area, so as to facilitate the subsequent extraction of the boundary between the working area and the non-working area.

[0069] In this embodiment, the image segmentation process includes multiple serial target feature extraction operations. Each target feature extraction operation includes multiple parallel convolution operations and a fusion operation on the results of the multiple convolution operations. The multiple serial target feature extraction operations are equivalent to connecting multiple target feature extraction operations in series in sequence. The output data of the previous target feature extraction operation is the input data of the next target feature extraction operation. The multiple parallel convolution operations mean that the multiple convolution operations are carried out in parallel or synchronously, and the data processed by one convolution operation has no direct association with the data processed by other convolution operations. The fusion operation on the results of the multiple convolution operations is to fuse the results of each convolution operation into one feature. For example, adding the results of each convolution operation, weighted summation, etc.

[0070] Exemplarily, Figure 3A Schematically shows a flowchart of the image segmentation process provided by an embodiment of the present application. As Figure 3A shown, in the K target feature extraction operations of the image segmentation process, the output data of the (i - 1)-th target feature extraction is the input data of the i-th target feature extraction, thus constituting K serial target feature extraction operations. Among them, 2 ≤ i ≤ K, and K is the preset number of feature extraction times. It should be noted that the input data of the first target feature extraction is the feature map obtained by performing a convolution operation on the environmental image, that is, the environmental image enters the target feature extraction operation after one convolution operation. After K target feature extraction operations, a segmentation image is output.

[0071] In the i-th target feature extraction operation, first perform multiple parallel convolution operations on the input data respectively. One convolution operation obtains one convolution result, so as to obtain multiple convolution results. Exemplarily, Figure 3B Schematically shows a schematic diagram of the target feature extraction process provided by an embodiment of the present application. As Figure 3B shown, the i-th target feature extraction operation includes M parallel convolution operations, and the input data of each convolution operation is the input data of the i-th target feature extraction operation.

[0072] Then, fuse the multiple convolution results to obtain a fused feature. Exemplarily, as Figure 3B shown, add (Add) the M convolution results corresponding to the M convolution operations to obtain a fused feature.

[0073] Finally, the fused features are activated to obtain the output data of the i-th target feature extraction. Activation functions can include ReLU (Linear rectification function) and sigmoid (S-shaped function).

[0074] It should be noted that the convolution processing, multiple convolution operations, etc. in this application all involve convolution calculations, but the calculation parameters involved in the convolution processing or convolution operations can be different. For example, the convolution kernel size, stride, number of channels, etc. involved in the convolution processing or convolution operation may be different. The multiple parallel convolution operations can have the same calculation parameters or involve different calculation parameters.

[0075] In an embodiment of this application, during some target feature extraction processes, in addition to multiple parallel convolution operations, it also includes normalization processing of the input data. Exemplarily, Figure 3C Schematically shows a schematic diagram of the target feature extraction process provided by an embodiment of this application, as Figure 3C shown. After the input data of the target feature extraction is normalized, it is added to the convolution result of the multiple convolution operations, and then through activation processing, the output data is obtained. The normalization processing can be batch normalization (BN) or group normalization (GN).

[0076] In the image segmentation processing in the related art, a residual network is used. The residual network is usually an ordinary serial convolution operation, and its image segmentation accuracy needs to be improved. From the image segmentation processing provided by the embodiments of this application, it can be seen that the serial multiple target feature extractions deepen the depth of feature extraction, enabling the extraction of deep features of the image; the multiple parallel convolution operations in each target feature extraction retain the shallow features of that feature extraction. Through this method of serial feature extraction and parallel convolution operations, the deep and shallow features of the image can be retained during the image segmentation processing, and the result fusion operation fuses the deep features and shallow features, which is beneficial to improving the accuracy of image segmentation, thereby improving the recognition and localization accuracy of the boundary.

[0077] Exemplarily, Table 1 shows the accuracy comparison between the existing residual network for image segmentation processing and the image segmentation processing provided by this application. It can be seen that this application improves the accuracy of image segmentation and is more conducive to accurately identifying the boundary.

[0078] Table 1

[0079] Average value / Accuracy threshold (%) 50 55 60 65 70 75 80 85 90 95 80 (Related technical solution) 95 94 93 92 89 87 82 75 62 37 82 (Solution of this application) 97 95 94 92 90 89 83 76 65 38

[0080] Step 230: Extract multiple boundary pixel points between the working area and the non-working area in the segmented image to obtain a boundary image.

[0081] Specifically, after obtaining the segmented image, multiple boundary pixel points in the segmented image can be extracted to obtain a boundary image including the boundary. In the segmented image, the pixel points in the working area and the pixel points in the non-working area are two different types of pixel points. For example, the pixel values of the pixel points in the working area and the pixel values of the pixel points in the non-working area are different. Since the boundary pixel points are located at the junction of the working area and the non-working area, there will be two types of pixel points around the boundary pixel points. When extracting the boundary pixel points, it can be determined whether a pixel point is a boundary pixel point by judging whether there are two types of pixel points around the pixel point.

[0082] In an embodiment of the present application, the pixel points of the segmented image include gradient values. When extracting the boundary pixel points, it can be determined whether the pixel points of the segmented image are boundary pixel points according to the gradient value range of the pixel points. For example, when the gradient value of the pixel point is within the range defined by the first threshold and the second threshold, the pixel point is considered a boundary pixel point.

[0083] Step 240: Control the self-moving device to move along the edge according to the boundary image.

[0084] Specifically, the boundary in the boundary image represents the exact boundary of the working area of the self-moving device. An edge-following path is generated according to this boundary, and then the self-moving device is controlled to move along the edge according to this edge-following path.

[0085] In the technical solution provided by the embodiment of the present application, when the self-moving device moves to a specified area in the working area, an environmental image is acquired and the environmental image is subjected to image segmentation processing to obtain a segmentation image that distinguishes the working area and the non-working area, and then a boundary image is obtained from the segmentation image; wherein, the image segmentation processing includes multiple serial target feature extractions, and each target feature extraction includes multiple parallel convolution operations and a result fusion operation of multiple parallel convolution operations; after the boundary image is obtained, the self-moving device is controlled to move along the edge according to the boundary image. On the one hand, the movement of the automatic moving device to the specified area is equivalent to a rough positioning of the self-moving device. In this specified area, processing the environmental image to obtain a boundary image is a fine positioning of the boundary where the self-moving device is located. Thus, a method of combining rough positioning and fine positioning is implemented to control the self-moving device to move along the edge, improving the accuracy of boundary positioning. On the other hand, in the segmentation processing of the environmental image, multiple serial feature extractions deepen the depth of feature extraction, thereby obtaining deep features of the environmental image; at the same time, multiple parallel convolution operations in each feature extraction process retain the shallow features of the environmental image. Then, the final result fusion operation fuses the deep features and shallow features of the environmental image, which is beneficial to improving the accuracy of image segmentation, and further improves the recognition accuracy of boundary pixel points in the boundary image, so that the self-moving device can more accurately locate the working area boundary during the edge movement process, effectively improving the edge movement effect of the self-moving device.

[0086] In an embodiment of the present application, the process of controlling the self-moving device to move along the edge further includes: determining the perpendicular bisector of the boundary image and the image edge of the boundary image; taking the intersection point of the perpendicular bisector and the image edge as the projection pixel point of the self-moving device; determining, from multiple boundary pixel points in the boundary image, the target boundary pixel point closest to the projection pixel point; and determining the moving direction of the self-moving device according to the position of the projection pixel point and the position of the target boundary pixel point.

[0087] Specifically, the perpendicular bisector of the boundary image refers to a line that passes through the center point of the boundary image and is perpendicular to the image edge. The perpendicular bisector and the image edge have two intersection points. Generally, the intersection point closer to the working area, that is, the intersection point of the perpendicular bisector and the lower edge of the image, is taken as the projection pixel point of the self-moving device in the boundary image. Exemplarily, Figure 4 Schematically shows a schematic diagram of a boundary image provided by an embodiment of the present application. As Figure 4 shown, the intersection points of the perpendicular bisector and the image edge include point A and point A'. Point A is inside the working area and belongs to the intersection point of the perpendicular bisector and the lower edge of the image. Point A' is in the non-working area and belongs to the intersection point of the perpendicular bisector and the upper edge of the image. Therefore, point A is the projection pixel point of the self-moving device. The determination of the projection pixel point is similar to the position of the camera in the image it captures, usually the intersection point of the perpendicular bisector and the lower edge of the image.

[0088] After determining the projection pixel points of the self - moving device, find a target boundary pixel point closest to the projection pixel point among multiple boundary pixel points on the boundary. According to the relative position between the target boundary pixel point and the projection pixel point, the moving direction of the self - moving device can be determined, and then the edge - following movement of the self - moving device can be controlled based on this moving direction. Specifically, if the target boundary pixel point is on the left of the projection pixel point, control the self - moving device to move left to the position of the target boundary pixel point; if the target boundary pixel point is on the right of the projection pixel point, control the self - moving device to move right to the position of the target boundary pixel point.

[0089] In an embodiment of the present application, assume that the upper - left corner of the boundary image is used as the origin O, and the two image edges intersecting at the origin are used as the x - axis and y - axis respectively to construct the coordinate system of the boundary image. When the position of the projection pixel point is the pixel coordinate (x0, y0) in this coordinate system, and the position of the boundary pixel point is the pixel coordinate (x, y) in this coordinate system, the distance D between the projection pixel point and the boundary pixel point is:

[0090]

[0091] Among the distances D corresponding to each boundary pixel point, find the boundary pixel point corresponding to the minimum distance, which is the target boundary pixel point.

[0092] Then calculate the relative position d between the target boundary pixel point and the projection pixel point:

[0093] d = x0 - x

[0094] When d > 0, it indicates that the target boundary pixel point is on the left of the projection point, representing that the moving direction of the self - moving device is to the left; when d < 0, it indicates that the target boundary pixel point is on the right of the projection point, representing that the moving direction of the self - moving device is to the right.

[0095] The distance for the self - moving device to move from the projection pixel point to the target boundary pixel point according to the moving direction is determined by PID control. After the self - moving device moves to the target boundary pixel point, it moves according to the boundary in the boundary image. Exemplarily, as Figure 4 shown, if the target boundary pixel point B is on the right of the projection pixel point A, then control the self - moving device to move right to the target boundary pixel point B, and then control the self - moving device to move along the edge according to the recognized boundary.

[0096] In an embodiment of the present application, the edge - following control method of the self - moving device of the present application further includes: during the process of the self - moving device moving along the edge, detect whether the self - moving device reaches a turning point; when the self - moving device reaches the turning point, adjust the moving direction of the self - moving device according to the currently detected working area.

[0097] Specifically, a turning point refers to a position where the moving direction of the self-moving device needs to be changed. Since the self-moving device moves along the edge, when the self-moving device reaches the turning point, it means that an edge path has been completed, and the working area in the boundary image must shrink. Therefore, the working area can be used to determine whether the self-moving device has reached the turning point.

[0098] In an embodiment of the present application, the process of detecting whether the self-moving device reaches the turning point includes: calculating the area of the currently detected working area; when the area of the currently detected working area is less than a preset area threshold, determining that the self-moving device reaches the turning point.

[0099] Exemplarily, assume that the self-moving device starts moving along the edge from the target boundary pixel point B in the Figure 4 shown boundary image. Figure 4 In the shown boundary image, the working area is large, and obviously the target boundary pixel point B is not a turning point. Divide the Figure 4 shown boundary into boundary 1, boundary 2, and boundary 3. The self-moving device starts from the target pixel point B and moves along boundary 1. During the edge movement process, continuously obtain the boundary image and calculate the area of the working area. When the self-moving device moves to the Figure 4 point C in the shown boundary image, the boundary image obtained by the self-moving device at this time is as shown in Figure 5 . In the Figure 5 shown boundary image, the area of the working area is less than the preset area threshold, so it is determined that the self-moving device is currently at the turning point.

[0100] In an embodiment of the present application, it is also possible to detect whether the turning point is reached by the ratio of the working area to the non-working area, that is, when the ratio of the working area to the non-working area is less than a preset threshold, it is determined that the self-moving device reaches the turning point.

[0101] After determining that the self-moving device reaches the turning point, adjust the moving direction of the self-moving device according to the currently detected working area. Specifically, adjust the moving direction of the self-moving device according to the working area on both sides of the turning point. The moving direction of the self-moving device is adjusted towards the side with the larger working area.

[0102] In an embodiment of the present application, the process of adjusting the moving direction at the turning point includes: dividing the currently detected working area into a first working area and a second working area according to a preset dividing line; when the area of the first working area is greater than the area of the second working area, controlling the self - moving device to adjust a first preset angle in a first direction; where the first direction is the direction towards the first working area; when the area of the second working area is greater than the area of the first working area, controlling the self - moving device to adjust a second preset angle in a second direction; where the second direction is the direction towards the second working area.

[0103] Specifically, the preset dividing line is a line passing through the projection pixel point of the self - moving device in the boundary image, for example, the perpendicular bisector of the boundary image. After dividing the working area into a first working area and a second working area through the preset dividing line, when the area of the first working area is greater than the area of the second working area, it indicates that there is a high probability that the next edge - following path is in the direction of the first working area. Therefore, control the self - moving device to adjust the first preset angle in the first direction towards the first working area. After adjusting the angle, the moving direction of the self - moving device is towards the first working area. When the area of the second working area is greater than the area of the first working area, it indicates that there is a high probability that the next edge - following path is in the direction of the second working area. Therefore, control the self - moving device to adjust the second preset angle in the second direction towards the second working area. After adjusting the angle, the moving direction of the self - moving device is towards the second working area.

[0104] Exemplarily, as Figure 5 shown in the boundary image, the working area is divided into a first working area and a second working area by the perpendicular bisector of the image. The area of the first working area is greater than the area of the second working area. Therefore, control the self - moving device to adjust the first preset angle in the first direction, for example, control the self - moving device to rotate 90° to the left.

[0105] In an embodiment of the present application, the first preset angle and the second preset angle can be set according to the angle between the current orientation of the self - moving device and the next boundary. Exemplarily, as Figure 5 shown in the boundary image, the current orientation of the self - moving device is the direction of the perpendicular bisector of the image, and the next boundary is boundary 2. An angle θ is formed between the perpendicular bisector of the image and boundary 2. Then, the self - moving device can be controlled to rotate counterclockwise by an angle θ so that the direction of the self - moving device is consistent with the extension direction of boundary 2.

[0106] In some other embodiments, the first preset angle and the second preset angle can also be pre - set values, such as angle values of 30°, 40°, 50°, etc. The specific setting method of the first preset angle and the second preset angle in the embodiments of the present application is not limited.

[0107] It should be noted that although the various steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that these steps must be executed in that specific order, or that all the steps shown must be executed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution, etc.

[0108] The following introduces the device embodiments of this application, which can be used to execute the edge following control method of the self - moving device in the above - mentioned embodiments of this application. Figure 6 The structural block diagram of the edge following control device of the self - moving device provided by the embodiment of this application is schematically shown. As Figure 6 shown, the device includes:

[0109] An environmental image acquisition module 610, configured to acquire an environmental image when it is detected that the self - moving device moves to a specified area;

[0110] An image segmentation module 620, configured to perform image segmentation processing on the environmental image to obtain a segmented image; the image segmentation processing includes multiple serial target feature extraction operations, and each target feature extraction operation includes multiple parallel convolution operations and a fusion operation on the results of the multiple convolution operations; the segmented image is used to indicate the working area and the non - working area in the environmental image;

[0111] A boundary image acquisition module 630, configured to extract multiple boundary pixel points between the working area and the non - working area in the segmented image to obtain a boundary image;

[0112] An edge following module 640, configured to control the self - moving device to move along the edge according to the boundary image.

[0113] In an embodiment of this application, the image segmentation module 620 is specifically configured to:

[0114] Use the output data of the (i - 1)th target feature extraction as the input data of the ith target feature extraction; where 2 ≤ i ≤ K, and K is the preset number of feature extraction times; the input data of the first target feature extraction is the feature map obtained by performing a convolution operation on the environmental image;

[0115] Perform multiple parallel convolution operations on the input data of the ith target feature extraction to obtain multiple convolution results; where one convolution operation corresponds to obtaining one convolution result;

[0116] Fuse the multiple convolution results to obtain a fused feature, and perform activation processing on the fused feature to obtain the output data of the ith target feature extraction.

[0117] In one embodiment of the present application, the device further includes:

[0118] A moving direction determination module, configured to determine the perpendicular bisector of the boundary image and the image edge of the boundary image; use the intersection point of the perpendicular bisector and the image edge as the projection pixel point of the self-moving device; determine, from multiple boundary pixel points in the boundary image, the target boundary pixel point closest to the projection pixel point; and determine the moving direction of the self-moving device according to the position of the projection pixel point and the position of the target boundary pixel point.

[0119] In one embodiment of the present application, the device further includes:

[0120] A turning point detection module, configured to detect whether the self-moving device reaches a turning point during the process of the self-moving device moving along the edge;

[0121] A moving direction adjustment module, configured to, when the self-moving device reaches the turning point, adjust the moving direction of the self-moving device according to the currently detected working area.

[0122] In one embodiment of the present application, the turning point detection module is specifically configured to:

[0123] Calculate the area of the currently detected working area;

[0124] When the area of the currently detected working area is less than a preset area threshold, determine that the self-moving device reaches the turning point.

[0125] In one embodiment of the present application, the moving direction adjustment module is specifically configured to:

[0126] Divide the currently detected working area into a first working area and a second working area according to a preset dividing line;

[0127] When the area of the first working area is greater than the area of the second working area, control the self-moving device to adjust a first preset angle in a first direction; wherein, the first direction is the direction towards the first working area;

[0128] When the area of the second working area is greater than the area of the first working area, control the self-moving device to adjust a second preset angle in a second direction; wherein, the second direction is the direction towards the second working area.

[0129] In one embodiment of the present application, the device further includes:

[0130] A detection module is configured to obtain a working map of the self - moving device; wherein, the working map includes the boundaries of each working area; obtain the positioning information of the self - moving device; determine the distance from the self - moving device to the boundary according to the positioning information; and when the distance is within a preset distance range, determine that the self - moving device moves to a specified area in the working area.

[0131] The specific details of the edge - following control device of the self - moving device provided in each embodiment of the present application have been described in detail in the corresponding method embodiments, and will not be elaborated here.

[0132] Figure 7 Schematically shows a block diagram of a computer system for implementing the self - moving device of the embodiments of the present application.

[0133] It should be noted that Figure 7 The self - moving device 700 shown is only an example, and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0134] As Figure 7 shown, the self - moving device 700 includes a central processing unit 701 (Central Processing Unit, CPU), which can perform various appropriate actions and processes according to the program stored in the read - only memory 702 (Read - Only Memory, ROM) or the program loaded from the storage section 708 into the random access memory 703 (Random Access Memory, RAM). In the random access memory 703, various programs and data required for system operation are also stored. The central processing unit 701, the read - only memory 702, and the random access memory 703 are connected to each other via a bus 704. An input / output interface 705 (Input / Output interface, i.e., I / O interface) is also connected to the bus 704.

[0135] The following components are connected to the input / output interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including, for example, a cathode ray tube (Cathode Ray Tube, CRT), a liquid crystal display (Liquid Crystal Display, LCD), etc. and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a local area network card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A driver 710 is also connected to the input / output interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto - optical disk, a semiconductor memory, etc., is installed on the driver 710 as needed, so that the computer program read from it can be installed into the storage portion 708 as needed.

[0136] In particular, according to the embodiments of the present application, the processes described in each method flowchart can be implemented as computer software programs. For example, the embodiments of the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit 701, various functions defined in the system of the present application are executed.

[0137] It should be noted that the computer-readable medium shown in the embodiments of the present application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device. In the present application, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program codes. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program codes contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0138] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code, and the above-mentioned module, segment of a program, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0139] It should be noted that although several modules or units of devices for performing actions are mentioned in the above detailed description, such a division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of the two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0140] From the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or can be implemented by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a portable hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.

[0141] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present application. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include common general knowledge or conventional technical means in the technical field not disclosed in the present application.

[0142] It should be understood that the present application is not limited to the exact structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

Claims

1. A method for edge control of a self - moving device, characterized in that, Including: When it is detected that the self - moving device moves to a specified area, obtain an environmental image; Perform image segmentation processing on the environmental image to obtain a segmented image; The image segmentation processing includes multiple serial target feature extraction operations. Each target feature extraction operation includes multiple parallel convolution operations and a fusion operation on the results of the multiple convolution operations; the segmented image is used to indicate the working area and non - working area in the environmental image; Extract multiple boundary pixel points between the working area and the non - working area in the segmented image to obtain a boundary image; Control the self - moving device to move along the edge according to the boundary image; Among them, the i - th target feature extraction operation of the image segmentation processing includes: Use the output data of the (i - 1) - th target feature extraction as the input data of the i - th target feature extraction; where 2 ≤ i ≤ K, and K is the preset number of feature extraction times; the input data of the first target feature extraction is the feature map obtained by performing a convolution operation on the environmental image; Perform multiple parallel convolution operations on the input data of the i - th target feature extraction to obtain multiple convolution results; where one convolution operation corresponds to obtaining one convolution result; Perform normalization processing on the input data of the i - th target feature extraction to obtain a normalization result; After sequentially fusing the multiple convolution results, fuse them with the normalization result to obtain a fused feature, and perform activation processing on the fused feature to obtain the output data of the i - th target feature extraction.

2. The edge-following control method of the self-moving device according to claim 1, wherein Before controlling the self - moving device to move along the edge according to the boundary image, the method further includes: Determine the perpendicular bisector of the boundary image and the image edge of the boundary image; Use the intersection point of the perpendicular bisector and the image edge as the projection pixel point of the self - moving device; Determine the target boundary pixel point closest to the projection pixel point from the multiple boundary pixel points in the boundary image; Determine the moving direction of the self - moving device according to the position of the projection pixel point and the position of the target boundary pixel point.

3. The edge-following control method of the self-moving device according to claim 1 or 2, characterized in that The method further includes: During the process of the self - moving device moving along the edge, detect whether the self - moving device reaches a turning point; When the self - moving device reaches the turning point, adjust the moving direction of the self - moving device according to the currently detected working area.

4. The edge-following control method of the self-moving device according to claim 3, wherein The detecting whether the self - moving device reaches the turning point includes: Calculate the area of the currently detected working area; When the area of the currently detected working area is less than a preset area threshold, determine that the self - moving device reaches the turning point.

5. The edge-following control method of the self-moving device according to claim 3, characterized in that, The adjusting the moving direction of the self - moving device according to the currently detected working area includes: Divide the currently detected working area into a first working area and a second working area according to a preset dividing line; When the area of the first working area is greater than the area of the second working area, control the self - moving device to adjust a first preset angle in a first direction; where the first direction is the direction towards the first working area; When the area of the second working area is larger than the area of the first working area, control the self - moving device to adjust a second preset angle in a second direction; wherein, the second direction is the direction towards the second working area.

6. The edge-following control method of the self-mobile device according to claim 1, wherein Before obtaining the environmental image when it is detected that the self - moving device moves to a specified area, the method further includes: Obtain a working map of the self - moving device; wherein, the working map includes the boundaries of each working area; Obtain the positioning information of the self - moving device; According to the positioning information, determine the distance from the self - moving device to the boundary; When the distance is within a preset distance range, determine that the self - moving device moves to the specified area in the working area.

7. An edge-following control device for a self-moving device, characterized in that, It includes: An environmental image acquisition module, configured to obtain an environmental image when it is detected that the self - moving device moves to a specified area; An image segmentation module, configured to perform image segmentation processing on the environmental image to obtain a segmented image; the image segmentation processing includes multiple serial target feature extraction operations, and each target feature extraction operation includes multiple parallel convolution operations and a fusion operation on the results of the multiple convolution operations; the segmented image is used to indicate the working area and the non - working area in the environmental image; A boundary image acquisition module, configured to extract multiple boundary pixel points between the working area and the non - working area in the segmented image to obtain a boundary image; An edge - following module, configured to control the self - moving device to move along the edge according to the boundary image; Wherein, the image segmentation module is specifically configured to: Use the output data of the (i - 1)th target feature extraction as the input data of the ith target feature extraction; wherein, 2 ≤ i ≤ K, and K is the preset number of feature extractions; the input data of the first target feature extraction is the feature map obtained by performing a convolution operation on the environmental image; Perform multiple parallel convolution operations on the input data of the ith target feature extraction to obtain multiple convolution results; wherein, one convolution operation corresponds to obtaining one convolution result; Perform normalization processing on the input data of the ith target feature extraction to obtain a normalization result; After sequentially fusing the multiple convolution results, fuse them with the normalization result to obtain a fused feature, and perform activation processing on the fused feature to obtain the output data of the ith target feature extraction.

8. A computer-readable medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the edge - following control method of the self - moving device according to any one of claims 1 to 6.

9. A self - moving device, characterized in that, It includes: A vehicle body, the vehicle body includes a body and wheels; And A control module, configured to execute the edge - following control method of the self - moving device according to any one of claims 1 to 6.

Images