Method, apparatus, readable medium and electronic device for generating a movement path

Abstract

The present application discloses a method, apparatus, readable medium and electronic device for generating a movement path. The method includes: obtaining a boundary image, where the boundary image includes at least one boundary, and the boundary includes a plurality of boundary pixel points; constructing a first pixel matrix with each boundary pixel point as the center point to obtain a plurality of first pixel matrices corresponding to each boundary; in all the boundary pixels of any boundary, when at most two boundary pixel points are included in the first pixel matrix corresponding to any boundary pixel point, determining the boundary as the target boundary; generating a movement path according to the plurality of boundary pixel points of the target boundary. The present application enables the robot to work along the edge normally without a boundary induction line, reducing the dependence on hardware devices for boundary recognition and the wiring cost required for traditional robots to work along the boundary.

Description

Technical Field

[0001] This application belongs to the technical field of image processing, and particularly relates to a method, apparatus, readable medium, and electronic device for generating a moving path. Background Art

[0002] With the development of artificial intelligence technology, products developed based on artificial intelligence technology can be used in more and more occasions, such as using robots to perform lawn maintenance operations, using floor-sweeping robots to clean the working area, and using patrolling robots to patrol and inspect the working area. Traditional robots can operate efficiently in the central area of the working area, but they cannot well identify the boundary of the working area, which makes it difficult for robots to maintain the boundary of the working area. Currently, by laying an inductive boundary line underground at the boundary of the working area to notify the robot that it has reached the boundary of the working area and making the robot move along the laid path to complete the maintenance work of the boundary of the working area. However, this method requires the robot to be able to normally sense the laid boundary line. When the sensing is abnormal, the recognition accuracy of the robot for the boundary of the working area decreases, resulting in the robot being unable to accurately generate a boundary movement path, and the stability of the generated robot movement path is poor; at the same time, since a corresponding length of boundary line needs to be laid for each working area boundary, the cost of maintaining the working area is greatly increased.

[0003] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, readable medium, and electronic device for generating a moving path to solve the problems in the related art that the stability of generating the moving path is poor, the recognition of the boundary of the working area is not accurate enough, and the cost of maintaining the boundary of the working area is relatively high.

[0005] Other features and advantages of this application will become apparent through the following detailed description, or will be partially learned through the practice of this application.

[0006] According to one aspect of the embodiments of this application, a method for generating a moving path is provided, including:

[0007] Obtaining a boundary image when the robot moves, where the boundary image includes at least one boundary, and each boundary includes a plurality of boundary pixel points;

[0008] Constructing a first pixel matrix with each boundary pixel point as the center point to obtain a plurality of first pixel matrices corresponding to each boundary;

[0009] Among all the boundary pixel points on any boundary, when there are at most two boundary pixel points in the first pixel matrix corresponding to any boundary pixel point, determine that the boundary is the target boundary;

[0010] Generate a movement path according to the multiple boundary pixel points corresponding to the target boundary.

[0011] According to one aspect of the embodiments of the present application, there is provided a device for generating a movement path, including:

[0012] A boundary image acquisition module, configured to acquire a boundary image when the robot moves, where the boundary image includes at least one boundary, and each boundary includes multiple boundary pixel points;

[0013] A pixel matrix construction module, configured to construct a first pixel matrix with each boundary pixel point as the center point, and obtain multiple first pixel matrices corresponding to each boundary, where the pixel points surrounding the center point and adjacent to the center point in the first pixel matrix are the first surrounding point set;

[0014] A target boundary determination module, configured to determine that the boundary is the target boundary when there are at most two boundary pixel points in the first pixel matrix corresponding to any boundary pixel point among all the boundary pixel points on any boundary;

[0015] A movement path generation module, configured to generate a movement path according to the multiple boundary pixel points corresponding to the target boundary.

[0016] According to one aspect of the embodiments of the present application, there is provided an electronic device, which includes: a processor; and a memory for storing executable instructions of the processor; wherein, the processor executes the executable instructions to enable the electronic device to execute the method for generating a movement path in the above technical solution.

[0017] According to one aspect of the embodiments of the present application, there is provided a computer program product or a computer program, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to enable the computer device to execute the method for generating a movement path in the above technical solution.

[0018] In the technical solution provided by the embodiments of the present application, by constructing a first pixel matrix centered on a boundary pixel point in the boundary image, it is possible to determine whether the center of the first pixel matrix is a boundary point in a closed boundary according to the number of boundary pixel points in the first pixel matrix. Among all the boundary pixel points in any boundary, when the first pixel matrix corresponding to any boundary pixel point includes at most two boundary pixel points, the center of the first pixel matrix is actually a boundary pixel point in an open boundary. Thus, the open boundary is used as the target boundary, and then a movement path is generated based on the boundary pixel points on the identified target boundary. Through the technical solution of the present application, the open boundary in the boundary image can be accurately identified, and the open boundary is usually the boundary in the boundary image obtained during the movement of the robot. In this way, in the case where no boundary induction line is arranged at the boundary of the working area, the robot can also accurately identify the boundary between the working area and the non-working area and perform edge following work based on the identified boundary. This not only reduces the dependence on hardware devices for boundary recognition, improves the accuracy of boundary recognition, but also enables the robot to perform normal edge following work even when no boundary induction line is arranged at the boundary, improves the stability of the generation of the robot movement path, and reduces the wiring cost required for the traditional robot to work along the boundary.

[0019] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application and used together with the specification to explain the principles of the present application. Obviously, the accompanying drawings in the following description are only some embodiments of the present application, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0021] Figure 1 Schematically shows an exemplary system architecture block diagram applying the technical solution of the present application.

[0022] Figure 2 Schematically shows a flowchart of a method for generating a movement path provided by an embodiment of the present application.

[0023] Figure 3 Schematically shows a flowchart of determining boundary pixel points provided by an embodiment of the present application.

[0024] Figure 4 Schematically shows an image obtained by segmenting a region of an original image provided by an embodiment of the present application.

[0025] Figure 5Schematically shows a schematic diagram of a blurring process provided by an embodiment of the present application.

[0026] Figure 6 Schematically shows a blurred image provided by an embodiment of the present application.

[0027] Figure 7 Schematically shows a preprocessed image provided by an embodiment of the present application.

[0028] Figure 8 Schematically shows a boundary image provided by an embodiment of the present application.

[0029] Figure 9 Schematically shows an image of a target boundary provided by an embodiment of the present application.

[0030] Figure 10 Schematically shows a structural block diagram of a generating device for a moving path provided by an embodiment of the present application.

[0031] Figure 11 Schematically shows a computer system structural block diagram of an electronic device suitable for implementing the embodiments of the present application. Detailed implementation manners

[0032] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more complete and comprehensive, and will fully convey the concept of the example embodiments to those skilled in the art.

[0033] In addition, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present application. However, those skilled in the art will realize that the technical solutions of the present application can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present application.

[0034] The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0035] The flowcharts shown in the accompanying drawings are merely illustrative and not necessarily include all the content and operations / steps, nor are they necessarily executed in the described order. For example, some operations / steps can be decomposed, while some operations / steps can be combined or partially combined. Therefore, the actual execution order may change according to the actual situation.

[0036] Figure 1 Schematically shows an exemplary system architecture block diagram to which the technical solution of the present application is applied.

[0037] As Figure 1 shown, the system architecture 100 may include a terminal device 110, a network 120, and a server 130. The terminal device 110 may include a smart phone, a tablet computer, a laptop computer, a smart voice interaction device, a smart home appliance, a vehicle-mounted terminal, a robot, and so on. The server 130 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The network 120 may be a communication medium of various connection types capable of providing a communication link between the terminal device 110 and the server 130. For example, it may be a wired communication link or a wireless communication link.

[0038] According to the implementation requirements, the system architecture in the embodiments of the present application may have any number of terminal devices, networks, and servers. For example, the server 130 may be a server group composed of multiple server devices. In addition, the technical solution provided by the embodiments of the present application may be applied to the terminal device 110, or may be applied to the server 130, or may be jointly implemented by the terminal device 110 and the server 130. The present application does not make special limitations on this.

[0039] For example, the method for generating a movement path provided by the embodiments of the present application is implemented by the server 130. The server 130 acquires a boundary image when the robot moves, where the boundary image includes at least one boundary, and the boundary includes a plurality of boundary pixel points. The boundary image may be acquired by the terminal device 110 (such as a robot) through shooting, and then transmitted to the server 130 through the network 120. The server 130 constructs a first pixel matrix with each boundary pixel point as the center point, and obtains a plurality of first pixel matrices corresponding to each boundary. When, in the plurality of first pixel matrices corresponding to the same boundary, any first pixel matrix includes at most two boundary pixel points, the server 130 determines that this boundary is the target boundary. Finally, the server 130 generates a movement path according to the plurality of boundary pixel points corresponding to the target boundary. After generating the movement path, the server 130 may send the movement path to the terminal device 110 through the network 120. The terminal device 110 may display the movement path to the user or move according to the movement path.

[0040] The following will make a detailed description of the method for generating a moving path provided by the present application in combination with specific embodiments.

[0041] Figure 2 Schematically shows a flowchart of a method for generating a moving path provided by an embodiment of the present application. This method can be implemented by a server, such as Figure 1 the server 130 shown. This method can also be implemented by a terminal device, such as Figure 1 the terminal device 110 shown. As Figure 2 shown, the method for generating a moving path provided by the embodiment of the present application includes steps 210 to 240, specifically as follows:

[0042] Step 210: Obtain a boundary image when the robot moves. Among them, the boundary image includes at least one boundary, and each boundary includes a plurality of boundary pixel points.

[0043] Specifically, a boundary refers to the dividing line between different regions in an image, and the pixel points that make up the dividing line in the image are called boundary pixel points. For example, there is an object in the image, and there is a boundary between the image region representing the object and the image region outside the object. In this case, the boundary is actually the contour line of the object. In the embodiment of the present application, a boundary image refers to an image including at least one boundary. The boundary image when the robot moves can be obtained by a camera device carried by the robot itself, or can be obtained by a camera device set at a specific position.

[0044] In an embodiment of the present application, a boundary refers to the dividing line between regions with different attributes in an image. The attribute of an image region is equivalent to the type of the image region. For example, there is a boundary between a region with a road attribute and a region with a grassland attribute in the image, and there is a boundary between the region where a car is located in the image and its surrounding environment region. It can be seen from this that the attribute of an image region is obtained by classifying the image region, and the classification of the image region can be achieved through image segmentation. Image segmentation refers to the process of dividing an image into several image regions with similar properties, and the divided image regions do not intersect each other, so there is a boundary between different image regions. Therefore, the boundary can be determined by performing image segmentation on the unprocessed original image, and then the boundary image can be obtained. Image segmentation can be processing methods such as image binarization, semantic segmentation, k-means clustering segmentation, etc., which are not limited here.

[0045] In one embodiment of the present application, the boundary refers to the dividing line between the working area and the non-working area of the robot. The process of obtaining the boundary image includes: obtaining the original image when the robot moves, where the original image includes the working area and the non-working area of the robot; determining multiple boundary pixel points between the working area and the non-working area in the original image; determining at least one boundary based on the multiple boundary pixel points; and generating the boundary image when the robot moves based on the at least one boundary.

[0046] Specifically, the original image when the robot moves refers to the image captured in real time during the movement of the robot. This original image includes the working area and the non-working area of the robot. The working area refers to the area covered by the movement range (i.e., the operation range) of the robot, and the non-working area is the area other than the working area. For example, for a lawn mowing robot, the lawn is the working area, and the areas outside the lawn are non-working areas, such as paths, flower beds, and pet houses. By distinguishing between the working area and the non-working area in the original image, the pixel points at the junction of the working area and the non-working area are the boundary pixel points. At least one boundary can be determined from the multiple boundary pixel points, and thus the boundary image when the robot moves can be obtained.

[0047] In one embodiment of the present application, as Figure 3 shown, the process of determining the boundary pixel points includes steps 310 to 330, specifically:

[0048] Step 310: Preprocess the original image to obtain a preprocessed image. Among them, in the preprocessed image, the pixel points in the working area and the pixel points in the non-working area have different pixel values.

[0049] Specifically, preprocessing the original image is to distinguish between the working area and the non-working area in the original image. Generally, different pixel values can be set for the pixel points in the working area and the non-working area to distinguish them. Then, in the preprocessed image, the pixel values of the pixel points belonging to the working area and the pixel values of the pixel points belonging to the non-working area are different.

[0050] In one embodiment of the present application, the preprocessing process of the original image includes: traversing the pixel points in the original image and obtaining the similarity between the pixel points in the original image; dividing the original image into multiple segmentation regions according to the similarity, where the segmentation regions include the working area and the non-working area; performing blurring processing on each pixel point in the segmentation regions to obtain a blurred region including multiple blurred pixel points; and performing threshold processing on each blurred pixel point in the blurred region to obtain the preprocessed image.

[0051] In the embodiments of the present application, the original image is segmented based on the similarity between pixel points, and the original image is divided into multiple segmentation regions. There are two types of these multiple segmentation regions: working regions and non-working regions. It can be understood that the technical solution of the present application processes the boundary image when the robot moves. Therefore, the multiple segmentation regions include at least one working region and at least one non-working region. When all the multiple segmentation regions belong to the working region or all belong to the non-working region, it indicates that there is no boundary in the original image, or in other words, the robot has not moved into the boundary range, and at this time, there is no need to process the boundary image.

[0052] When segmenting the original image, first calculate the similarity between each pixel point. When the similarity between two pixel points is greater than the preset threshold, it indicates that there is a certain similarity between the two pixel points and they belong to the same segmentation region (which may be a working region or a non-working region). When the similarity between two pixel points is less than the preset threshold, it is considered that they do not belong to the same segmentation region. By traversing the pixel points in the original image in this way, the segmentation process of the original image is realized, and multiple segmentation regions are obtained. Among the multiple segmentation regions, the similarity between pixel points belonging to the same segmentation region will be greater than the similarity between pixel points in different segmentation regions.

[0053] In an embodiment of the present application, the process of segmenting the original image can also be as follows: First, randomly select a pixel point from the pixel points of the original image as the center of the segmentation region, and then calculate the similarity between each pixel point in the original image and the center of the segmentation region. Based on this similarity, select the next center of the segmentation region. The selection method is: the lower the similarity between a pixel point and the center of the segmentation region, the greater the probability that the pixel point is selected as the next center of the segmentation region. Then repeat the steps of calculating the similarity and determining the next center of the segmentation region according to the similarity until the number of selected centers of the segmentation region no longer changes. A center of the segmentation region and multiple pixel points whose similarity to it is greater than the threshold form a segmentation region, thereby dividing the original image into multiple segmentation regions.

[0054] In an embodiment of the present application, after dividing the original image into multiple segmentation regions, in order to more efficiently distinguish between the working region and the non-working region, further binarization processing can be performed, that is, set the pixel points in the working region to the same first pixel value, and set the pixel points in the non-working region to the same second pixel value. Exemplarily, as Figure 4 shown, set the pixel values of the pixel points in the working region to 255, as Figure 4 shown in the white region, and set the pixel values of the pixel points in the non-working region to 0, as Figure 4 shown in the black region.

[0055] After distinguishing the working area from the non-working area in the original image, blur processing is performed on each pixel point in the segmented area to obtain a blurred area including a plurality of blurred pixel points. The blur processing is to eliminate noise in the image to prevent subsequent misidentification of pixel points that are not on the boundary as boundary pixel points, exclude the interference of non-boundary pixel points, and at the same time reduce the computational amount of non-boundary pixel points, thereby improving the accuracy and efficiency of boundary recognition.

[0056] In an embodiment of the present application, the process of blur processing includes: constructing a plurality of third pixel matrices with each pixel point in the segmented area as the center point; updating the pixel value of the center point in the third pixel matrix according to the pixel values of the pixel points adjacent to the center point in the third pixel matrix; taking the center point with the updated pixel value as the blurred pixel point, and obtaining a blurred area based on each blurred pixel point.

[0057] Specifically, a pixel matrix is constructed with the pixel point in the segmented area as the center. For the convenience of distinction, this pixel matrix is denoted as the third pixel matrix. At the same time, in the embodiment of the present application, the center point of the third pixel matrix can be denoted as the third center point. In the third pixel matrix, the pixel points other than the third center point form a third surrounding point set. The pixel points in this third surrounding point set are the pixel points that surround the third center point and are adjacent to the third center point. The pixel points in this third surrounding point set are also called third surrounding points. The pixel points adjacent to the third center point refer to the pixel points within a specific neighborhood of the third center point. The specific neighborhood can be a four-neighborhood, an eight-neighborhood, or a D-neighborhood, etc. In a nine-grid centered on a pixel point, the pixel points in the four directions of up, down, left, and right of the central pixel point are the four-neighborhood pixel points of the central pixel point. The pixel points on the diagonal of the nine-grid are the D-neighborhood pixel points of the central pixel point. The sum of the four-neighborhood pixel points and the D-neighborhood pixel points is the eight-neighborhood pixel points of the central pixel point, that is, the pixel points in the nine-grid except the central pixel point are the eight-neighborhood pixel points of the central pixel point. Exemplarily, the form of the third pixel matrix is a nine-grid form (that is, the third pixel matrix is a 3*3 matrix). The center of this nine-grid is the third center point, and the remaining 8 pixel points around the center form a third surrounding point set.

[0058] Updating the pixel value of the corresponding third center point according to the pixel values of the third surrounding point set in the third pixel matrix is to perform blur processing on the center point of the third pixel matrix. The center point of the third pixel matrix after blur processing is denoted as the blurred pixel point. Then, by traversing each pixel point in the segmented area and performing blur processing on each pixel point, an image of a blurred area composed of a plurality of blurred pixel points is obtained.

[0059] In one embodiment of the present application, updating the pixel value of the center point of the third pixel matrix may be to first perform statistical processing on the pixel values of each pixel point in the third set of surrounding points, and then update the pixel value of the corresponding center point of the third pixel matrix based on the statistical pixel values. For example, the average pixel value, median pixel value, maximum pixel value, etc. of each pixel point in the third set of surrounding points are used as the pixel value after updating the center point of the third pixel matrix. Exemplarily, as Figure 5 shown, Figure 5 schematically shows a schematic diagram of the blurring process provided by one embodiment of the present application, Figure 5 The pixel value after updating the center point A of the third pixel matrix shown is: (0 + 0 + 255 + 255 + 255 + 255 + 255 + 255) / 8 = 191.25. For the convenience of calculation, the integer value 191 can be taken, and this pixel value appears as gray. For Figure 4 the binary image shown, the blurred image obtained after blurring processing is as Figure 6 shown. From the comparison between Figure 4 and Figure 6 , it can be seen that the blurring processing eliminates the noise in the image and makes the image smoother.

[0060] In one embodiment of the present application, blurring the segmented region to obtain a blurred region, and multiple blurred regions constitute a blurred image. After obtaining the blurred image, continue to perform threshold processing on each blurred pixel point to obtain the final required preprocessed image.

[0061] In one embodiment of the present application, the process of performing threshold processing on blurred pixel points specifically includes: if the pixel value of a blurred pixel point is greater than a preset pixel threshold, set the pixel value of the blurred pixel point to a first pixel value; if the pixel value of a blurred pixel point is less than the preset pixel threshold, set the pixel value of the blurred pixel point to a second pixel value; traverse each blurred pixel point in the blurred region, and after completing the setting of the pixel values of the blurred pixel points, obtain the preprocessed image.

[0062] Specifically, performing threshold processing on blurred pixel points is to further update the pixel values of blurred pixel points according to a preset pixel threshold. When the pixel value is greater than the preset pixel threshold, set the pixel value of the blurred pixel point to the first pixel value; otherwise, set the pixel value of the blurred pixel point to the second pixel value. Traverse each blurred pixel point. After all blurred pixel points have their pixel values updated, the preprocessed image is obtained. Exemplarily, for Figure 6 the blurred image shown, the pixel points in the blurred image whose pixel values are greater than the preset pixel threshold (such as pixel values greater than 127) are reset to a pixel value of 0 (appearing as black); the pixel points in the blurred image whose pixel values are less than or equal to the preset pixel threshold are reset to a pixel value of 255 (appearing as white), and the obtained preprocessed image is asFigure 7 as shown

[0063] Continue to refer to Figure 3 , step 320: Construct multiple second pixel matrices with each pixel point in the preprocessed image as the center point.

[0064] Continue to refer to Figure 3 , step 330: Determine multiple boundary pixel points between the working area and the non-working area according to the pixel values of the pixel points in each second pixel matrix.

[0065] Specifically, after obtaining the preprocessed image, construct a second pixel matrix in the preprocessed image, and then determine the boundary pixel points according to the pixel points in the second pixel matrix.

[0066] The second pixel matrix is a pixel matrix centered on each pixel point in the preprocessed image. For the convenience of distinction, in the embodiments of the present application, the center point of the second pixel matrix can be denoted as the second center point. The pixel points surrounding the second center point in the second pixel matrix form a second surrounding point set, that is, the pixel points adjacent to the center point of the second pixel matrix are the pixel points in the second surrounding point set, which can also be called the second surrounding points. In the preprocessed image, the pixel points in the working area have the same first pixel value, and the pixel points in the non-working area have the same second pixel value. The boundary pixel points are located at the junction between the working area and the non-working area. Therefore, there may be both working area pixel points and non-working area pixel points around the boundary pixel points, that is, there should be two different pixel values around the boundary pixel points. Therefore, it is possible to determine whether the corresponding second center point is a boundary pixel point through the pixel values of the second surrounding point set in the second pixel matrix, where the pixel values of the second surrounding point set refer to the pixel values of each pixel point in the second surrounding point set. Then, when both the first pixel value and the second pixel value exist in the pixel values of the second surrounding point set, it means that there are both working area pixel points and non-working area pixel points around the corresponding second center point. Therefore, it can be considered that the second center point is a boundary pixel point. Obviously, when the pixel values of the second surrounding point set are only one of the first pixel value and the second pixel value, the corresponding second center point is not a boundary pixel point.

[0067] In this way, by traversing the pixel points in the preprocessed image and judging the second pixel matrix corresponding to each pixel point, multiple boundary pixel points in the preprocessed image can be determined. According to these multiple boundary pixel points, at least one boundary can be determined, and then the boundary image can be obtained. Exemplarily, for Figure 7 the preprocessed image shown Figure 8 as shown, after identifying the boundary pixel points, the obtained boundary image is as

[0068] Continue to refer to Figure 2, Step 220: Construct a first pixel matrix with each boundary pixel point as the center point to obtain multiple first pixel matrices corresponding to each boundary.

[0069] Specifically, in the boundary image, a first pixel matrix is constructed with the boundary pixel point as the center. For the convenience of distinction, in the embodiments of the present application, this boundary pixel point can also be referred to as the first center point. The pixel points in the first pixel matrix that enclose the first center point and are adjacent to the first center point form a first surrounding point set, and the pixel points in the first surrounding point set are also referred to as first surrounding points. The pixel points adjacent to the first center point refer to the pixel points within a specific neighborhood of the first center point, such as four-neighborhood pixel points, eight-neighborhood pixel points, D-neighborhood pixel points, etc.

[0070] Continue to refer to Figure 2 , Step 230: Among all the boundary pixel points in any boundary, when the first pixel matrix corresponding to any boundary pixel point includes at most two boundary pixel points, determine the boundary as the target boundary.

[0071] Specifically, there may be multiple boundaries in the boundary image, but not every boundary belongs to the target boundary that finally needs to generate a movement path. In the embodiments of the present application, the boundary image is an image obtained when the robot runs within the boundary range between the working area and the non-working area. The boundary between the working area and the non-working area should be a non-closed boundary. Therefore, the target boundary in the present application refers to a non-closed boundary.

[0072] If a boundary is a closed boundary, then for any boundary pixel point on the boundary, the boundary pixel point has at least one connected out-point and at least one in-point. Plus the boundary pixel point itself, there are at least three boundary pixel points in the first pixel matrix centered on it, that is, the number of boundary pixel points in the corresponding first surrounding point set is greater than 1. The understanding of the in-point and out-point is as follows: Suppose there are 3 pixel points on the boundary: A, B, and C. A is connected to B, and B is connected to C. Then for point B, A is the in-point and C is the out-point. Of course, if it is regarded as C connected to B and B connected to A, then for point B, C is the in-point and A is the out-point. Generally, most closed boundaries are the contour lines of obstacles, rather than the boundary between the working area and the non-working area.

[0073] In an embodiment of the present application, if a boundary is a closed boundary, then in the first pixel matrix with the boundary pixel point as the first center point, the number of boundary pixel points in the corresponding first surrounding point set is greater than 1. Plus the first center point itself, the number of pixel points in the first pixel matrix should be greater than 2. Therefore, when it is determined that the number of boundary pixel points in any first pixel matrix among the first pixel matrices corresponding to the respective boundary pixel points of a boundary is greater than 2, it can be considered that the boundary is a closed boundary.

[0074] The characteristics of a non-closed boundary can be inferred from those of a closed boundary. That is, if there is a boundary pixel on the boundary, and the number of boundary pixels in the first surrounding point set of the first pixel matrix centered on this boundary pixel is less than or equal to 1, then this boundary is a non-closed boundary. Therefore, when the first surrounding point set of the first pixel matrix includes at most one boundary pixel, it is equivalent to having no boundary pixel or only one boundary pixel around the center point, or having at most two boundary pixels within the first pixel matrix, indicating that after this center point, there are no other boundary pixels connected on the boundary, which also means that this boundary is not a closed boundary, so this boundary is the target boundary.

[0075] Exemplarily, for Figure 8 the shown boundary image, at the boundary pixel A of boundary 1, this boundary pixel A is only connected to the previous boundary pixel A'. Then, in the first pixel matrix constructed according to the boundary pixels in boundary 1, the number of boundary pixels in the first surrounding point set corresponding to this boundary pixel A is 1, thus determining that boundary 1 is a non-closed boundary, that is, boundary 1 is the target boundary. At any boundary pixel of boundary 2, taking the boundary pixel B as an example, this boundary pixel B is connected to other boundary pixels before and after, that is, in the first surrounding point set of the first pixel matrix corresponding to this boundary pixel B, the number of boundary pixels is at least 2; then in the first pixel matrix constructed according to each boundary pixel in boundary 2, the number of boundary pixels in any first surrounding point set is greater than 1, thus determining that boundary 2 is a closed boundary, so boundary 2 is not the target boundary. At this time, boundary 2 can be deleted from the boundary image to obtain an image such as Figure 9 shown that only contains the target boundary.

[0076] In an embodiment of the present application, after identifying multiple boundary pixels belonging to the target boundary, calculate the length of the boundary formed by each boundary pixel, and select the longest boundary as the target boundary. This is to ensure that the determined target boundary is a complete boundary rather than a segment of the boundary.

[0077] Continue to refer to Figure 2 Step 240: Generate a movement path according to the multiple boundary pixels corresponding to the target boundary.

[0078] Specifically, after determining the target boundary, a movement path can be generated according to the target boundary. This movement path is also called the edge-following path of the robot. Each boundary pixel on the target boundary can be regarded as each point on the robot's movement path, and the movement path can be generated based on the connection lines of each boundary pixel.

[0079] In one embodiment of the present application, the process of generating a movement path includes: obtaining depth information corresponding to each boundary pixel point in the target boundary; determining the spatial coordinates of each boundary pixel point in the target boundary in the robot coordinate system according to the depth information corresponding to each boundary pixel point in the target boundary and the pixel coordinates of each boundary pixel point in the boundary image; and generating a movement path according to the spatial coordinates of each boundary pixel point in the target boundary.

[0080] Specifically, the depth information of a boundary pixel point refers to the depth value corresponding to this boundary pixel point in the depth image obtained by a depth camera. The depth value of a pixel point represents the distance between the actual object corresponding to this pixel point and the depth camera. Generally, the boundary image is obtained by an RGB camera, and the depth image is obtained by a depth camera. In order to keep the pixel coordinates and depth information of each pixel point consistent, it is necessary to first align and calibrate the RGB camera and the depth camera. A common calibration method is the checkerboard calibration method. The RGB camera and the depth camera after calibration and alignment can be simply referred to as an RGB-D camera. Through the RGB-D camera, the pixel coordinates (u, v) and depth information d of the boundary pixel point can be obtained synchronously.

[0081] After obtaining the pixel coordinates (u, v) and depth information d of the boundary pixel point, through coordinate system conversion, the spatial coordinates of the boundary pixel point in the robot coordinate system can be determined. This coordinate system conversion includes the conversion between the pixel coordinate system and the camera coordinate system, and the conversion between the camera coordinate system and the robot coordinate system. First, the three-dimensional coordinates of the boundary pixel point in the camera coordinate system are obtained through the conversion between the pixel coordinate system and the camera coordinate system. Denote the three-dimensional coordinates of the boundary pixel point in the camera coordinate system as (x, y, z). Then, the calculation method of the three-dimensional coordinates (x, y, z) is as follows:

[0082]

[0083] where f x and f y are the focal lengths of the depth camera, and c x and c y are the principal points of the depth camera, and both are camera internal parameters that can be determined through camera calibration.

[0084] Then, based on the conversion relationship between the camera coordinate system and the robot coordinate system, the three-dimensional coordinates of the boundary pixel points are converted into the spatial coordinates in the robot coordinate system. Finally, according to the spatial coordinates of the boundary pixel points, each boundary pixel point is sorted to generate the movement path of the robot. Of course, it is also possible to sort each boundary pixel point first and then convert the three-dimensional coordinates into spatial coordinates. When sorting the boundary pixel points, they can be sorted according to the coordinates in the running direction of the robot. Exemplarily, in the camera coordinate system, the x-axis is in the front of the camera (also in the front of the robot), and the y-axis is in the running direction of the robot. Then, each boundary pixel point is sorted in ascending order according to the y-axis coordinate in the three-dimensional coordinates. Finally, the sorted boundary pixel points are mapped into the robot coordinate system to obtain the edge-following path of the robot.

[0085] In the technical solution provided by the embodiment of the present application, by constructing a first pixel matrix centered on the boundary pixel points in the boundary image, the center of the first pixel matrix can be determined whether it is a boundary point in the closed boundary according to the number of boundary pixel points in the first surrounding points of the first pixel matrix. When there is at most one boundary pixel point of the same boundary in the first surrounding points of the first pixel matrix, the center of the first pixel matrix is actually a boundary pixel point in the non-closed boundary. Thus, the non-closed boundary is used as the target boundary, and then the movement path is generated according to the boundary pixel points on the identified target boundary; through the technical solution of the present application, the non-closed boundary in the boundary image can be accurately identified, and the non-closed boundary is usually the boundary in the boundary image obtained during the movement of the robot. In this way, when there is no boundary induction line arranged at the boundary of the working area of the robot, the boundary between the working area and the non-working area can also be accurately identified, and the robot can perform edge-following work based on the identified boundary. This not only reduces the dependence on hardware devices for boundary recognition, improves the accuracy of boundary recognition, but also enables the robot to perform normal edge-following work when there is no boundary induction line at the boundary, improves the stability of the generation of the robot movement path, and also reduces the wiring cost required for the traditional robot to work along the boundary.

[0086] It should be noted that although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that these steps must be executed in this specific order, or that all the steps shown must be executed to achieve the desired result. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution, etc.

[0087] The following introduces the device embodiment of the present application, which can be used to execute the method for generating the movement path in the above embodiments of the present application. Figure 10 Schematically shows the structural block diagram of the device for generating the movement path provided by the embodiment of the present application. As Figure 10As shown in the figure, the generating device for the movement path includes:

[0088] A boundary image acquisition module 1010, configured to acquire a boundary image when the robot moves, where the boundary image includes at least one boundary, and each boundary includes a plurality of boundary pixel points;

[0089] A pixel matrix construction module 1020, configured to construct a first pixel matrix with each boundary pixel point as the center point, and obtain a plurality of first pixel matrices corresponding to each boundary, where the pixel points surrounding the center point and adjacent to the center point in the first pixel matrix are the first surrounding point set;

[0090] A target boundary determination module 1030, configured to determine the boundary as the target boundary when at most two boundary pixel points are included in the first pixel matrix corresponding to any boundary pixel point among all boundary pixel points in any boundary;

[0091] A movement path generation module 1040, configured to generate a movement path according to the plurality of boundary pixel points corresponding to the target boundary.

[0092] In an embodiment of the present application, the boundary image acquisition module 1010 includes:

[0093] An original image acquisition unit, configured to acquire an original image when the robot moves, where the original image includes the working area and the non-working area of the robot;

[0094] A boundary pixel point determination unit, configured to determine a plurality of boundary pixel points between the working area and the non-working area in the original image;

[0095] A boundary determination unit, configured to determine at least one boundary according to the plurality of boundary pixel points;

[0096] A boundary image generation unit, configured to generate a boundary image when the robot moves according to the at least one boundary.

[0097] In an embodiment of the present application, the boundary pixel point determination unit includes:

[0098] A preprocessing unit, configured to preprocess the original image to obtain a preprocessed image, where in the preprocessed image, the pixel points of the working area and the pixel points of the non-working area have different pixel values;

[0099] A second pixel matrix construction unit, configured to construct a plurality of second pixel matrices with each pixel point in the preprocessed image as the center point;

[0100] A boundary pixel determination subunit, configured to determine a plurality of boundary pixels between the working area and the non-working area according to the pixel values of the pixels in each of the second pixel matrices.

[0101] In an embodiment of the present application, in the preprocessed image, the pixels in the working area have the same first pixel value, and the pixels in the non-working area have the same second pixel value;

[0102] The boundary pixel determination subunit is specifically configured to: in the second pixel matrix, if both the first pixel value and the second pixel value exist in the pixel values of the pixels adjacent to the center point of the second pixel matrix, determine the center point of the second pixel matrix as the boundary pixel between the working area and the non-working area.

[0103] In an embodiment of the present application, the preprocessing unit includes:

[0104] A similarity determination subunit, configured to traverse the pixels in the original image and obtain the similarity between the pixels in the original image;

[0105] An image segmentation subunit, configured to divide the original image into a plurality of segmentation regions according to the similarity, and the segmentation regions include a working area and a non-working area;

[0106] A blurring processing subunit, configured to perform blurring processing on each pixel in the segmentation region to obtain a blurred region including a plurality of blurred pixels;

[0107] A threshold processing subunit, configured to perform threshold processing on each blurred pixel in the blurred region to obtain a preprocessed image.

[0108] In an embodiment of the present application, the threshold processing subunit is specifically configured to:

[0109] If the pixel value of the blurred pixel is greater than a preset pixel threshold, set the pixel value of the blurred pixel to the first pixel value;

[0110] If the pixel value of the blurred pixel is less than the preset pixel threshold, set the pixel value of the blurred pixel to the second pixel value;

[0111] Traverse each blurred pixel in the blurred region, and after completing the setting of the pixel values of the blurred pixels, obtain the preprocessed image.

[0112] In an embodiment of the present application, the blurring processing subunit is specifically configured to:

[0113] Construct a plurality of third pixel matrices with each pixel in the segmentation region as the center point;

[0114] Update the pixel value of the center point in the third pixel matrix according to the pixel values of the respective pixel points adjacent to the center point in the third pixel matrix;

[0115] Use the center point with the updated pixel value as a blurred pixel point, and obtain a blurred area based on the respective blurred pixel points.

[0116] In an embodiment of the present application, the movement path generation module 1040 includes:

[0117] A depth information acquisition unit configured to acquire the depth information corresponding to each boundary pixel point in the target boundary;

[0118] A coordinate conversion unit configured to determine the spatial coordinates of each boundary pixel point in the target boundary in the robot coordinate system according to the depth information corresponding to each boundary pixel point in the target boundary and the pixel coordinates of each boundary pixel point in the boundary image;

[0119] A movement path generation unit configured to generate a movement path according to the spatial coordinates corresponding to each boundary pixel point in the target boundary.

[0120] The specific details of the movement path generation device provided in the embodiments of the present application have been described in detail in the corresponding method embodiments, and will not be elaborated here.

[0121] Figure 11 Schematically shows a computer system block diagram of an electronic device for implementing the embodiments of the present application.

[0122] It should be noted that Figure 11 The computer system 1100 of the shown electronic device is only an example, and should not bring any limitations to the functions and usage scopes of the embodiments of the present application.

[0123] As Figure 11 shown, the computer system 1100 includes a central processing unit 1101 (Central Processing Unit, CPU), which can perform various appropriate actions and processes according to the program stored in the read-only memory 1102 (Read-Only Memory, ROM) or the program loaded from the storage section 1108 into the random access memory 1103 (Random Access Memory, RAM). In the random access memory 1103, various programs and data required for system operation are also stored. The central processing unit 1101, the read-only memory 1102, and the random access memory 1103 are connected to each other through a bus 1104. The input / output interface 1105 (Input / Output interface, i.e., I / O interface) is also connected to the bus 1104.

[0124] The following components are connected to the input / output interface 1105: an input section 1106 including a keyboard, a mouse, etc.; an output section 1107 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1108 including a hard disk, etc.; and a communication section 1109 including a network interface card such as a local area network card, a modem, etc. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the input / output interface 1105 as needed. A removable medium 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1110 as needed so that a computer program read from it can be installed into the storage section 1108 as needed.

[0125] Specifically, according to an embodiment of the present application, the processes described in each method flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product that includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 1109, and / or installed from the removable medium 1111. When the computer program is executed by the central processing unit 1101, various functions defined in the system of the present application are executed.

[0126] It should be noted that the computer-readable medium shown in the embodiments of the present application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present application, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0127] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the above module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0128] It should be noted that although several modules or units of a device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more of the above-described modules or units can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0129] From the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (such as a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.

[0130] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present application. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include known common knowledge or conventional technical means in the technical field not disclosed in the present application.

[0131] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

Claims

1. A method for generating a moving path, characterized in that, Including: Obtain a boundary image when the robot moves, where the boundary image includes at least one boundary, and each boundary includes a plurality of boundary pixel points; Construct a first pixel matrix with each of the boundary pixel points as the center point to obtain a plurality of first pixel matrices corresponding to each boundary; wherein, the first pixel matrix includes the center point and the pixel points within the eight-neighborhood of the center point; Among all the boundary pixel points in any boundary, when at most two boundary pixel points are included in the first pixel matrix corresponding to any boundary pixel point, determine that the boundary is a target boundary; Generate a movement path according to the plurality of boundary pixel points corresponding to the target boundary.

2. The method for generating a moving path according to claim 1, characterized in that, The obtaining the boundary image when the robot moves includes: Obtain an original image when the robot moves, where the original image includes the working area and the non-working area of the robot; Determine a plurality of boundary pixel points between the working area and the non-working area in the original image; Determine at least one boundary according to the plurality of boundary pixel points; Generate the boundary image when the robot moves according to the at least one boundary.

3. The method for generating a moving path according to claim 2, characterized in that, The determining a plurality of boundary pixel points between the working area and the non-working area in the original image includes: Preprocess the original image to obtain a preprocessed image, where in the preprocessed image, the pixel points in the working area and the pixel points in the non-working area have different pixel values; Construct a plurality of second pixel matrices with each pixel point in the preprocessed image as the center point; Determine a plurality of boundary pixel points between the working area and the non-working area according to the pixel values of the pixel points in each second pixel matrix.

4. The method for generating a moving path according to claim 3, characterized in that, In the preprocessed image, the pixel points in the working area have the same first pixel value, and the pixel points in the non-working area have the same second pixel value; The determining a plurality of boundary pixel points between the working area and the non-working area according to the pixel values of the pixel points in each second pixel matrix includes: In the second pixel matrix, if the pixel values of the pixel points adjacent to the center point of the second pixel matrix simultaneously have the first pixel value and the second pixel value, determine that the center point of the second pixel matrix is the boundary pixel point between the working area and the non-working area.

5. The method for generating a moving path according to claim 3, characterized in that, The preprocessing the original image to obtain a preprocessed image includes: Traverse the pixel points in the original image and obtain the similarity between the pixel points in the original image; Divide the original image into a plurality of segmentation regions according to the similarity, and the segmentation regions include a working area and a non-working area; Blur each pixel point in the segmentation region to obtain a blurred region including a plurality of blurred pixel points; Perform threshold processing on each blurred pixel point in the blurred region to obtain a preprocessed image.

6. The method for generating a moving path according to claim 5, characterized in that, The performing threshold processing on each blurred pixel point in the blurred region to obtain a preprocessed image includes: If the pixel value of the blurred pixel point is greater than a preset pixel threshold, set the pixel value of the blurred pixel point to the first pixel value; If the pixel value of the blurred pixel point is less than a preset pixel threshold, set the pixel value of the blurred pixel point to a second pixel value; Traverse each blurred pixel point in the blurred area, and after completing the setting of the pixel values of the blurred pixel points, obtain the preprocessed image.

7. The method for generating a moving path according to claim 5, characterized in that, The blurring of each pixel point in the segmentation area to obtain a blurred area including a plurality of blurred pixel points includes: Construct a plurality of third pixel matrices with each pixel point in the segmentation area as the center point; Update the pixel value of the center point in the third pixel matrix according to the pixel values of the pixel points adjacent to the center point in the third pixel matrix; Use the center point with the updated pixel value as the blurred pixel point, and obtain the blurred area based on each blurred pixel point.

8. The method for generating a movement path according to any one of claims 1-7, characterized in that, The generation of the movement path according to the multiple boundary pixel points corresponding to the target boundary includes: Obtain the depth information corresponding to each boundary pixel point in the target boundary; According to the depth information corresponding to each boundary pixel point in the target boundary and the pixel coordinates of each boundary pixel point in the boundary image, determine the spatial coordinates of each boundary pixel point in the target boundary in the robot coordinate system; Generate a movement path according to the spatial coordinates corresponding to each boundary pixel point in the target boundary.

9. A computer-readable medium, on which a computer program is stored, characterized in that, When the computer program is executed by a processor, it implements the method for generating a movement path according to any one of claims 1 to 8.

10. An electronic device, characterized in that, Including: A processor; And A memory for storing executable instructions of the processor; Wherein, when the processor executes the executable instructions, the electronic device executes the method for generating a movement path according to any one of claims 1 to 8.

Images