Obstacle avoidance method and system for movable sentry robot

Abstract

The invention discloses an obstacle avoidance method and system for a movable sentry robot, and the method comprises the following steps: 1, obtaining a front road video shot by a camera in the movable sentry robot in real time; 2, extracting other real-time images except the road as obstacle videos based on the video of the front road surface; 3, based on the obstacle video, extracting the moving direction and the moving speed of the obstacle, and simulating the obstacle on an electronic map in real time; and 4, based on the real-time simulation, establishing a navigation route, so that the movable sentry robot advances along the navigation route. According to the movable sentry robot, the navigation route of the movable sentry robot is planned according to the moving direction and the moving speed of the obstacle, when the movable sentry robot moves on a construction site, the route can be adjusted according to the implementation condition of the construction site, the obstacle is effectively avoided, and the movable sentry robot moves flexibly; therefore, the construction site is managed efficiently.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to an obstacle avoidance method and system for a movable sentinel robot. Background Art

[0002] In modern construction sites, in order to manage the construction site intelligently, there are often one or more intelligent mobile sentinel robots at the construction site to assist in the management of the construction site and facilitate the smooth progress of the construction.

[0003] However, the terrain at the construction site is relatively complex, and various construction debris are often temporarily piled on the road surface. Construction workers do not constantly shuttle on the road surface, and traditional robots move forward based on maps and positioning navigation. The map cannot update and judge the temporarily piled debris in real time. Therefore, traditional robots cannot move flexibly at construction sites with complex terrain conditions, which makes it inconvenient to manage the construction site in an information-based manner. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide an obstacle avoidance method and system for a mobile sentinel robot in view of the deficiencies in the above-mentioned prior art, so as to solve the problems raised in the above-mentioned background technology.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: an obstacle avoidance method and system for a movable sentinel robot, wherein the system and the method are based on the same inventive concept and are used in conjunction with the method to complement each other.

[0006] An obstacle avoidance method for a mobile sentry robot comprises the following steps:

[0007] Step 1: Obtain the road surface video captured in real time by the camera of the mobile sentinel robot;

[0008] Step 2: Based on the front road surface video, extract other real-time images outside the road as obstacle videos;

[0009] Step 3: Based on the obstacle video, extract the moving direction and moving speed of the obstacle, and simulate the obstacle in real time on the electronic map;

[0010] Step 4: Based on the real-time simulation, a navigation route is established so that the mobile sentinel robot moves along the navigation route.

[0011] Further, in step 2, based on the front road surface video, extracting other real-time images outside the road as obstacle videos includes:

[0012] Decomposing each frame of the front road surface video to obtain a front road surface image of the front road surface video at each moment;

[0013] Removing the road surface image from the front road surface image at each moment respectively to obtain an obstacle image;

[0014] The obstacle images at each moment are spliced ​​in chronological order to obtain the obstacle video.

[0015] Furthermore, the removing the road surface image from the front road surface image at each moment to obtain the obstacle image includes:

[0016] Based on the front road surface image, separate each element in the front road surface image by image processing;

[0017] Removing road surface elements from the front road surface image to obtain obstacle elements;

[0018] removing obstacle elements whose sizes are smaller than a first set value from the obstacle elements, updating the obstacle elements, and obtaining the obstacle image;

[0019] The above three steps are performed to traverse the front road surface image at each moment to obtain the obstacle image at each moment.

[0020] Furthermore, the separating each element in the road surface image ahead by image processing based on the road surface image ahead includes:

[0021] According to the set shapes of the elements, the elements are mapped onto the front road surface image by means of projection;

[0022] Each element on the front road surface image is determined according to the projection difference.

[0023] Furthermore, the projecting of each element onto the front road surface image according to the set shape of each element includes:

[0024] Get the set shape of the element;

[0025] converting the front road surface image into a line drawing;

[0026] The set shape of the element is scaled and moved, and matched with the line drawing to obtain each element in the front road surface image.

[0027] Further, in step 3, the moving direction and moving speed of the obstacle are extracted, including:

[0028] According to the obstacle video, the moving track of the obstacle is marked on the electronic map by using the reference object in the front road video;

[0029] The initial coordinates and the end coordinates of the movement of the obstacle are obtained from the electronic map, and the moving direction and the moving speed of the obstacle are obtained.

[0030] Furthermore, the step of obtaining the initial coordinates and the final coordinates of the movement of the obstacle from the electronic map includes:

[0031] Obtain two adjacent video frames in the obstacle video;

[0032] Based on the earlier video frames, the initial coordinates of the obstacles are obtained from the electronic map;

[0033] Based on the later video frames, the end coordinates of the obstacle are obtained from the electronic map.

[0034] Furthermore, the step of obtaining the initial coordinates and the final coordinates of the movement of the obstacle from the electronic map includes:

[0035] Obtaining the first video frame and the last video frame in the obstacle video;

[0036] Based on the first video frame, the initial coordinates of the obstacle are obtained from the electronic map;

[0037] Based on the last video frame, obtain the end coordinates of the obstacle from the electronic map

[0038] An obstacle avoidance system for a mobile sentry robot, comprising:

[0039] A road surface acquisition module is used to acquire the road surface video ahead captured in real time by the camera in the mobile sentry robot;

[0040] An obstacle extraction module, used to extract other real-time images outside the road as obstacle videos based on the front road surface video;

[0041] An obstacle simulation module, used to extract the moving direction and moving speed of the obstacle based on the obstacle video, and simulate the obstacle in real time on the electronic map;

[0042] The route navigation module is used to establish a navigation route based on the real-time simulation, so that the movable sentinel robot moves along the navigation route.

[0043] Compared with the prior art, the present invention has the following advantages:

[0044] The present invention provides an obstacle avoidance method and system for a mobile sentry robot, which obtains a video of a road ahead and extracts obstacles from the road video, displays the obstacles on an electronic map, and plans a navigation route of the mobile sentry robot according to the moving direction and speed of the obstacles. When the mobile sentry robot moves at a construction site, it can adjust the route according to the implementation situation of the construction site, effectively avoid obstacles, and move flexibly, thereby efficiently managing the construction site.

[0045] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of the overall process of an obstacle avoidance method for a mobile sentinel robot provided by the present invention.

[0047] Figure 2 A schematic diagram of the overall structure of an obstacle avoidance system for a mobile sentinel robot provided by the present invention. DETAILED DESCRIPTION

[0048] like Figure 1-2 As shown, the present invention provides an obstacle avoidance method and system for a mobile sentry robot, wherein the system and the method are based on the same inventive concept and are used in conjunction with the method to complement each other. Below we introduce the method and the system separately.

[0049] like Figure 1 As shown, the present invention proposes an obstacle avoidance method for a mobile sentinel robot, comprising the following steps:

[0050] Step 1: Obtain the road surface video captured in real time by the camera of the mobile sentinel robot;

[0051] Step 2: Based on the front road surface video, extract other real-time images outside the road as obstacle videos;

[0052] Step 3: Based on the obstacle video, extract the moving direction and moving speed of the obstacle, and simulate the obstacle in real time on the electronic map;

[0053] Step 4: Based on the real-time simulation, a navigation route is established so that the mobile sentinel robot moves along the navigation route.

[0054] In the above steps 1-4, the obstacle video is extracted from the road surface video in front taken by the camera in the mobile sentinel robot, so as to determine the obstacle in front of the mobile sentinel robot. Then, by combining with the electronic map, the moving speed and direction of the obstacle are determined, so as to avoid the obstacle based on these and plan the navigation route so that the mobile sentinel robot can cross the obstacle and move forward according to the navigation route. Then, when the mobile sentinel robot moves at the construction site, it can adjust the route according to the implementation situation of the construction site, effectively avoid obstacles, move flexibly, and thus efficiently manage the construction site.

[0055] In step 1, the video captured by the camera of the mobile sentinel robot is imported in real time. The video of the road ahead can be imported once every five seconds, or every 2-3 seconds. The time setting depends on the moving speed of the mobile sentinel robot.

[0056] In step 2, the video imported in step 1 is processed. Specifically, based on the front road surface video, other real-time images outside the road are extracted as obstacle videos, including the following steps:

[0057] Decomposing each frame of the front road surface video to obtain a front road surface image of the front road surface video at each moment;

[0058] Removing the road surface image from the front road surface image at each moment respectively to obtain an obstacle image;

[0059] The obstacle images at each moment are spliced ​​in chronological order to obtain the obstacle video.

[0060] Step 2 extracts obstacles from each video frame through video frames, and then splices the obstacles according to the timestamps on the video frames to obtain the obstacle video. High-resolution video can effectively extract obstacles, and when judging the moving direction and speed of obstacles in the subsequent process, the shape of obstacles can be accurately extracted, so that obstacles can be avoided when planning routes, ensuring that the mobile sentinel robot will not collide or rub against obstacles when moving.

[0061] The step of removing the road surface image from the front road surface image at each moment to obtain the obstacle image includes:

[0062] Based on the front road surface image, separate each element in the front road surface image by image processing;

[0063] Removing road surface elements from the front road surface image to obtain obstacle elements;

[0064] removing obstacle elements whose sizes are smaller than a first set value from the obstacle elements, updating the obstacle elements, and obtaining the obstacle image;

[0065] The above three steps are performed to traverse the front road surface image at each moment to obtain the obstacle image at each moment.

[0066] When obtaining the obstacle image, the present invention extracts the obstacles in the front road surface image in an element-based manner. Generally, the road surface is one of the elements, and the piled debris is also an element. The shapes of the elements are different, and the corresponding elements are removed. For removing obstacles whose sizes are smaller than the first set value, we default to removing obstacles that are farther away and smaller obstacles, because these obstacles cannot affect the movement of the movable sentinel robot.

[0067] The separating of the elements in the road surface image ahead by image processing based on the road surface image ahead includes:

[0068] According to the set shapes of the elements, the elements are mapped onto the front road surface image by means of projection;

[0069] Each element on the front road surface image is determined according to the projection difference.

[0070] When separating each element, the present invention uses the shape of the element to separate the elements in the image, while the traditional method is to extract the image based on the color, and then determine the shape of the image and compare it with the shape of the element, so as to separate the element and determine the type of the element. The present invention optimizes the above steps, directly uses the shape of the element to compare in the image, and then obtains each element on the image of the road ahead.

[0071] The following are the process and steps of extracting each element in the image. According to the set shape of each element, each element is projected onto the front road surface image, including the following steps:

[0072] Get the set shape of the element;

[0073] converting the front road surface image into a line drawing;

[0074] The set shape of the element is scaled and moved, and matched with the line drawing to obtain each element in the front road surface image.

[0075] When matching, the present invention only requires that the shape of the element and the line diagram roughly match, that is, the occurrence of coordinate errors is allowed, and then the elements in the front road image are determined within the allowed error range.

[0076] In step 3, the moving direction and moving speed of the obstacle are extracted, including:

[0077] According to the obstacle video, the moving track of the obstacle is marked on the electronic map by using the reference object in the front road video;

[0078] The initial coordinates and the end coordinates of the movement of the obstacle are obtained from the electronic map, and the moving direction and the moving speed of the obstacle are obtained.

[0079] The present invention determines the time difference by video timestamp, determines the travel distance by position, and then calculates the moving direction and moving speed of the obstacle. When calculating, the present invention provides the following two methods, both of which are calculated by coordinates. Among them, the introduction of electronic maps can more accurately obtain the coordinates of the obstacle. For obstacles with a certain area, we use their center points to determine the coordinates of the obstacle, which is concise and representative when calculating its moving direction and moving speed.

[0080] Method 1: The method of obtaining the initial coordinates and the final coordinates of the movement of the obstacle from the electronic map includes:

[0081] Obtain two adjacent video frames in the obstacle video;

[0082] Based on the earlier video frames, the initial coordinates of the obstacles are obtained from the electronic map;

[0083] Based on the later video frames, the end coordinates of the obstacle are obtained from the electronic map.

[0084] Method 2: The method of obtaining the initial coordinates and the end coordinates of the obstacle from the electronic map includes:

[0085] Obtaining the first video frame and the last video frame in the obstacle video;

[0086] Based on the first video frame, the initial coordinates of the obstacle are obtained from the electronic map;

[0087] Based on the last video frame, obtain the end coordinates of the obstacle from the electronic map

[0088] The difference between the above-mentioned method 1 and method 2 is that different timestamps are used to determine the different positions of obstacles. The accuracy of method 1 is higher than that of method 2. In practice, it can be set according to the needs of the actual construction site. For a wide construction site, method 2 can be selected, and for a smaller construction site, method 1 can be selected.

[0089] Based on the inventive concept of the above method, Figure 2 As shown, the present invention also proposes an obstacle avoidance system for a mobile sentinel robot, comprising:

[0090] A road surface acquisition module is used to acquire the road surface video ahead captured in real time by the camera in the mobile sentry robot;

[0091] An obstacle extraction module, used to extract other real-time images outside the road as obstacle videos based on the front road surface video;

[0092] An obstacle simulation module, used to extract the moving direction and moving speed of the obstacle based on the obstacle video, and simulate the obstacle in real time on the electronic map;

[0093] The route navigation module is used to establish a navigation route based on the real-time simulation, so that the movable sentinel robot moves along the navigation route.

[0094] Each module of the present invention will also implement the above method, be used to realize the above method, and provide a hardware implementation basis for the above method. Generally, each module is implemented by computer software.

[0095] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any simple modification, change and equivalent structural change made to the above embodiment based on the technical essence of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. An obstacle avoidance method for a mobile sentinel robot, characterized in that: The following steps are involved: Step 1: Obtain the road surface video captured in real time by the camera of the mobile sentinel robot; Step 2: Based on the front road surface video, extract other real-time images outside the road as obstacle videos; Step 3: Based on the obstacle video, extract the moving direction and moving speed of the obstacle, and simulate the obstacle in real time on the electronic map; Step 4: Based on the real-time simulation, a navigation route is established so that the mobile sentinel robot moves along the navigation route.

2. The obstacle avoidance method of a mobile sentinel robot according to claim 1, characterized in that: In step 2, based on the front road surface video, extracting other real-time images outside the road as obstacle videos includes: Decomposing each frame of the front road surface video to obtain a front road surface image of the front road surface video at each moment; Removing the road surface image from the front road surface image at each moment respectively to obtain an obstacle image; The obstacle images at each moment are spliced ​​in chronological order to obtain the obstacle video.

3. The obstacle avoidance method of a mobile sentinel robot according to claim 2, characterized in that: The step of removing the road surface image in the front road surface image at each moment to obtain an obstacle image includes: Based on the front road surface image, separate each element in the front road surface image by image processing; Removing road surface elements from the front road surface image to obtain obstacle elements; removing obstacle elements whose sizes are smaller than a first set value from the obstacle elements, updating the obstacle elements, and obtaining the obstacle image; The above three steps are performed to traverse the front road surface image at each moment to obtain the obstacle image at each moment.

4. The obstacle avoidance method of a mobile sentry robot according to claim 3, characterized in that: The separating each element in the road surface image ahead by image processing based on the road surface image ahead includes: According to the set shapes of the elements, the elements are mapped onto the front road surface image by means of projection; Each element on the front road surface image is determined according to the projection difference.

5. The obstacle avoidance method of a mobile sentry robot according to claim 4, characterized in that: The step of projecting each element onto the front road surface image according to the set shape of each element includes: Get the set shape of the element; converting the front road surface image into a line drawing; The set shape of the element is scaled and moved, and matched with the line drawing to obtain each element in the front road surface image.

6. The obstacle avoidance method of a mobile sentry robot according to claim 1, characterized in that: In step 3, the moving direction and moving speed of the obstacle are extracted, including: According to the obstacle video, the moving track of the obstacle is marked on the electronic map by using the reference object in the front road video; The initial coordinates and the end coordinates of the movement of the obstacle are obtained from the electronic map, and the moving direction and the moving speed of the obstacle are obtained.

7. The obstacle avoidance method of a mobile sentry robot according to claim 6, characterized in that: The step of obtaining the initial coordinates and the final coordinates of the movement of the obstacle from the electronic map includes: Obtain two adjacent video frames in the obstacle video; Based on the earlier video frames, the initial coordinates of the obstacles are obtained from the electronic map; Based on the later video frames, the end coordinates of the obstacle are obtained from the electronic map.

8. The obstacle avoidance method of a mobile sentry robot according to claim 6, characterized in that: The step of obtaining the initial coordinates and the final coordinates of the movement of the obstacle from the electronic map includes: Obtaining the first video frame and the last video frame in the obstacle video; Based on the first video frame, the initial coordinates of the obstacle are obtained from the electronic map; Based on the last video frame, the end coordinates of the obstacle are obtained from the electronic map.

9. An obstacle avoidance system for a mobile sentry robot, characterized in that: include: A road surface acquisition module is used to acquire the road surface video ahead captured in real time by the camera in the mobile sentry robot; An obstacle extraction module, used to extract other real-time images outside the road as obstacle videos based on the front road surface video; An obstacle simulation module, used to extract the moving direction and moving speed of the obstacle based on the obstacle video, and simulate the obstacle in real time on the electronic map; The route navigation module is used to establish a navigation route based on the real-time simulation, so that the movable sentinel robot moves along the navigation route.

Images