Indoor blind-view obstacle avoidance method and system for mobile robot

Abstract

The invention discloses an indoor blind-view obstacle avoidance method and system for a mobile robot, and relates to the technical field of robot design, cameras are arranged at indoor corners or view blind areas and other positions, pictures of pedestrians are shot when the pedestrians appear, all possible indoor positions where the pedestrians appear can be covered in an all-around mode, and the indoor blind-view obstacle avoidance method and system for the mobile robot are achieved. The method comprises the following steps of: firstly, firstly, identifying pedestrians in a moving image and framing the identified pedestrians in the moving image by adopting a rectangular frame to form a candidate frame when judging the distance between a camera and the pedestrians, namely, covering all view blind areas in all directions; according to the method, the depth camera is used for acquiring the depth value of the candidate frame, and the distance between the depth camera and the pedestrian is accurately judged according to the depth value, so that when the pedestrian appears in the accurate range of the threshold value of the area range corresponding to the depth camera, the robot accurately avoids the obstacle, and potential safety hazards are avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of robot design, and in particular to a method and system for avoiding obstacles in the blind field of an indoor mobile robot. Background Art

[0002] With the continuous improvement of social science and technology and the development of productivity, robots and artificial intelligence technologies have been widely used in production and life in all walks of life; the rapid development of the field of artificial intelligence, the rapid innovation of robotics technology, the in-depth promotion of intelligent manufacturing projects, and the promotion of innovative development of high-end manufacturing industries including intelligent robots have become the current development trend.

[0003] With the increasing demand for robots to be able to navigate autonomously and avoid obstacles in real time, more and more research has been conducted on obstacle avoidance in mobile robot navigation. Traditional robot autonomous obstacle avoidance uses sensors such as lidar installed on the robot to add a cost map to the global map to achieve real-time obstacle avoidance with pedestrians. However, when the robot is indoors, due to the complexity of the indoor environment, the robot will have many blind spots that are difficult to cover by radar. At the same time, it is difficult for the robot to accurately judge the positional relationship between pedestrians, making it difficult to achieve more accurate obstacle avoidance. Therefore, when danger approaches, if the robot cannot process the sudden obstacle information in time, it will cause major safety hazards. Summary of the Invention

[0004] The embodiments of the present invention provide a method and system for indoor blind-sight obstacle avoidance for a mobile robot, which can solve the problem in the prior art that, when the current sensor detection technology is applied, the robot will have many blind spots that are difficult to cover by radar, and it is also difficult for the robot to accurately judge the positional relationship between pedestrians, making it difficult to achieve more accurate obstacle avoidance.

[0005] An embodiment of the present invention provides a method for avoiding obstacles in an indoor blind field of view for a mobile robot, comprising the following steps: Use a depth camera to capture moving images of pedestrians moving indoors; Identify and calibrate pedestrians in moving images to generate calibrated candidate frames; use a depth camera to obtain the depth value of the candidate frame, and obtain the distance between the depth camera and the pedestrian based on the depth value; When the distance between the depth camera and the pedestrian is within the preset area of the depth camera, the robot is controlled to avoid obstacles.

[0006] Preferably, the depth camera is deployed at a corner of an indoor space or in a blind spot, and the depth camera and the robot are in the same local area network, and two-way communication between the depth camera and the robot is established through the TCP transmission control protocol.

[0007] Preferably, controlling the robot to avoid obstacles comprises: The robot establishes a map coordinate system based on the position of the depth camera in the indoor space. The depth camera and the robot both have a position coordinate in the map coordinate system, and the depth camera has an area range in the map coordinate system; In the process of controlling the robot to avoid obstacles, when the robot's position coordinates are within the area of the depth camera, the robot sends a presence signal to the depth camera.

[0008] Preferably, the controlling the robot to avoid obstacles and determining whether the robot should continue to travel comprises: When the depth camera detects that the distance between the pedestrian and the depth camera is within the area preset by the depth camera, and the depth camera receives the presence signal sent by the robot, the depth camera sends a stop signal to the robot, and the robot stops moving; When the depth camera detects that the distance between the pedestrian and the depth camera is within the area preset by the depth camera, but the depth camera does not receive the presence signal sent by the robot, the depth camera sends a keep-connected signal to the robot, and the robot continues to drive; When the depth camera does not detect that the distance between the pedestrian and the depth camera is within the area preset by the depth camera, and the depth camera receives the presence signal sent by the robot, the depth camera sends a keep-connected signal to the robot, and the robot continues to drive; When the depth camera does not detect that the distance between the pedestrian and the depth camera is within the area preset by the depth camera, and the depth camera does not receive the presence signal sent by the robot, the depth camera sends a keep-connected signal to the robot, and the robot continues to drive.

[0009] An embodiment of the present invention further provides a mobile robot indoor blind field obstacle avoidance system, comprising: An image acquisition module, used to capture motion images of pedestrians moving indoors using a depth camera; The decision module is used to identify and calibrate pedestrians in the moving image and generate calibrated candidate frames; obtain the depth value of the candidate frame using the depth camera, and obtain the distance between the depth camera and the pedestrian based on the depth value; The obstacle avoidance module is used to control the robot to avoid obstacles when the distance between the depth camera and the pedestrian is within the area preset by the depth camera.

[0010] An embodiment of the present invention further provides an electronic device, including a memory and a processor; The memory is used to store computer programs; The processor is configured to implement the steps of the above-mentioned method for avoiding obstacles in an indoor blind field of view for a mobile robot when executing the computer program stored in the memory.

[0011] An embodiment of the present invention further provides a computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the steps of the above-mentioned method for avoiding obstacles in an indoor blind field of view for a mobile robot.

[0012] The embodiments of the present invention provide a method and system for avoiding obstacles in blind areas of indoor vision for a mobile robot. Compared with the prior art, the methods and systems have the following advantages: The present invention arranges cameras at corners or blind spots in the room to take pictures of pedestrians when they appear, thereby comprehensively covering all possible positions of pedestrians in the room, that is, comprehensively covering all blind spots in the field of view. At the same time, when judging the distance between the camera and the pedestrian, the pedestrian in the moving image is first identified, and the pedestrian identified in the moving image is calibrated and framed with a rectangular frame to form a candidate frame; the depth camera is used to obtain the depth value of the candidate frame, and the distance between the depth camera and the pedestrian is accurately judged according to the depth value, so that when the pedestrian appears within the precise range of the threshold value of the corresponding area range of the depth camera, the robot can achieve accurate obstacle avoidance and avoid the occurrence of safety hazards. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A schematic diagram of an obstacle avoidance process for a mobile robot in an indoor blind field of view provided by an embodiment of the present invention; Figure 2 A schematic diagram of the joint blind-field obstacle avoidance operation of the camera side and the robot side in a method for avoiding obstacles in an indoor blind-field obstacle of a mobile robot provided by an embodiment of the present invention; Figure 3 A schematic diagram of camera-side pedestrian detection and received message effects in a method for avoiding obstacles in a blind field of view for a mobile robot indoors provided by an embodiment of the present invention; Figure 4 This is a schematic diagram of the effect of a message sent by a camera end received by a robot end of a mobile robot indoor blind field obstacle avoidance method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0014] To make the above-mentioned objects, features, and advantages of the present invention more readily apparent, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings. The following description sets forth numerous specific details to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art may make similar modifications without departing from the scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0015] See also Figure 1, an embodiment of the present invention provides a method for indoor blind-field obstacle avoidance for a mobile robot, specifically, a depth camera is placed at a fixed position outside the robot's body in the space where the robot is located and connected to a computer; a TCP communication mechanism is used between the car and the depth camera, and the present invention sets a threshold area at the corner of the robot's space, and the robot sends a posture topic in real time; the depth camera uses a deep learning algorithm to detect pedestrians and obtains the depth information of the detection frame, and sets a detection frame depth threshold on the depth camera; when the robot enters the threshold area at the corner of the space where it is located, and the camera detects a pedestrian and the detection frame depth is within the threshold, the robot publishes a speed topic with zero linear velocity and zero angular velocity to stop the robot from moving. When the pedestrian has passed, that is, the depth camera cannot detect the pedestrian or the detection frame does not meet the threshold requirement, the car continues to drive to complete the blind-field obstacle avoidance. The specific steps include:

[0016] Step 1: Place a depth camera outside the robot body, connect the camera to the computer, and perform TCP communication between the robot and the camera.

[0017] TCP communication between the robot and the camera is as follows: The robot side is set as the client side and the depth camera side is set as the server side. Since the implementation environment of the present invention is indoors, the robot and the camera side communicate under the same local area network.

[0018] Among them, by deploying multiple OAK intelligent depth cameras and building a camera network, more comprehensive environmental perception can be achieved; using the TCP / IP protocol to fuse and collaborate data between multiple cameras, the robot's perception of complex environments can be enhanced.

[0019] At the same time, according to the real-time needs of the communication process, the bandwidth allocation of TCP communication is dynamically adjusted to ensure the timely transmission of key data and improve communication efficiency.

[0020] Step 2: The car publishes a pose topic while driving in the space it is in, and updates its position in real time. The depth camera uses a deep learning algorithm to detect pedestrians in the space it is in.

[0021] The camera uses a deep learning algorithm for pedestrian detection, including: The present invention selects the YOLO V5 lightweight neural network algorithm for pedestrian detection and uses a depth camera to extract the depth information of the pedestrian detection frame.

[0022] At the same time, the pedestrian detection model is dynamically selected based on the complexity of the environment (such as pedestrian flow and lighting conditions). In simple environments, a lightweight model is used to save computing resources, while in complex environments, a high-precision model is switched to ensure detection accuracy.

[0023] Step 3: The robot car drives to a designated area and sends a specified message to the depth camera. The camera detects pedestrians and determines whether the robot car can continue to drive.

[0024] The camera determines whether the car can continue to drive, specifically including: When the depth camera detects that a pedestrian appears within the set depth threshold of the pedestrian detection frame and the camera receives the "appear" message sent by the robot, the camera sends a "stop" message to the robot; When the depth camera detects that a pedestrian appears within the set depth threshold of the pedestrian detection frame but the camera does not receive the "appear" message sent by the robot, the camera sends a "keep connected" message to the robot; When the depth camera does not detect a pedestrian within the set depth threshold of the pedestrian detection frame but the camera receives the "appear" message sent by the robot, the camera sends a "keep connected" message to the robot; When the depth camera does not detect a pedestrian within the set depth threshold of the pedestrian detection frame and the camera does not receive the "appear" message sent by the robot, the camera sends a "keep connected" message to the robot.

[0025] Among them, machine learning algorithms are used to dynamically adjust regional thresholds based on environmental perception data (such as pedestrian flow and obstacle distribution); for example, the threshold range is automatically narrowed in areas with dense pedestrian flow to improve obstacle avoidance sensitivity; the threshold range is appropriately expanded in open areas to reduce unnecessary obstacle avoidance operations.

[0026] According to the complexity of the environment and the robot's task requirements, the indoor space is divided into multiple levels of areas (such as high-risk area, medium-risk area, and low-risk area), and different obstacle avoidance strategies and threshold parameters are set for different areas.

[0027] Step 4: The camera makes a decision and sends a message to the robot, which then adjusts its actions based on the message.

[0028] The robot adjusts its actions based on the messages it receives, including: When the robot car receives a "stop" message, it publishes a topic with a speed of 0; when the robot car receives a "keep connected" message, it drives according to the originally planned route.

[0029] like Figure 1 As shown in the figure, pedestrian detection on the camera side specifically includes: The camera uses the YOLO V5 algorithm for pedestrian detection and uses a depth camera to extract the depth information of the detection frame. A depth threshold is set. That is, when a pedestrian walks into the threshold area, the camera starts to make corresponding decisions based on the message sent by the robot.

[0030] Regardless of the setting of the robot position coordinate threshold or the setting of the pedestrian detection frame depth threshold on the camera side, when outside the threshold, the robot car and the pedestrian will not meet near the corner, and the general lidar obstacle avoidance method can also complete the obstacle avoidance; and within the range of the threshold, the general obstacle avoidance method will no longer be applicable. The method provided by the present invention can be used within this range, ensuring the practicality of the present invention.

[0031] like Figure 1 As shown in the figure, the indoor blind field obstacle avoidance method for mobile robots needs to meet two conditions: the robot car reaches the designated area and sends a "appearance" message to the camera end; the camera end detects that the pedestrian appears within the depth threshold range.

[0032] When the above two conditions are met, the camera sends a "stop" message to the robot car. After receiving the message, the robot car publishes a speed topic with a linear velocity and angular velocity of 0, and the car stops moving. When only one of the above conditions is met or none of them are met, the camera sends a "keep connected" message to the robot car, and the robot car continues to drive along the original route.

[0033] The present invention provides various situation descriptions for robot-side position judgment and camera-side pedestrian detection; through the robot's entry feedback into the designated area at the indoor corner and the pedestrian appearance prompt on the camera side, the robot car can also perform real-time obstacle avoidance in invisible blind spots, thereby improving the safety of the robot's indoor environment navigation, and has a high application prospect in the field of robot indoor navigation.

[0034] The embodiment of the present invention randomly sets up a spatial environment and places a depth camera on the wall at its corner, as shown in the schematic diagram. Figure 2 As shown in the figure, the robot and the camera exchange information through TCP. The robot car updates its position information in real time by publishing the posture topic. When the robot car reaches the specified area and meets the preset threshold, the camera sends an "appear" message to the robot car. The effect is as follows: Figure 3 As shown in the figure, the camera uses the YOLO V5 pedestrian detection algorithm to detect pedestrian targets in indoor scenes and obtain the depth information of the pedestrian detection frame. After receiving the "appear" message sent by the robot, if the camera detects that the pedestrian appears within the depth threshold, it sends a "stop" message to the robot car; if the camera does not receive the "appear" message, or the camera does not detect that the pedestrian appears within the depth threshold, it sends a "hand!" message to the robot car, as shown in the figure. Figure 4When the robot car receives a "stop" message, it publishes the "geometry_msgs / Twist" topic and updates the car's linear velocity and angular velocity to 0, thereby stopping the movement to ensure the safety of pedestrians. When the robot car receives a "hand!" message, it indicates that the robot end and the camera end are communicating normally, and the robot car drives along the original route. The method for avoiding obstacles in blind spots indoors provided by the present invention can easily and accurately avoid obstacles in blind spots indoors. Through the operation of this embodiment, the robot car completes indoor blind spot obstacle avoidance, which also verifies the feasibility of the method for avoiding obstacles in blind spots indoors provided by the present invention.

[0035] The present invention achieves the detection and position determination of pedestrians in indoor environments through depth extraction of a depth camera and pedestrian detection using the YOLO V5 algorithm. Through TCP communication, information exchange between the camera and robot ends is achieved, allowing the robot car to make autonomous adjustments based on the decisions made by the camera end. The present invention places the camera used to detect pedestrians outside the vehicle body, and can perform TCP communication with different robot cars, thereby achieving blind field obstacle avoidance. It is convenient and fast to use and can cope with various blind field problems caused by complex indoor environments. Compared with the existing technology, this mobile robot indoor blind field obstacle avoidance method solves the problem of complex indoor environments and the many blind fields that occur in robot car navigation. It only requires one depth camera to perform TCP communication with multiple robot cars, making it easy to use and has broad prospects in the field of indoor robot navigation and obstacle avoidance.

[0036] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be determined by the appended claims.

Claims

1. A method for avoiding obstacles in indoor blind field of view of a mobile robot, characterized in that: The following steps are involved: Use a depth camera to capture moving images of pedestrians moving indoors; Identify and calibrate pedestrians in moving images to generate calibrated candidate frames; use a depth camera to obtain the depth value of the candidate frame, and obtain the distance between the depth camera and the pedestrian based on the depth value; When the distance between the depth camera and the pedestrian is within the preset area of the depth camera, the robot is controlled to avoid obstacles.

2. The method for avoiding obstacles in indoor blind field of a mobile robot according to claim 1, characterized in that: The depth camera is deployed at a corner of an indoor space or in a blind spot, and the depth camera and the robot are in the same local area network, and two-way communication is established between the depth camera and the robot through the TCP transmission control protocol.

3. The method for avoiding obstacles in indoor blind field of a mobile robot according to claim 1, characterized in that: The controlling the robot to avoid obstacles includes: The robot establishes a map coordinate system based on the position of the depth camera in the indoor space. The depth camera and the robot both have a position coordinate in the map coordinate system, and the depth camera has an area range in the map coordinate system; In the process of controlling the robot to avoid obstacles, when the robot's position coordinates are within the area of the depth camera, the robot sends a presence signal to the depth camera.

4. The method for avoiding obstacles in indoor blind field of a mobile robot according to claim 3, characterized in that: During the obstacle avoidance control of the robot, determining whether the robot should continue to travel includes: When the depth camera detects that the distance between the pedestrian and the depth camera is within the area preset by the depth camera, and the depth camera receives the presence signal sent by the robot, the depth camera sends a stop signal to the robot, and the robot stops moving; When the depth camera detects that the distance between the pedestrian and the depth camera is within the area preset by the depth camera, but the depth camera does not receive the presence signal sent by the robot, the depth camera sends a keep-connected signal to the robot, and the robot continues to drive; When the depth camera does not detect that the distance between the pedestrian and the depth camera is within the area preset by the depth camera, and the depth camera receives the presence signal sent by the robot, the depth camera sends a keep-connected signal to the robot, and the robot continues to drive; When the depth camera does not detect that the distance between the pedestrian and the depth camera is within the area preset by the depth camera, and the depth camera does not receive the presence signal sent by the robot, the depth camera sends a keep-connected signal to the robot, and the robot continues to drive.

5. A mobile robot indoor blind field obstacle avoidance system, characterized in that: include: An image acquisition module, used to capture motion images of pedestrians moving indoors using a depth camera; The decision module is used to identify and calibrate pedestrians in the moving image and generate calibrated candidate frames; obtain the depth value of the candidate frame using the depth camera, and obtain the distance between the depth camera and the pedestrian based on the depth value; The obstacle avoidance module is used to control the robot to avoid obstacles when the distance between the depth camera and the pedestrian is within the area preset by the depth camera.

6. An electronic device, characterized in that: include: memory and processor; The memory is used to store computer programs; The processor is configured to implement the steps of a method for avoiding obstacles in an indoor blind field of view for a mobile robot as described in any one of claims 1 to 4 when executing the computer program stored in the memory.

7. A computer-readable storage medium, characterized in that Used to store a computer program, which, when executed by a processor, implements the steps of a method for avoiding obstacles in an indoor blind field of view of a mobile robot as described in any one of claims 1 to 4.

Images