Indoor location recognition device and method based on indoor robot

Abstract

The present invention discloses an indoor location recognition device and method based on an indoor robot. The device comprises a Raspberry Pi computer, a laser radar (LIDAR), a CMOS camera, an ultrasonic sensor, a first microcontroller, and a servo motor. The LIDAR is connected to the Raspberry Pi computer and driven by the servo motor. The ultrasonic sensor is connected to the Raspberry Pi computer via the first microcontroller, and the Raspberry Pi controls the LIDAR and ultrasonic sensor to scan locations. The CMOS camera is connected to the Raspberry Pi computer and issues commands to control the CMOS camera to photograph objects and locations. This device can continuously identify indoor locations under various complex conditions, while improving the accuracy and reliability of location recognition.

Description

Technical Field

[0001] The present invention belongs to the field of indoor location recognition based on laser radar and vision, and particularly relates to an indoor location recognition device and method based on an indoor robot. Background Art

[0002] Traditional indoor location recognition devices and methods mostly use a single instrument, such as a camera, and are vision-based. These cameras take numerous photos of indoor locations as a training set. Local features are then extracted from these training set photos. The YOLOv3 algorithm network is then used for recognition. Specific objects suitable for environmental modeling are selected, and convolutional neural network features are extracted. Experiments are conducted on a publicly available standard dataset for testing indoor location recognition algorithms, yielding their own average recognition accuracy and achieving the desired recognition results. Convolutional neural networks and deep learning offer promising and robust methods for extracting visual features. However, these algorithms are susceptible to factors such as lighting intensity and the external environment, resulting in generally low recognition rates in complex environments. Recognition performance improves in single environments, but recognition speed is also less than ideal. This is because higher accuracy requires more information, which increases the computational load and memory consumption, leading to slower recognition speeds. Summary of the Invention

[0003] Purpose of the invention: The purpose of the present invention is to provide an indoor location recognition device and method based on an indoor robot, which enhances the accuracy and reliability of location recognition. The use of laser radar to scan the location and the use of CMOS camera to identify objects in the location further enhance the accuracy and reduce the impact of other external environments on the recognition results.

[0004] Technical solution: The indoor location recognition device based on an indoor robot of the present invention includes a computer Raspberry Pi, a laser radar, a CMOS camera, an ultrasonic sensor, a first microcontroller, and a servo motor; the laser radar is connected to the computer Raspberry Pi, and the laser radar is driven by the servo motor, and the measurement direction of the laser radar is controlled by the rotation of the servo motor. The ultrasonic sensor is connected to the computer Raspberry Pi through the first microcontroller, and the computer Raspberry Pi controls the laser radar to measure the width and length of the current location, and controls the ultrasonic sensor to scan the location up and down to measure the distance between the ultrasonic sensor and the ground and the ceiling respectively, and obtain the height of the indoor room; the CMOS camera is connected to the computer Raspberry Pi, and the computer Raspberry Pi issues instructions to control the CMOS camera, takes pictures of objects and locations, and uses OpenCV to achieve the purpose of identification.

[0005] The servo motor is installed on the top of the device and controls the rotation angle of the laser radar.

[0006] The present invention also includes an indoor location recognition method based on an indoor robot, which is applied to an indoor location recognition device based on an indoor robot. The method includes the following steps:

[0007] (1) Storing the measurement location information in the memory of the Raspberry Pi computer;

[0008] (2) The computer Raspberry Pi controls the first microcontroller to control the ultrasonic sensor to measure the indoor height;

[0009] (3) The computer Raspberry Pi controls the laser radar to scan the indoor environment in the horizontal direction, and the servo motor controls the laser radar to move upward at a fixed angle from the horizontal direction to further scan. Each upward movement obtains a set of distances between the laser radar and the wall or surrounding objects; until the two largest distances are found in each distance set, the distance from the laser radar to the top angle between the ceiling and the wall is obtained, and then the width and length of the room are calculated based on the angle between the two distance lines and the cosine formula;

[0010] (4) The computer Raspberry Pi controls the CMOS camera to identify special objects in the room and compares the information obtained in the above steps with the information pre-stored in the memory; repeats the identification and compares again to obtain the identification result.

[0011] In step (1), the information of the measurement location is stored in the memory of the Raspberry Pi computer, which specifically includes the following steps:

[0012] (1.1) The data of each location to be measured is stored in the local database of the Raspberry Pi computer. After starting the program, the information in the local database will be compared with the actual measured data;

[0013] (1.2) Each time the device is started, the indoor dimensions and the shape of a specific object are detected. If the detected location does not match any visited location in the local database, the location recognition module will add the new location to the local database using a unique name.

[0014] In step (2), the first microcontroller controls the ultrasonic sensor to measure the indoor height. Specifically, the two ultrasonic sensors installed on the base respectively measure the distance h2 from the ultrasonic sensor to the ceiling h1 and to the ground. The sum of the two is the height of the room h=h1+h2.

[0015] The step (3) specifically includes the following steps:

[0016] (3.1) After measuring the indoor height and determining that the room is indoors, the horizontal servo motor will rotate from the horizontal direction to the vertical direction at a preset step size. Every time a preset angle is reached, the laser radar scanning result is stored in a list Hread[n];

[0017] (3.2) Pick out the two maximum values from the distance list and put them into Vread[m], find the two distance peaks from Vread[m], and thus find the position closest to the ceiling corner;

[0018] (3.3) Use this information and the cosine function from trigonometric functions to calculate the width and length of the room.

[0019] The step (4) specifically includes the following steps:

[0020] (4.1) The computer Raspberry Pi controls the CMOS camera to use OpenCV technology to identify special objects in the room and compare the information obtained from the above measurements with the information stored in the database;

[0021] (4.2) If the accuracy of the comparison result is greater than or equal to 95%, the identification result is obtained, and the purpose of identifying the location is achieved;

[0022] (4.3) If the accuracy of the comparison result is less than 95%, then the surface accuracy is too low. In this case, use a CMOS camera to take a picture of the room and compare it with the database again to obtain the recognition result, thereby achieving the purpose of identifying the location.

[0023] Beneficial Effects: Compared with the existing technology, the technical solution of the present invention has the following advantages: (1) By adding image processing and object recognition technologies to the measurement of laser radar, the accuracy of indoor location recognition is greatly enhanced compared with traditional vision-based indoor location recognition methods. Compared with other 3D radars and expensive sensors, the equipment used in this device is more affordable; (2) A unique signature is generated based on the information extracted for each different location, thereby improving the accuracy and reliability of indoor location recognition by mobile robots. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic diagram showing the connection of components of an indoor location recognition device based on an indoor robot according to the present invention;

[0025] Figure 2 Schematic diagram of the flow of the indoor location recognition method based on the indoor robot of the present invention. DETAILED DESCRIPTION

[0026] The technical solution of the present invention is described in detail below in conjunction with specific implementation methods and the accompanying drawings.

[0027] like Figure 1 As shown, the device of the present invention includes a Raspberry Pi computer, a laser radar (LIDAR), a CMOS camera, an ultrasonic sensor, a first microcontroller, a servo motor, and a power module. The LIDAR is connected to the Raspberry Pi computer and driven by the servo motor. The ultrasonic sensor is connected to the Raspberry Pi computer via the first microcontroller, which controls the LIDAR and ultrasonic sensors to scan locations. The CMOS camera is connected to the Raspberry Pi computer and issues commands to control the CMOS camera to take photos of objects and locations. The first microcontroller is mounted vertically on top of the device base, and the power module supplies power to the entire device. The Raspberry Pi computer is mounted in the center of the device, and the CMOS camera is located at the bottom of the Raspberry Pi computer. The LIDAR is mounted on top of the Raspberry Pi computer, and the ultrasonic sensor is mounted on top of the device base. The LIDAR and ultrasonic sensor are placed vertically above and below the Raspberry Pi computer.

[0028] The rplidar A1 uses a 2D lidar developed by Silamtech, which uses laser triangulation for ranging. Combined with a proprietary visual acquisition and processing mechanism, it can perform over 8,000 ranging operations per second. The rplidar A1's ranging core rotates clockwise, enabling 360-degree scanning and ranging detection of the surrounding environment, thereby obtaining a contour map of the surroundings. The comprehensive use of internal optical and algorithmic systems enables the robot to build maps more quickly and accurately. Traditional non-solid-state lidars often use slip rings to transmit energy and data, but due to mechanical wear, their continuous operating life is limited to only a few thousand hours. By integrating wireless power supply and optical communication technologies, the innovative design of optical-magnetic fusion technology completely solves the problem of electrical connection failure and short lidar life caused by physical contact wear. The configurable scanning frequency of 2-10Hz can be controlled by the motor PWM signal. The rplidar A1 is directly usable by connecting it to a Raspberry Pi single-board computer via a USB cable, without any coding.

[0029] The signal amplifier matched to each pixel of the CMOS camera has a charge-to-voltage conversion. The address selection switch of the CMOS can randomly sample to achieve sub-window output. When only the sub-window image is output, a higher speed can be achieved. Therefore, the CMOS camera has a very fast readout speed and a great advantage.

[0030] The servo motor is mounted on top of the device. It's an auxiliary motor with an indirect speed change device that controls the rotation angle of the LiDAR. The servo motor is controlled by a pulse-width modulated signal via a control line. The drive shaft rotates 90 degrees clockwise and counterclockwise, increasing the amount of measurement data from other instruments. It boasts high precision, stable speed, and a strong overload capacity. Specifically, by controlling the servo motor's speed and rotation angle, it converts voltage signals into torque and speed to drive the controlled object. The servo motor can control the rotation angle and speed of the LiDAR, allowing for more information to be obtained from the LiDAR during slow rotation. The servo motor used in this solution exhibits a unique characteristic: it exhibits no self-rotation when the signal voltage is zero, and its speed decreases uniformly as torque increases.

[0031] The device first uses a rechargeable battery to power a first microcontroller, an Arduino UNO, ensuring normal operation. The first microcontroller, the Arduino UNO, is then connected to two ultrasonic sensors, which convert ultrasonic signals into other energy signals. Ultrasonic waves are mechanical waves with a vibration frequency exceeding 20kHz. They have high frequency, short wavelength, minimal diffraction, and, in particular, excellent directivity, capable of propagating in a directional manner as rays. Therefore, the two ultrasonic sensors can respectively measure the distance of the device from the ground and the ceiling.

[0032] like Figure 2 As shown, the present invention also includes an indoor location recognition method based on an indoor robot, comprising the following steps:

[0033] (1) Storing the measurement location information in the memory of the Raspberry Pi computer;

[0034] (1.1) The data of each location to be measured is stored in the local database of the Raspberry Pi computer. After starting the program, the information in the local database will be compared with the actual measured data;

[0035] (1.2) Each time the device is started, the indoor dimensions and the shape of a specific object are detected. If the detected location does not match any visited location in the local database, the location recognition module will add the new location to the local database using a unique name.

[0036] (2) The two ultrasonic sensors installed on the base measure the distance from the ultrasonic sensor to the ceiling h1 and to the ground h2 respectively. The sum of the two is the height of the room h = h1 + h2.

[0037] (3) The computer Raspberry Pi controls the laser radar to scan the indoor environment in the horizontal direction, and the servo motor controls the laser radar to move upward from the horizontal direction at a certain fixed angle value for further scanning. Each upward movement will obtain a set of distances between the laser radar and the wall or surrounding objects; until the two largest distances are found in each distance set. These two largest distances are the distances from the laser radar to the top angles of the ceiling and the wall respectively. Then, according to the angle between the two distance lines and the cosine formula, the width and length of the room are calculated; the specific steps are as follows:

[0038] (3.1) When the indoor height is measured and it is confirmed that the environment is indoors, the horizontal servo motor will rotate from the horizontal direction to the vertical direction at a preset step size. Every time a preset angle is reached, the laser radar scanning result, that is, the distance, is stored in a list Hread[n];

[0039] (3.2) Pick out the two maximum values from the distance list and put them into Vread[m], find the two distance peaks from Vread[m], and thus find the position closest to the ceiling corner;

[0040] (3.3) Use this information and the cosine function from trigonometric functions to calculate the width and length of the room.

[0041] (4) The computer Raspberry Pi controls the CMOS camera to identify special objects in the room and compares the information obtained in the above steps with the information pre-stored in the memory; repeats the identification and compares again to obtain the identification result; the specific steps are as follows:

[0042] (4.1) The computer Raspberry Pi controls the CMOS camera to use OpenCV technology to identify special objects in the room and compare the information obtained from the above measurements with the information stored in the database;

[0043] (4.2) If the accuracy of the comparison result is greater than or equal to 95%, the identification result is obtained, and the purpose of identifying the location is achieved;

[0044] (4.3) If the accuracy of the comparison result is less than 95%, then the surface accuracy is too low. In this case, use a CMOS camera to take a picture of the room and compare it with the database again to obtain the recognition result, thereby achieving the purpose of identifying the location.

Claims

1. A method for indoor location recognition based on an indoor robot, characterized by: This recognition method is applied to an indoor location recognition device based on an indoor robot. The indoor location recognition device based on the indoor robot includes a Raspberry Pi computer, a laser radar, a CMOS camera, an ultrasonic sensor, a first microcontroller, and a servo motor. The laser radar is connected to the Raspberry Pi computer, and the laser radar is driven by the servo motor. The rotation of the servo motor controls the measurement direction of the laser radar. The ultrasonic sensor is connected to the Raspberry Pi computer via the first microcontroller. The Raspberry Pi computer controls the laser radar to measure the width and length of the current location, and controls the ultrasonic sensor to scan the location up and down to measure the distance between the ultrasonic sensor and the ground and the ceiling respectively, thereby obtaining the indoor height. The CMOS camera is connected to the Raspberry Pi computer, and the Raspberry Pi computer issues instructions to control the CMOS camera to take pictures of objects and locations and use OpenCV to achieve the purpose of recognition. The identification method includes the following steps: (1) Storing the information of the measurement location in the memory of the Raspberry Pi computer; wherein the information of the measurement location is indoor dimensions and shape information of a specific object; (2) The computer Raspberry Pi controls the first microcontroller to control the ultrasonic sensor to measure the indoor height; (3) The computer Raspberry Pi controls the laser radar to scan the indoor environment in the horizontal direction, and the servo motor controls the laser radar to move upward at a fixed angle from the horizontal direction to further scan. Each upward movement obtains a set of distances between the laser radar and the wall or surrounding objects; until the two largest distances are found in each distance set, the distance from the laser radar to the top angle between the ceiling and the wall is obtained, and then the width and length of the room are calculated based on the angle between the two distance lines and the cosine formula; (4) The computer Raspberry Pi controls the CMOS camera to identify special objects in the room, and compares the height, width, length and shape information of the specific object in the room with the information pre-stored in the memory; repeats the recognition and compares again to obtain the recognition result.

2. The indoor location recognition method based on an indoor robot according to claim 1, characterized in that: The servo motor is installed on the top of the device to control the rotation angle of the laser radar.

3. The indoor location recognition method based on an indoor robot according to claim 1, characterized in that: In step (1), the information of the measurement location is stored in the memory of the Raspberry Pi computer, which specifically includes the following steps: (1.1) The data of each location to be measured is stored in the local database of the Raspberry Pi computer. After starting the program, the information in the local database will be compared with the actual measured data; (1.2) Each time the device is started, the indoor dimensions and the shape of a specific object are detected. If the detected location does not match any visited location in the local database, the location recognition module will add the location to the local database using a unique name.

4. The indoor location recognition method based on an indoor robot according to claim 1, characterized in that: In step (2), the first microcontroller controls the ultrasonic sensor to measure the indoor height. Specifically, the two ultrasonic sensors installed on the base respectively measure the distance h2 from the ultrasonic sensor to the ceiling h1 and to the ground. The sum of the two is the height of the room h=h1+h2.

5. The indoor location recognition method based on an indoor robot according to claim 1, characterized in that: The step (3) specifically includes the following steps: (3.1) After measuring the indoor height and determining that the room is indoors, the horizontal servo motor will rotate from the horizontal direction to the vertical direction at a preset step size. Every time a preset angle is reached, the laser radar scanning result is stored in a list Hread[n]; (3.2) Pick out the two maximum values from the distance list and put them into Vread[m], find the two distance peaks from Vread[m], and thus find the position closest to the ceiling corner; (3.3) Use this information and the cosine function from trigonometric functions to calculate the width and length of the room.

6. The indoor location recognition method based on an indoor robot according to claim 1, characterized in that: The step (4) specifically includes the following steps: (4.1) The computer Raspberry Pi controls the CMOS camera and uses OpenCV technology to identify special objects in the room and compare the height, width, length and shape information of the specific object in the room with the information stored in the database; (4.2) If the accuracy of the comparison result is greater than or equal to 95%, the identification result is obtained, and the purpose of identifying the location is achieved; (4.3) If the accuracy of the comparison result is less than 95%, then the surface accuracy is too low. In this case, use a CMOS camera to take a picture of the room and compare it with the database again to obtain the recognition result, thereby achieving the purpose of identifying the location.

Images