Multi-lidar autonomous network building system, method and terminal based on adaptive registration

Abstract

The present invention discloses a multi-laser radar autonomous network construction system, method and terminal based on adaptive registration, and relates to the technical field of network construction and network information processing. Multiple laser radars are used to register and fuse the point cloud images of different radars using a point cloud registration algorithm. The point cloud registration uses 4PCS for coarse registration and Trimmed ICP for fine registration, so that the laser radars in the same indoor environment automatically form a radar network and obtain the fused indoor environment point cloud. The data is saved locally and in the cloud at the same time, and the user changes the corresponding registration parameters or obtains the original point cloud data through the cloud. The present invention is a multi-laser radar autonomous network construction system based on adaptive registration designed for home-based elderly care projects. The user can remotely view the laser radar network status and indoor environment point cloud images in the elderly's residence. Through the radar network and environmental point cloud data, the user can use it in the further design and use of home-based elderly care projects.

Description

Technical Field

[0001] The present invention belongs to the technical field of network construction and network information processing, and in particular relates to a multi-laser radar autonomous network construction system and method based on adaptive registration, a storage medium for receiving user input programs, and an information data processing terminal. Background Art

[0002] At present, LiDAR has developed rapidly since the 1990s. LiDAR obtains information such as the coordinates of the environment by emitting lasers. Due to the characteristics of the laser itself, it is not easily affected by the external environment and can operate normally at night, with high reliability. Therefore, it has been widely used in various fields. Around 1960, 2D point cloud registration technology was first used. Later, with the development of technology and equipment and the deepening of inventions, 3D point cloud registration was gradually applied.

[0003] At present, 2D point cloud registration is mainly divided into two stages: rough registration and fine registration. The main purpose of rough registration is to roughly register two point clouds at the initial position, and provide a better initial position for fine registration. At present, the most common method for rough registration of point clouds is registration based on the idea of ​​RANSAC (Random Sample Consensus). It was first proposed by Fischler et al. in 1981. In the early 21st century, Aiger et al. proposed the 4PCS algorithm based on the RANSAC idea. The algorithm selects four coplanar points each time, finds approximately congruent four point sets according to rigid changes, and solves them as corresponding point sets. Although the 4PCS registration effect is good, the computational complexity is high and the efficiency is low. In order to solve this problem, Rusu et al. proposed the SAC-IA algorithm in 2009. The algorithm is based on RANSAC and FPFH descriptors, which significantly reduces the computational complexity and improves the registration efficiency. The most typical precise registration method is ICP. After years of invention and development after the first generation of ICP was proposed, Liu et al. proposed LA-ICP, and CHETVERIKOV proposed a registration method based on overlap rate, Trimmed ICP.

[0004] In recent years, domestic inventions for laser radar have become more and more abundant. Point cloud registration plays an irreplaceable role in many fields, such as: automatic reconstruction of simulation models, laser reconstruction of rock drilling, large-scale industrial measurement, building scanning, etc. Although there are already many software that provide manual calibration methods to accurately calibrate point clouds, the use of automatic registration is still needed in many places. Many scholars have made improvements and inventions to the laser radar point cloud registration algorithm. In 2007, Dai Jinglan and others began to invent ICP in registration. In 2009, Yang Xianhui improved the ICP algorithm in the invention of ICP registration application in 3D point clouds, so that ICP no longer directly globally registers but first calculates the curvature to obtain the corresponding point cloud feature points. In 2015, Han Baochang and others proposed a point cloud automatic registration method based on regional growth to solve the problem that point cloud global registration cannot effectively process point cloud data with low overlap rate. In 2018, Chen Xu and others proposed a point cloud coarse registration method to calculate the principal component vector of the point cloud and form a principal component coordinate system by coordinating its own center point.

[0005] Through the above analysis, the problems and defects of the prior art are as follows:

[0006] (1) The registration and fusion of multiple laser radars has always been a hot topic of invention, but most inventions pre-set key points as calibration parameters, or set the relative positions of the laser radars, and add manual operations for registration and fusion. Manual intervention in registration and fusion can achieve a good registration accuracy, but in some cases, automatic registration and fusion of multiple laser radars is required. However, the existing technology has poor automatic registration and fusion effects.

[0007] (2) The existing technology has poor accuracy in obtaining information on the laser radar network status of the elderly’s residence and the point cloud image of the indoor environment.

[0008] (3) The existing technology has poor protection for user privacy information.

[0009] (4) The integrity and reliability of existing technical data are poor.

[0010] The difficulty and significance of solving the above problems and defects are as follows:

[0011] LiDAR has its own unique advantages in obtaining environmental information. LiDAR is an active sensor that is less susceptible to external interference, has a large detection range, and high ranging accuracy. It can obtain the coordinate information of the current environment relatively quickly. Compared with computer vision, LiDAR is not affected by the external environment or light, has a smaller amount of data, is faster to calculate, consumes less resources, and is more convenient to use.

[0012] However, due to the limitations of a single laser radar, there will be occlusion and precision limitations in more complex environments, resulting in low recognition accuracy. The present invention uses multiple laser radars to independently build a network to form a radar network to form a small Internet of Things, which can reduce occlusion and improve recognition accuracy and range.

[0013] There are many methods for calibration and fusion of multi-lidar point cloud images, but most of the current calibrations are manually set key points or relative positions of radars. In some cases, manual calibration is impossible. The multi-lidar adaptive registration of the present invention can complete registration and point cloud fusion with good accuracy in unknown environments.

[0014] The present invention is aimed at home-based elderly care, and its purpose is to invent a simple and feasible solution to obtain the indoor point cloud environment. The point cloud environment is of great significance for further inventions in medical and elderly care. On the one hand, multi-lidar target recognition requires calibration and registration, and this design can provide effective parameters for target recognition; on the other hand, this design uses adaptive point cloud registration, which can realize the prerequisite for automated detection of elderly care.

[0015] At present, there are still many aspects to be improved in the point cloud registration algorithm, and it still faces many difficulties. For example, how to set the parameters of point cloud registration in an unknown environment; the image calibration and registration accuracy of different laser radars is low under low overlap; the initial posture of the point cloud has a great influence on the accuracy and effect of point cloud registration. This design can achieve a certain degree of adaptive point cloud registration in the indoor environment of the elderly at home and obtain better results.

[0016] At the same time, the invention has great social significance. Under the current severe challenge of the aging population, using computer technology to help the elderly reduce health risks and reduce social burdens is a very practical and promising invention topic. The present invention is based on laser radar, which is low-cost, simple to operate, light and compact, and has little impact on daily life. Monitoring the living environment of the elderly can not only facilitate understanding of the elderly's living conditions, but also obtain the elderly's living environment data, and further conduct a comprehensive analysis of the elderly's health status. In general, the present invention is easy to operate, practical and scalable, and has a more far-reaching significance in the field of walking analysis and health of the elderly. Summary of the invention

[0017] In order to overcome the problems existing in the related art, the embodiments disclosed in the present invention provide a multi-lidar autonomous network building system and method based on adaptive registration, a program storage medium for receiving user input, and an information data processing terminal.

[0018] The purpose of the present invention is to realize the autonomous network construction of multiple laser radars through point cloud registration. To achieve this goal, it is necessary to select a suitable solution, taking into account factors such as indoor monitoring environment, privacy protection, experimental simplicity and feasibility. The present invention will scan the indoor environment based on laser radar and other necessary hardware equipment, obtain the point cloud data of the environment, and register the point clouds of different radars, add the laser radars of the same indoor environment to the radar network, and use the established radar network to obtain the fused point cloud map of the current environment.

[0019] The present invention also aims to collect and process laser radar data. The data content of laser radar scanning is relatively large, including some useless information that needs to be discarded. In addition, due to factors such as the scanning accuracy limitation of laser radar, it is necessary to process the scanned point cloud.

[0020] Point cloud registration. It mainly includes two aspects: adaptive calculation of registration parameters, coarse registration and fine registration based on the registration parameters, and moving the target point cloud to the same coordinate system as the source point cloud through the transformation matrix to achieve point cloud registration and image fusion.

[0021] Self-built network. Based on point cloud registration, the lidars that can be successfully registered will be added to the radar network to form a radar network. Before point cloud registration, each machine will automatically update the IP and host name of each machine in the LAN and the ROS system registration.

[0022] The technical solution is as follows:

[0023] According to a first aspect of the disclosed embodiment of the present invention, a method for autonomously building a network of multiple laser radars based on adaptive registration is provided, comprising:

[0024] Step 1: The Raspberry Pi host modifies the registration parameters delta, overlap, and dist according to the approximate area of ​​the indoor environment calculated by the point cloud after data processing; first find the point A (x1, y1) with the largest x and y in the 2D point cloud, the point B (x2, y2) with the largest x and the smallest y, the point C (x3, y3) with the smallest x and the largest y, and the point D (x4, y4) with the smallest x and the smallest y; based on these four points, calculate the approximate area of ​​the point cloud (in m 2 ):

[0025]

[0026] Modify delta, overlap and dist according to the size of Area;

[0027] Step 2: multiple laser radars are used to register and fuse point cloud images of different radars using a point cloud registration algorithm. The point cloud images of different radars are registered, and the registration adopts coarse registration and fine registration, so that the laser radars in the same indoor environment automatically form a radar network and obtain a fused indoor environment point cloud;

[0028] Step three, the obtained fused indoor environment point cloud data is saved locally and in the cloud at the same time, and the user changes the corresponding registration parameters or obtains the original point cloud data through the cloud.

[0029] According to a second aspect of the disclosed embodiment of the present invention, a multi-lidar autonomous network building system based on adaptive registration is provided, comprising:

[0030] The laser radar module includes the operation of the laser radar, which is used to drive the laser radar and upload the acquired data;

[0031] Data processing module, used for processing and analyzing point cloud data and displaying processing results;

[0032] Cloud data module, used to update the IP and host name in the LAN, upload raw data, and obtain registration parameters;

[0033] Adaptive registration module, used to adjust registration parameters according to the environment, rough registration, and fine registration;

[0034] Point cloud visualization display module, used to display point cloud fusion display and visualization display;

[0035] Autonomous network construction module, used for automated network construction;

[0036] The data acquisition module is used to obtain the laser radar point cloud data and download and acquire it using the intelligent gateway download platform.

[0037] According to a third aspect of the disclosed embodiment of the present invention, there is provided a storage medium for receiving a program input by a user, wherein the stored computer program enables an electronic device to execute the method for autonomously building a network of multiple laser radars based on adaptive registration.

[0038] According to a fourth aspect of the disclosed embodiment of the present invention, there is provided an information data processing terminal, characterized in that the information data processing terminal comprises a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the multi-lidar autonomous network building method based on adaptive registration.

[0039] The technical solution provided by the embodiments disclosed in the present invention may have the following beneficial effects:

[0040] The goal of the present invention is to design a simple system to automatically build a radar network and reduce the preliminary preparation work of the inventor. Since the audience of the home-based elderly care system is the elderly, the simpler the system installation, the better. This system only needs to place the laser radar and Raspberry Pi in the corresponding position and turn them on, which greatly reduces the threshold for use. The present invention uses multiple laser radars to use a point cloud registration algorithm to align and fuse the point cloud images of different radars. The point cloud registration uses 4PCS for coarse registration and Trimmed ICP for fine registration, so that the laser radars in the same indoor environment automatically form a radar network and obtain the fused indoor environment point cloud. The data is saved locally and in the cloud at the same time, and the user can change the corresponding registration parameters or obtain the original point cloud data through the cloud.

[0041] The present invention automatically calculates the registration parameters required for point cloud registration according to the environment, and can complete point cloud registration in different indoor environments. The present invention has been tested in laboratory environments and actual elderly home environments, and can better complete point cloud registration and autonomous network construction.

[0042] Compared with the prior art, the advantages of the present invention further include:

[0043] The present invention is mainly designed for home-based elderly care projects. The system is based on adaptive registration and is used to build a multi-lidar autonomous network. Through this system, users can remotely view the lidar network status and indoor environmental point cloud images in the elderly's residence. Through the radar network and environmental point cloud data, users can use it in the further design and use of home-based elderly care projects.

[0044] The innovation of the present invention mainly includes the following aspects:

[0045] (1) The present invention is part of a comprehensive monitoring platform for home-based elderly care. The present invention provides automated assistance and data support for further inventions of home-based elderly care.

[0046] (2) LiDAR is highly reliable and is not affected by the light environment. It can be used at night. Moreover, the radar is small and easy to place. The data volume is small, and the speed of uploading to the cloud and computing processing is fast, and no privacy information will be exposed.

[0047] (3) The present invention realizes the autonomous network construction of LiDAR, which greatly facilitates the use of analysts. Cloud-edge collaboration ensures the integrity and reliability of data.

[0048] The present invention aims to solve the problem that the registration accuracy is not very high due to the hardware limitation of laser radar. By replacing the laser radar with more scanning points per second and higher scanning accuracy, a better registration effect can be achieved.

[0049] The multi-lidar autonomous network construction has high requirements on the placement of the radar. If the placement is not good, it may lead to misjudgment and network construction failure. The present invention further adopts the form of camera plus laser radar, which will achieve better results.

[0050] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0052] Figure 1 It is a flow chart of a method for autonomously building a network of multiple laser radars based on adaptive registration provided in an embodiment of the present invention.

[0053] Figure 2 It is a schematic diagram of a multi-lidar autonomous networking system based on adaptive registration provided in an embodiment of the present invention.

[0054] Figure 2 In: 1. LiDAR module; 2. Data acquisition module; 3. Data processing module; 4. Cloud data module; 5. Adaptive registration module; 6. Point cloud visualization module; 7. Autonomous network building module.

[0055] Figure 3 is a flow chart of an adaptive registration procedure provided by an embodiment of the present invention.

[0056] Figure 4 This is an architecture diagram of a multi-lidar autonomous networking system based on adaptive registration provided in an embodiment of the present invention.

[0057] Figure 5 This is a flowchart of the autonomous network construction provided by an embodiment of the present invention.

[0058] Figure 6 This is a pairing effect diagram of radar point cloud No. 1 and indoor environment standard point cloud provided by an embodiment of the present invention.

[0059] Figure 7 This is a pairing effect diagram of radar point cloud No. 2 and radar point cloud No. 1 provided in an embodiment of the present invention.

[0060] Figure 8 This is a pairing effect diagram of radar point cloud No. 4 and radar point cloud No. 1 provided in an embodiment of the present invention.

[0061] Fig. 9 This is a diagram of the fusion effect of radar point clouds No. 1, 2, and 4 provided in an embodiment of the present invention.

[0062] Fig.10This is a point cloud effect diagram of the laser radar scanning provided by an embodiment of the present invention.

[0063] Fig.11 This is a diagram showing the result of point cloud registration of machines 1 and 2, provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0064] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0065] 1. The method for autonomously building a network of multiple laser radars based on adaptive registration provided by the disclosed embodiment of the present invention includes:

[0066] Step 1: The Raspberry Pi host modifies the registration parameters delta, overlap, and dist according to the approximate area of ​​the indoor environment calculated by the point cloud after data processing; first find the point A (x1, y1) with the largest x and y in the 2D point cloud, the point B (x2, y2) with the largest x and the smallest y, the point C (x3, y3) with the smallest x and the largest y, and the point D (x4, y4) with the smallest x and the smallest y; based on these four points, calculate the approximate area of ​​the point cloud (in m 2 ):

[0067]

[0068] Modify delta, overlap and dist according to the size of Area;

[0069] Step 2, using multiple laser radars to register and fuse point cloud images of different radars using a point cloud registration algorithm, wherein the point cloud images of different radars are registered, and the registration adopts coarse registration and fine registration, so that the laser radars in the same indoor environment automatically form a radar network, and obtain a fused indoor environment point cloud;

[0070] Step 3: The obtained fused indoor environment point cloud data is saved locally and in the cloud at the same time, and the user changes the corresponding registration parameters or obtains the original point cloud data through the cloud.

[0071] 2. The technical solution of the present invention is further described below in combination with system requirement analysis.

[0072] 2.1 System overall requirements analysis

[0073] The present invention needs to design a system that can automatically form a radar network for the laser radars in the same indoor environment and obtain the fused point cloud map of the current indoor environment. The user only needs to place the laser radar in an appropriate position and turn it on to form a radar network, and then obtain the fused point cloud map of the indoor environment from the minio cloud or host.

[0074] 2.1.1 Overall business process

[0075] like Figure 1 As shown in the figure, the business process of this system includes laser radar data collection, data preprocessing, point cloud registration, image fusion, and result display. These processes represent the main working steps of the entire system. The data source of this system is laser radar. Before starting, multiple laser radars are placed in appropriate positions. The data scanned by the radar is saved in the host, and the data is uploaded to the cloud for cloud and local operations. Point cloud registration and image fusion are both run in the Raspberry Pi host, and the results are displayed by connecting the Raspberry Pi via VNC.

[0076] 2.1.2 Functional Requirements Analysis

[0077] (1) LiDAR data collection

[0078] This system collects data through multiple laser radars at the same time and transmits it to the host and the cloud. In order to simultaneously transmit and receive data sent by multiple slaves, ROS is used to implement multi-machine communication. ROS multi-machine communication is completed through ROS topics, nodes, and messages. Topic is equivalent to a transit area. The sender node sends a message to the topic, and the receiver node subscribes to the topic to obtain the sent message. The host receives the message sent by the slave by subscribing to the topic. After receiving the data, it is uploaded to the minio cloud at the same time.

[0079] (2) LiDAR data processing

[0080] The raw data of the laser radar obtained by this system cannot be directly used for the next step of point cloud registration due to the vibration and accuracy limitations of the laser radar itself. Data processing is required. Data processing first requires removing the moving points in the data, extracting the static points and removing all duplicate points in the static points. Then remove the outliers in the point cloud data.

[0081] (3) Point cloud registration and image fusion

[0082] After data preprocessing, the host point cloud and each slave point cloud are aligned in turn, and the lidars that can be aligned (in the same indoor environment) are added to the radar network. Finally, the scanning point clouds of each lidar in the radar network are fused to form a relatively complete indoor environment point cloud map.

[0083] (4) Visual display

[0084] This system uses PCL Visualizer and cloudcompare of PCL to achieve visualization. In order to show the registration effect, different colors are given to the source point cloud, target point cloud, and registered point cloud and displayed in the same window. Finally, the indoor environment point cloud image fused by the radar network is displayed.

[0085] 2.1.3 Non-functional requirements analysis

[0086] (1) Operating environment

[0087] This system requires multiple (more than three) 2D laser radars to collect data. The present invention uses the Delta-1A series laser radar produced by Shenzhen Shanchuan Robot Co., Ltd. Figure 2-1 The scanning frequency of this model of LiDAR is 6 Hz, that is, 6 frames of data are scanned in 1 second, the number of scanning points per second is 4000, and each frame of data is about 700. It is less affected by light and can operate normally even at night.

[0088] Multiple (consistent with the number of laser radars) Raspberry Pi 4B microcomputers are used to connect to the radar and perform corresponding calculations and storage. Raspberry Pi is a microcomputer whose system is based on Linux, but can complete the basic functions of a home PC. Using Raspberry Pi to drive the radar is more friendly and convenient for the environmental layout of the laser radar. At the same time, it can ensure the demand for continuous work. The present invention uses Raspberry Pi 4B+ model Raspberry Pi, the slave machine installs Ubuntu mate 20.04 server version and ROS noetic, and the host machine installs Ubuntu mate 20.04 desktop version, ROS noetic, SSH, realVNC and PCL 1.10. SSH is used to connect other slaves, realVNC is used to share the desktop to the PC under the Windows system for easy operation and use, and PCL is used for point cloud computing. A PC equipped with realVNC, SSH service, and XFTP is used for personal debugging data and alignment of Raspberry Pi 4B. All programs are compiled and run in Raspberry Pi 4B.

[0089] A cloud storage server MiniO, used to store the original point cloud data scanned by Delta-1A LiDAR and the preset initial parameters for point cloud registration. The size is more than 4GB.

[0090] (2) Performance requirements

[0091] The present invention requires the Delta-1A laser radar and the Raspberry Pi 4B+ to work continuously for more than 6 hours at the same time, ensuring that the laser radar can continuously scan the surrounding environment within a day, and the Raspberry Pi computer synchronizes and keeps uploading data files. Since this system only needs the latest indoor environment point cloud data, only the latest point cloud data needs to be saved, and the data size does not exceed 500kb, so the 8G memory Raspberry Pi can fully complete the storage and computing tasks.

[0092] (3) Portability requirements

[0093] The laser radar module of the present invention needs to be built in the indoor environment to be monitored and driven by a Raspberry Pi computer. The Raspberry Pi needs to be installed with Ubuntu MATE 20.04LTS system, ROS noetic, PCL 1.10 and other dependent packages.

[0094] (4) Reliability requirements

[0095] Due to the small amount of data, the data size of each laser radar scanning 500 frames is less than 100kb after processing, and the probability of data reception problems is low. In addition, if data reception errors occur, the system will automatically retransmit the data based on the errors.

[0096] 2.2 System Function Modules

[0097] The outline design of each functional module of this system. Figure 2 As shown, the overall modules of the entire adaptive registration system are as follows. The entire system is divided into a laser radar module 1, a data acquisition module 2, and a data processing module 3.

[0098] The laser radar module 1 includes the operation of the laser radar, mainly driving the laser radar and uploading the acquired data.

[0099] The data acquisition module 2 is used to obtain the laser radar point cloud data, and the intelligent gateway download platform is used for downloading and acquisition.

[0100] The data processing module 3 is the core of this system, which is mainly responsible for processing and analyzing point cloud data and displaying the processing results.

[0101] The cloud data module 4 is used to update the IP and host name in the local area network, upload the original data, and obtain the registration parameters.

[0102] The adaptive registration module 5 is used to adjust the registration parameters according to the environment, perform rough registration, and perform fine registration.

[0103] The point cloud visualization display module 6 is used to display point cloud fusion display and visualization display.

[0104] The autonomous network building module 7 is used for automatic network building.

[0105] 2.3 System Core Business Process

[0106] The core of this system is adaptive registration. The flowchart of the adaptive registration program is as follows: Figure 3 As shown in the figure, this figure describes the operation process of the core functions of the system. The main processes include reading the alignment parameters from the cloud, calculating the approximate area of ​​the room, adjusting the parameters according to the environment, coarse alignment, fine alignment, and determining whether to align.

[0107] 2.4 System Architecture

[0108] like Figure 4 As shown. The laser radar is used as the data source, and the Raspberry Pi slave obtains data from the laser radar and sends messages through ROS. The Raspberry Pi host and MiniO cloud subscribe to the messages sent by the slave. After receiving the message, the Raspberry Pi host processes the message to obtain the processed point cloud data. At the same time, the host obtains the initial registration parameters from the MiniO cloud. At this time, the approximate area of ​​the indoor environment is calculated, and the corresponding registration parameters are adjusted according to the area. The point cloud is roughly registered using the adjusted registration parameters. If the registration is successful, the initial position of the point cloud is already very good at this time, and then fine registration is used to improve the registration accuracy. After fine registration, image fusion and visualization are performed. Then, the network is autonomously built based on the registration results.

[0109] 3. The technical solution of the present invention is further described below in conjunction with the detailed design of the system.

[0110] 3.1 LiDAR Module 1 Design

[0111] The present invention utilizes a series of operations of laser radar to collect the required data, including the setting of laser radar, the startup of laser radar, the data transmission format, ROS multi-machine communication, etc.

[0112] 3.1.1 LiDAR placement

[0113] Although the laser radar has high accuracy, it is seriously affected by obstructions in the environment because it emits lasers in a straight line. Therefore, the location of the laser radar needs to be selected in a place with less obstruction and can scan most objects in the room. In addition, the adaptive point cloud registration of multiple laser radars requires the minimum point cloud overlap to be above 40%, so it is best for multiple laser radars to scan each other, so that the point cloud overlap will be relatively high and the point cloud registration effect will be better. Finally, the laser radar is placed near the wall and under the sofa, where it is less obstructed but can scan most of the environment. Finally, in order to avoid sweeping or other factors that may affect the position of the radar, the laser radar is placed in a small box composed of thin columns. The metal shell ensures that the laser radar will not be moved due to factors such as being stepped on, and the buckle inside allows the laser radar and Raspberry Pi to be well fixed, which will not block the laser radar and reduce the placement area.

[0114] 3.1.2 LiDAR Driver

[0115] Connect the Delta-1A LiDAR to the Raspberry Pi 4B+. There are three types of drivers for this type of LiDAR, including Windows, Linux, and ROS. Here we use the ROS version of the driver. Upload the driver to the Raspberry Pi via XFTP, install the corresponding driver, and open the driver port permissions. Then compile and run. After the build is passed, the LiDAR successfully scans the data and displays it in ROS.

[0116] 3.1.3 LiDAR Data Collection

[0117] The data acquisition of the laser radar in this system uses ROSbridge to obtain the point cloud data scanned by the laser radar driver, and uses custom messages to format the data. The Delta-1a laser radar model used in this system has a scanning frequency of 6hz and obtains about 4000 points per second. In order to obtain a better registration effect and reduce errors, it is planned to collect 2-3 minutes of data, about 500 frames.

[0118] 3.1.4 Raspberry Pi multi-machine communication

[0119] In order to realize the simultaneous communication between multiple machines of Raspberry Pi host and slave, ROS is used to complete the message transmission and communication between the master and slave. The communication method of ROS is the most important concept of ROS system. This system adopts the topic method, which is a point-to-point one-way communication method. The transmission process is basically as follows: both the sender node and the receiver node must register with the node manager, and then the sender will publish the topic, and the receiver will subscribe to the corresponding topic to realize ROS communication. Since multiple machines transmit data at the same time, data loss may occur. In order to ensure the integrity of the data, the host logs in to the slave through SSH without password, controls the order and time of sending the slave information, and thus avoids data transmission failure.

[0120] 3.2 Design of data processing module 3

[0121] Function: After obtaining the point cloud data information of the laser radar, how to process and reduce noise on the original data to obtain initial data with good point cloud registration.

[0122] 3.2.1 LiDAR Data Analysis

[0123] The data type information obtained by this laser radar scan is stored in sensor_msgs / num.msg. The specific data structure content is shown in Table 1 and Table 2. In this system, for this model of laser radar, each scan frame obtains about 690-700 point data. However, in the laser radar driver of this model, only 360 point data are retained, and all other data are discarded. Although this method reduces the amount of data and the calculation time to a certain extent, it greatly affects the accuracy of the later point cloud registration. Therefore, this system chooses to retain all data points, uses indefinite length ranges and angles arrays to save the distance and angle of the laser radar scan, uses timestamps to save the time of each frame, and names each radar scan data file as hostname through the ROS node name. Since some points in the range have values ​​of inf and 0, and these points are invalid, a loop judgment is used in the sender node to remove these invalid points.

[0124]

[0125] Since polar coordinates are difficult to calculate and represent, it is necessary to convert the default polar coordinate system of the lidar into a direct coordinate system. Before the sender node sends a message, it directly uses the above formula (3.1) to convert the scanned distance and angle into x, y coordinates in the rectangular coordinate system, and then transmits the converted point cloud data information through ROS for point cloud registration.

[0126] Table 1 Walking parameters

[0127]

[0128] Table 2 Header table

[0129]

[0130] 3.2.2 Static point extraction

[0131] First, the present invention preliminarily processes the original point cloud data and screens out most of the static points. The initial screening of the point cloud data can basically extract the points that are useful for the analysis of this system, eliminate the influence of the moving points on the experimental target, thereby reducing the difficulty of subsequent work and maintaining higher accuracy.

[0132] For extracting moving points, the main method is to find points whose positions remain unchanged for a period of time. Because the point cloud scanned by each LiDAR is the sum of the data scanned within a period of time, the specific method is as follows: scan 500 frames of data; retain the points that appear more than or equal to 40 times in the 500 frames of data and remove the repeated points. There are about 30,000-40,000 points in 500 frames. After extraction, the number of static points is sufficient to complete point cloud registration.

[0133] 3.2.3 Outlier Removal

[0134] For LiDAR, errors are inevitable in data collection. Due to the influence of the actual environment and the placement of the LiDAR, the density of the point cloud scanned by the radar is generally not uniform. In addition, the hardware factors of the LiDAR itself will also cause measurement errors. The outliers generated by these errors will affect the point cloud registration and need to be removed in advance. Outliers are different from the noise points generated by LiDAR scanning. Since noise points are generally uniform and have a low probability of occurrence, they have been deleted in the extraction of static points in Section 3.2.2. Outliers generally refer to abnormal data and isolated points.

[0135] Because of the characteristics of outliers, as its name suggests, outliers are low-density, so a statistical method can be used to traverse the laser radar scan point cloud and delete points that are far away from other points. Here, the statistical filtering (Statistical Outlier Removal) method is used to remove global outliers. The specific method is: traverse all points in the point cloud data and calculate the average distance from each point to its surrounding points. If the value of the average distance is greater than the given standard domain, the point is considered an outlier and is deleted from the point cloud. The standard domain is determined by the mean, standard deviation, and a given standard deviation multiple. Let the mean be μ, the standard deviation be σ, the coordinates of the nth point in the point cloud data be Pn (Xn, Yn, Zn), and the distance from this point to any point P m (Xm, Ym, Zm) is:

[0136]

[0137] The average distance formula of each point in the point cloud to any point is:

[0138]

[0139] The standard deviation formula of the distance from each point to any point in the point cloud is:

[0140]

[0141] Assume that the standard deviation multiple is std. The algorithm implementation requires two thresholds, k and std. When the average distance of the k points around a point is within the normal selected range (μ-σ·std, μ+σ·std), the point is retained. If it is not within the range, it is considered an outlier and deleted. In this system, 500 frames of point cloud data are statistically filtered together, and finally the filtered point cloud is used for registration.

[0142] 3.3 Cloud Data Module 4 Design

[0143] Function: How to upload the raw data scanned by Delta-1A LiDAR to the cloud and download the initial registration parameters from the cloud to the Raspberry Pi host.

[0144] 3.3.1 Data upload

[0145] When each laser radar scan is completed, the Raspberry Pi connected to the laser radar will save the data in text format, and convert the point cloud data in text format into the pcd data format specifically used for processing point clouds through txt to pcd. Then upload the file to the cloud through the interface between MiniO cloud and python3. If the cloud file already exists, delete the existing file and upload the latest original data.

[0146] 3.3.2 Obtaining cloud registration parameters

[0147] Before registration, the Raspberry Pi host first obtains the preset registration parameters from the MiniO cloud. The registration parameters can be adjusted by the user according to the specific environment and the previous registration results and then uploaded to the cloud. The preset registration parameters are shown in Table 3, and the statistical filtering parameters are shown in Table 4.

[0148] Table 3 Registration parameters

[0149]

[0150] Table 4 Statistical filtering parameters

[0151]

[0152] 3.4 Design of Adaptive Registration Module 5

[0153] Function: How to choose a better registration parameter for point cloud registration, rough registration, fine registration, judgment of registration, image fusion and display.

[0154] 3.4.1 Adaptive adjustment of registration parameters

[0155] Before registration, the Raspberry Pi host modifies the registration parameters delta, overlap, and dist according to the approximate area of ​​the indoor environment calculated by the point cloud after data processing. First, find the point A (x1, y1) with the largest x and y in the 2D point cloud, the point B (x2, y2) with the largest x and the smallest y, the point C (x3, y3) with the smallest x and the largest y, and the point D (x4, y4) with the smallest x and the smallest y. Based on these four points, calculate the approximate area of ​​the point cloud (in m 2 ):

[0156]

[0157] Modify delta, overlap and dist according to the size of the Area.

[0158] The following are the specific parameter settings:

[0159] If the Area is less than 10m 2 , it is considered a small room, delta is set to 0.02m, overlap is set to 0.65, and dist is set to 0.15. If Area is less than 20m 2 , it is considered a medium room, delta is set to 0.03m, overlap is set to 0.6, and dist is set to 0.25. If Area is less than 20m 2 , it is considered a large room, delta is set to 0.05m, overlap is set to 0.5, and dist is set to 0.4. If it is larger than 30m 2 , it will exceed the limit of LiDAR and will have a great impact on the registration. However, in reality, the distance between the elderly’s room is probably larger than 30m. 2 , so this system does not consider 30m 2 If you find that a certain set of registration parameters is better according to the actual situation, you can modify the parameters and upload them to the cloud. The latest registration parameters will be downloaded from the cloud before registration.

[0160] 3.4.2 Rough registration of point cloud based on 4PCS

[0161] Point cloud registration is to translate and rotate two laser radar scanned point clouds to the same coordinate system to achieve maximum overlap. Registration is divided into two steps: coarse registration and fine registration. Fine registration generally uses the ICP algorithm. Although the point cloud registration effect of the ICP algorithm is good, the initial position of the point cloud has a great influence on the registration effect. If the initial position is not good, the fine registration can easily fall into the local optimum and cause registration failure. Therefore, before fine registration, coarse registration is performed to preliminarily register the two point clouds. Coarse registration refers to the preliminary registration of two point clouds when the position of the point cloud cannot be determined and there is no manual calibration, so as to provide a better initial position for fine registration. There are mainly the following methods for coarse registration: PFH, 3Dsc, NDT, RANSC and 4PCS.

[0162] This system uses 4PCS for coarse registration. 4PCS (4-Points Congruent Sets) is a global registration algorithm that can achieve fast registration of point clouds with any initial position, and is very suitable for coarse registration of small point clouds. The general principle is as follows: Assuming the target point cloud P and the source point cloud Q, find the corresponding n (n>3) groups of points in the target point cloud P and the source point cloud Q, rotate and translate to solve T, and then select a group of T with the maximum overlap from the candidate n groups of T. 4PCS uses the affine invariance of the corresponding four-point group on the same plane (in fact, this system does not require affine invariance when using 2D point clouds) to find corresponding point pairs. The specific description is as follows: Find a coplanar four-point basis in the target point cloud P that meets the requirements of a long baseline (the registration parameter selected in Section 3.4 estimates that the minimum overlap rate is directly related to the baseline selection, and the larger the overlap rate, the longer the baseline). For example, select a coplanar four-point basis B = {a, b, c, d} and extract the topological information of B. According to the affine invariance of r1 and r2 in the rotation and translation of the point cloud and equation (3.6), the two proportional factors r1 and r2 between the coplanar four-point basis B are calculated. Then, according to equation (3.7), the intersection position set of q1, q2∈Q (source point cloud) is calculated to calculate the intersection coordinates of all long baseline point pairs in Q. The intersection coordinates are compared to determine the matching set. ei≈ej means that the corresponding consistent four points are found. i and j represent the i-th and j-th long baseline point pairs in Q respectively. The consistent four points corresponding to B={a,b,c,d} are expressed as C={q1,q3,q4,q5}. The set of coplanar four points of all calculated P in the point cloud is represented by E={B1,B2,...,Bm}, where m represents the number of P sets. Repeat the above steps to obtain the set of consistent four points D={C1,C2,...,Cn}, where n is the total number of consistent four point sets. Finally, the optimal congruent four-point pair is found in D = {C1, C2, ..., Cn}. 4PCS uses the LCP strategy to find the optimal congruent four-point match, that is, calculate the rotation and translation change parameters of the congruent four-points, apply the four-point transformation to the global point cloud transformation, and record the largest consistent area match in the global registration as the optimal match. This completes the local coarse registration work of the 4PCS algorithm.

[0163]

[0164]

[0165] Where e is the intersection of the four coplanar but non-colinear points a, b, c, d. r1 and r2 represent the ratio, q1 and q2 represent any point pair in the target point cloud Q, and e1 and e2 represent the possible intersection of the corresponding point pairs. If the difference between e1 and e2 is within the allowable range, it is considered to be the corresponding coplanar point of a, b, c, d.

[0166] 3.4.3 Point cloud precise registration based on TRICP

[0167] After the rough registration, the target point cloud and the source point cloud are already in a good initial position. The fine registration algorithm generally uses the ICP algorithm. This system uses the improved ICP algorithm Trimmed ICP. Since this algorithm is based on ICP, the ICP algorithm is introduced first.

[0168] ICP algorithm (Iterative Closest Point) is to calculate a change matrix to change the target point cloud to the source point cloud A. All changes are in the same Cartesian coordinate system.

[0169] ICP is divided into the following 6 steps:

[0170] 1) Point cloud normalization

[0171] Purpose: To select a standard and effective Cartesian coordinate system for point cloud data and eliminate the influence of coordinate transformation before registration.

[0172] The following formula (3.8) takes the center of the point cloud:

[0173]

[0174] Among them A i represents the coordinates of the i-th point in the A point cloud. Next, subtract this average value A from each point center , that is, to translate the point cloud so that the centroid of the point cloud moves to the origin.

[0175] 2) Find the nearest point (corresponding point)

[0176] Purpose: Find the corresponding points of two point cloud sets A and B. That is, after theoretically correct translation and rotation, point cloud B should overlap with point cloud A, and each pair of overlapping points is the corresponding point. In theory, the distance between corresponding points should be 0. However, in practice, there may not be overlapping points. In this case, the point with the closest distance is the corresponding point. The following formula (3.9) calculates the point with the closest distance between point clouds A and B.

[0177]

[0178] in Represents the x-coordinate of the i-th point in point cloud A.

[0179] 3) Calculate the change matrix (rotation angle, translation distance)

[0180] In step 2), all corresponding point pairs of A and B are saved. The following is to calculate the change matrix that minimizes the sum of the distances of these point pairs (that is, after rotating a certain angle and translating a certain distance, the sum of the distances of the point pairs is minimized). Let R be the change matrix, the distance function of the corresponding point pairs be D(R, t), and the sum of the distances of all corresponding points is as follows (3.10)

[0181]

[0182] The above formula (3.10) is expanded to the following formula (3.11):

[0183]

[0184] The above formula (3.11) is simplified to the following formula (3.12)

[0185] in represents the x-coordinate of the i-th point in point cloud A. The rotation radian is calculated as follows (3.13):

[0186]

[0187] in Represents the x-coordinate of the i-th point in point cloud A. The translation vector t is the difference between the coordinates of the center point of A and the coordinates of the center point of B. The above B is rotated, because after the rotation, the distance between point cloud B and A should be smaller.

[0188] 4) Change the point cloud by changing the matrix

[0189] The original point clouds A and B (before homogenization) are transformed by the change matrix R, which is the following formula (3.14):

[0190] B′=R·B+t (3.14)

[0191] R is the change matrix, and t is the translation vector. In fact, all points in the B point cloud are rotated by θ and translated by t. At this point, an ICP operation has been completed.

[0192] 5) Calculate the loss

[0193] Compare the changed point cloud B' with B, calculate the difference between the sum of the distances of the corresponding point pairs (closest points) of the two point clouds, and the Loss is calculated as follows (3.15).

[0194]

[0195] A i -B i ′ represents the distance between the corresponding points of two point clouds.

[0196] 6) Whether to continue iteration

[0197] Generally speaking, there are two conditions for ICP iteration: ① the loss is less than the preset loss threshold ② the number of iterations is greater than the preset number of iterations. After field testing, the loss threshold is not easy to test, and it is impossible to set a loss function relatively well, so the present invention only uses the number of iterations as a condition for determining whether to continue iteration. After field testing, in a simple indoor environment, the sum of the distances of about 30 point pairs no longer changes, so the number of iterations is set to about 50 times.

[0198] The principle of Trimmed ICP is as follows: for source point cloud A and target point cloud B with different number of points, some points in A may not have corresponding relationship in target point cloud B. Assume that the minimum proportion of the number of corresponding points that can be matched to all points is σ, and the set of all corresponding points is N P , the number of corresponding point pairs that can be matched is N po =σN P , when σ=1, Trimmed ICP is equal to ICP. The basic steps are the same as ICP, only in step 3) when calculating the distances of corresponding point pairs and D(R,t), the distances of each point pair calculated according to formula (3.9) are i ,b j} Sort in ascending order, select the first N po The selection of σ is the same as the overlap ratio of the coarse registration in Section 3.3.3.

[0199] 3.4.4 Registration effect judgment

[0200] After point cloud registration, the source point cloud needs to be determined whether it is registered with the target point cloud based on the dist in the registration parameters in Section 3.3.2. Since an incorrect registration will result in a larger point cloud distance, limiting the nearest point distance threshold to select corresponding points will cause incorrect point pairs to be eliminated, resulting in a smaller point cloud average distance. This will make it difficult to set dist. If dist is too small, even if the registration is correct, the average distance between the two point clouds will be greater than dist due to the influence of some points that have no corresponding relationship, and the registration will be judged to have failed. If dist is too large, the point cloud that originally failed to be registered will also be judged to be correctly registered. Therefore, this system sets the nearest point distance threshold to a larger value, and uses the idea of ​​TrimmedICP to select N. po =σN P The average distance between the target point cloud and the source point cloud is calculated, that is, the nearest point distance and dist corresponding to the registered point cloud are used to determine whether the registration is performed and the registration score. The evaluation index dist is calculated as follows (3.16):

[0201]

[0202] p i and q i Represents the distance between corresponding points of two point clouds.

[0203] 3.5 Point cloud visualization module 6 design

[0204] Function: How to fuse multiple point clouds based on the results of point cloud registration and visualize the point cloud map.

[0205] 3.5.1 Point Cloud Fusion

[0206] Due to factors such as laser radar scanning distance and occlusion, the image scanned by a single laser radar is not complete. It is necessary to fuse the data of multiple laser radars together to form a relatively complete indoor environment point cloud map. According to the transformation matrix R obtained by point cloud registration, the source point cloud is converted to the same coordinate system as the target point cloud through formula (3-14). At this time, since part of the data of the source point cloud and the target point cloud overlap, the density of the point cloud map is uneven and does not conform to the actual situation. Therefore, the uniform sampling method is used to process the point cloud to obtain a relatively uniform density and indoor environment fusion point cloud that conforms to the actual situation. After the point cloud is fused, the fused point cloud is used to align with the point cloud scanned by other laser radars, which can improve the effect and accuracy of point cloud registration.

[0207] 3.5.2 Visualization

[0208] This module uses an automated script to start ROS multi-machine communication. After receiving the data, the host automatically calculates the registration. According to the registration result, the successfully registered laser radar is added to the radar network, and finally the fused point cloud of the radar network is displayed (the matrix R obtained by the registration of the source point cloud is translated and rotated to the same coordinate system as the target point cloud). The specific business logic is as follows: It is mainly divided into two parts for display. The first is the display after a single point cloud registration, which is divided into source point cloud, target point cloud and registered point cloud. The second is to display the fused point cloud of the radar network after all point clouds are registered. The first part displays three point clouds in one viewer, and gives different colors to the three point clouds. The second part is divided into three viewers for display. One viewer displays the original point cloud of all laser radars, represented by different colors; the second viewer displays the image of point cloud fusion, giving different colors to different point clouds to indicate the effect of registration; the third viewer displays the standard point cloud of the indoor environment, uniformly using red point clouds to indicate the general environment of the room.

[0209] 3.6 Design of Self-built Network Module 7

[0210] Function: How to automatically build a radar network based on the results of point cloud registration.

[0211] First, you need to dynamically bind the hostname of each Raspberry Pi to the dynamic IP address distributed by DHCP and update the ROS master and slave settings. The steps are as follows:

[0212] (1) Each machine automatically connects to WIFI when it is turned on, and the router or gateway Raspberry Pi automatically assigns an IP address. Then the machine automatically runs a script as a ROOT user to download the hosts.txt file from the cloud. If the file includes the host name and IP address of the machine, the IP address of the machine in the cloud is updated to the latest local IP address. Otherwise, the host name and IP address of the machine are added to the end of hosts.txt. The file is then uploaded to the cloud. The cloud always maintains the latest hosts.txt file.

[0213] (2) Write the corresponding IP addresses and host names uploaded by all Raspberry Pis in the network segment into the / etc / hosts file of each host system. Then modify the master and slave settings of ROS. Write the obtained host IP to the end of / etc / profile. If ROS_MASTER_URI already exists, update it. Otherwise, write the obtained host IP.

[0214] (3) Write the local IP address to the end of / etc / profile. If ROS_HOSTNAME exists, update it. Otherwise, write the local IP address. Then register the system file and apply the settings.

[0215] Then the host uses an automated script to start ROS multi-machine communication. After receiving the data, the host automatically calculates the registration, adds the successfully registered lidar to the radar network according to the registration result, and finally displays the fused point cloud of the radar network (translates and rotates the matrix R obtained by the registration of the source point cloud to the same coordinate system as the target point cloud). The specific business logic is as follows:

[0216] 1) The host downloads hosts.txt from the cloud and obtains the IP address and host name of the Raspberry Pi that can be normally connected in the current network segment.

[0217] 2) Each radar in the radar network scans the current environment to obtain point cloud data.

[0218] 3) Determine whether it is the first time to start the machine, if so, proceed to 4), otherwise proceed to 5).

[0219] 4) Use the point cloud registration algorithm to compare the point cloud data of each slave with the point cloud data of the host. If the average distance between the two point clouds is greater than dist, the radar network is removed. At this point, the radar network of the indoor environment where the host is located is successfully established. At the same time, the first radar network point cloud standard map is established.

[0220] 5) Use the point cloud registration algorithm to register the host point cloud with the standard map in the radar network. If they are registered, proceed to 6); otherwise, the host is in a new indoor environment and proceed to 7).

[0221] 6) Use the point cloud registration algorithm to compare the point cloud data of each slave with the point cloud standard map in the radar network. If the average distance between the two point clouds is greater than dist, the radar network is removed. After registration, the updated radar network point cloud standard map is obtained and 8) is performed.

[0222] 7) Use the point cloud registration algorithm to compare the point cloud data of each slave with the point cloud data of the host. If the average distance between the two point clouds is greater than dist, the radar network is removed. At this point, the radar network of the indoor environment where the host is located is successfully established. At the same time, a new radar network point cloud standard map is established.

[0223] 8) Repeat the above steps every 2 hours to update the radar network point cloud standard map.

[0224] Self-built network flow chart, such as Figure 5 shown.

[0225] The data processing process of the laser radar of the present invention, after acquiring the radar data, respectively undergoes data processing, adaptive selection of registration parameters, coarse registration, fine registration and autonomous network building.

[0226] 4. The technical solution of the present invention is further described below in conjunction with system implementation.

[0227] 4.1 Implementation of LiDAR Module

[0228] How to implement the LiDAR module. Including:

[0229] First, install Ubuntu Mate 20.04 LTS on the Raspberry Pi 4B, and install the corresponding ROSnoetic version. Install the driver of the Delta-1A laser radar, and grant all operation permissions to ttyUSB0. Configure the ROS system, create a catkin workspace catkin_ws, configure the corresponding packuage.xml, write rostopic, the topic name is delta_lidar, write the sender node delta_lidar_node, the receiver node build_map.py, and then add the above custom messages and nodes to the CmakeList.txt file. Finally, register and compile the program, and the running node can successfully transmit messages. The data scanned by the laser radar is relatively complex, and the only useful data for this system are angle, distance and timestamp. Therefore, only the above useful data is sent in the sender node.

[0230] In order to ensure that the host can receive the point cloud data of the slaves at the same time, this system uses an automated script to connect to each slave using SSH to control the start and end time of the ROS node.

[0231] 4.2 Implementation of point cloud data processing

[0232] The specific implementation method for the preliminary processing of point cloud data mainly includes the extraction of static points and the removal of outliers.

[0233] 4.2.1 Implementation of static point extraction

[0234] The laser radar scans the map for about 2 minutes each time, with 500 frames of data and about 30,000-40,000 points. The method for extracting static points is as follows: traverse 500 frames of data. If a point appears more than n times, it is considered a static point, otherwise it is considered a dynamic point and deleted. After the traversal is completed, the duplicate points in the point cloud are deleted. After actual testing, it is found that when the value of n is 40, static points can be extracted better. After extraction, the number of non-duplicate static points retained is about 2,000-4,000 points, which can complete point cloud registration.

[0235] 4.2.2 Implementation of outlier removal

[0236] This system uses statistical filtering to remove outliers. Install PCL1.10 and other dependent packages. PCL (Point Cloud Library) includes the statistical filtering used in this system. Use pcl::StatisticalOutlierRemoval <pcl::pointxyz>Statistical filtering object, initially set meanK to 2, std to 1. When the mean distance of the k nearest neighbors of the judgment point is greater than the global 1 standard deviation + average distance (global distances mean and standard), it is an outlier. Since the number of points in the source point cloud A and the target point cloud B is not necessarily the same, and the number of points after statistical filtering cannot be determined, in order to ensure that the two point clouds have the same number of points during point cloud registration, statistical filtering is performed on the two point clouds at the same time and then 1000 points are randomly sampled. After testing, the number of points after statistical filtering of 500 frames of data is greater than 1000, so it is feasible. The parameters of statistical filtering also need to be adjusted according to the actual environment.

[0237] 4.3 Implementation of the Adaptive Registration Module

[0238] The adaptive registration module is implemented by / root / pcltest1 and pcltest2. First, the registration parameters are obtained from the cloud by / root / cloud11.py. This file will automatically upload point cloud data according to the passed parameters, and will automatically download the registration parameter file config.txt to the / root folder.

[0239] pcltest1 first assigns the two global variables filename and filename2 with the argv pointer of the main function according to the parameters passed in by the shell script, and then reads the registration parameter file downloaded from the cloud. Then, the function calarea() is used to calculate the approximate area of ​​the indoor environment based on the point cloud data. calarea calculates the coordinates of the four points on the edge of the point cloud, and then divides the quadrilateral into two triangles. The area of ​​the triangle is calculated by the function calculateTwoPointArea, and the sum of the areas of the two triangles is returned, that is, the approximate area. According to the size of the area, the module adjusts the minimum distance between the paired points and the average distance for judging whether to register. If the area is large, the two parameters will be increased proportionally, otherwise, they will be reduced proportionally. After adjusting the parameters, the outlier removal method will be called to remove the global outliers and obtain two 2D point clouds with the same number of points. Then, the 4PCS rough registration will be started, starting from the preset minimum overlap rate, and the registration will be performed. At the same time, the average distance between the two point clouds will be calculated by mydist(), and then the overlap rate will be increased by 0.1, and the calculation will be repeated three times. Select the registration with the smallest average distance between point clouds and keep the matrix_tras change matrix. Use pcl::transformPointCloud() to translate and rotate the source point cloud to the target point cloud through the change matrix, and perform Trimmed ICP fine registration on the successfully registered point cloud to obtain a better fusion effect. Then display the point cloud image. Compare the average distance between the point clouds after fine registration with the previously set registration parameter dist. If it is less than dist, the registration is considered successful and added to the radar network; otherwise, the registration fails.

[0240] 4.4 Difficulties of the Adaptive Registration Module

[0241] This system was finally tested in a real environment. Affected by the environment and hardware equipment, it encountered many difficulties. The main ones are as follows:

[0242] First, the accuracy of the laser radar, the scanning distance, and the number of scan points per second are not enough, which directly affects the effect of the registration. In fact, different radars are placed in almost the same position, and the point cloud images they see are 20-30cm apart. After some parts are registered, others are not. The reason is the scanning error and individual differences of the laser radar. This problem cannot be solved, and we can only select data with better measurements for registration.

[0243] Second, environmental factors are very complex. Due to its characteristics, laser radar is greatly affected by occlusion. If the laser radar is not placed in a good position, the image will be too different from other radar point cloud images. The point cloud registration algorithm will fail to register when processing point clouds with too low overlap rates. Therefore, when placing the laser radar, it needs to be placed in a place with less occlusion and able to scan most of the indoor environment.

[0244] 4.5 Realization of autonomous network construction

[0245] First, each Raspberry Pi will automatically start the script as the ROOT user after booting, download hosts.txt from the cloud, update their respective hosts files and / etc / profile files, and then upload the latest hosts.txt to the cloud. After the update is complete, in order to display the running process and results, the host needs to run an additional terminal and run the script in the terminal. And because the file is in the root folder, you need to enter a password, create a password file _PWD_TEMP_, compile it with bash, and then use the shell to automatically enter the password. Here, the pipeline method echo password|sudo-S apt-get update is used. Add the running script task to the ubuntu startup item. Run the script mytest.sh. The script will determine whether it is the first time to start the machine according to whether lidar.pcd (indoor environment fusion point cloud) exists. If it is the first time to start the machine, the host will download the latest hosts.txt from the cloud to obtain the host name and IP that can communicate normally under the current network segment, add the machines that can be connected to the radar network, and then use pcltest1 to align the host and slave point clouds, delete the radars that cannot be aligned from the radar network, fuse the point clouds that can be aligned, and use the fused point cloud to continue the slave point cloud alignment. When all alignments are completed, the host runs pcltest2 to display the hostnames of each host in the radar network and the fused point cloud of the indoor environment. Then every hour, the host will start the script to automatically update the radar network and fused point cloud.

[0246] 5. The technical solution of the present invention is further described below in conjunction with system testing and verification.

[0247] 5.1 Overall system testing

[0248] The whole system is tested to verify whether the deployment of the system can achieve the expected results. The system test results are shown in Table 5.

[0249] Table 5 System test table

[0250]

[0251] 5.2 Adaptive Registration Module Test

[0252] like Figure 6 As shown, this is the pairing effect diagram of radar point cloud No. 1 and standard point cloud of indoor environment.

[0253] like Figure 7 As shown, this is the pairing effect diagram of radar point cloud No. 2 and radar point cloud No. 1.

[0254] like Figure 8 As shown, this is the pairing effect diagram of radar point cloud No. 4 and radar point cloud No. 1.

[0255] like Fig. 9 As shown, this is the point cloud fusion effect diagram of radars 1, 2, and 4.

[0256] 5.3 Self-built network test

[0257] First, it is necessary to test whether the hosts and / etc / profile files are updated according to the dynamic IP assigned by the DHCP server, and whether the cloud has the latest hosts file. Through the present invention, the cloud hosts file is automatically updated to the latest host name and corresponding IP address. The corresponding host name and IP are automatically updated locally. The ROS master and slave settings are automatically updated locally.

[0258] After the configuration information automatic update test is completed, it is mainly divided into four situations for testing. The first: Machine No. 1 is turned on first, and the other machines No. 2 and No. 4 are turned on later, and placed in the same indoor environment; the second: After the radar network is established for the first time, machines No. 1 and No. 2 are moved, but still in the same room; the third: After the first radar network update, machine No. 4 is moved from the indoor environment to another environment. The first three are all tested in a laboratory environment. The laboratory environment is relatively open and has a large area, which is still a certain distance from the indoor environment where the elderly actually live. The fourth situation is that machines No. 1 and No. 4 independently build a network test in the elderly’s home to establish a radar network.

[0259] 5.3.1 Laboratory environment test

[0260] The test is divided into three situations: one is that the three radars are in the same room; one is that two of the radars are moved; and the last is that one radar is placed outdoors.

[0261] (1) Three radars are placed in the same room

[0262] First, turn on machine 1, then turn on machines 2 and 4 in turn. Open VNC to connect to machine 1. You can see that after machine 1 is turned on, it automatically opens the terminal and starts the script to build the radar network. Then, machine 1 and machine 2 start point cloud registration. The result is: the average distance of the point cloud is the smallest at an overlap rate of 0.7, and the registration is successful.

[0263] After the registration, the fused point cloud uses different colors to represent different point clouds, which is better to show the registration effect. Indoor environment fused point cloud, showing the indoor environment After all the registrations are completed, the host will display the hostname of the radar network.

[0264] (2) Move the two radar positions

[0265] Move the No. 1 and No. 2 LiDARs to the same location, but they are still in one room. After a while, the host automatically updates the radar network. The point cloud fusion image is basically unchanged, indicating that the indoor environment has not changed and the LiDARs are in one room.

[0266] Therefore, the individual radars in the radar network will not change.

[0267] (3) Place a radar outdoors

[0268] This time, move the No. 4 radar to another indoor environment and wait for the host to automatically update the radar network. The registration of No. 1 and No. 4 radars failed.

[0269] Since the registration of radar No. 4 failed, there are only radars 1 and 2 in the radar network, and the fused point cloud will only show the point clouds of 1 and 2.

[0270] The current radar network only has shu4b and shu4b2, and unit 4 has been deleted from the radar network.

[0271] 5.3.4 Actual home environment test for the elderly

[0272] This time, the test was conducted in the room of an elderly person living alone. In reality, two LiDARs were placed in the elderly person’s living room, one next to the table in the corner and the other under the sofa.

[0273] The point cloud scanned by LiDAR is as follows: Fig.10 The white part is the point cloud scanned by the radar of machine No. 1, and the gray part is the point cloud scanned by the radar of machine No. 2. The extra part of the point cloud scanned by machine No. 1 is the living room corridor of the elderly. Fig.11 The results of point cloud registration of machines 1 and 2 are shown. 1 and 2 are successfully registered, indicating that the two lidars are in the same room.

[0274] Experiments show that:

[0275] The present invention is mainly a multi-lidar autonomous network building system based on adaptive registration designed for home-based elderly care projects. This article introduces the implementation scheme of the present invention in an all-round way from the design and implementation aspects. Through this system, users can remotely view the lidar network status and indoor environmental point cloud images in the elderly's residence. Through the radar network and environmental point cloud data, users can use it in the further design and use of home-based elderly care projects.

[0276] The innovation of the present invention mainly includes the following aspects:

[0277] (1) The present invention is part of a comprehensive monitoring platform for home-based elderly care. The present invention provides automated assistance and data support for further inventions of home-based elderly care.

[0278] (2) LiDAR is highly reliable and is not affected by the light environment. It can be used at night. Moreover, the radar is small and easy to place. The data volume is small, and the speed of uploading to the cloud and computing processing is fast, and no privacy information will be exposed.

[0279] (3) The present invention realizes the autonomous network construction of LiDAR, which greatly facilitates the use of analysts. Cloud-edge collaboration ensures the integrity and reliability of data.

[0280] Due to the hardware limitations of the LiDAR, the registration accuracy is not very high. If the LiDAR with more scanning points per second and higher scanning accuracy is replaced, a better registration effect can be achieved.

[0281] The autonomous network construction of multiple laser radars has high requirements for the placement of the radars. If the placement is not good, it may lead to misjudgment and network construction failure. If the form of camera plus laser radar is used, better results may be achieved.

[0282] It should be understood that the present disclosure is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure should be limited by the appended claims.< / pcl::pointxyz>

Claims

1. A multi-lidar autonomous network building method based on adaptive registration, characterized in that: The multi-lidar autonomous network building method based on adaptive registration includes: Multiple laser radars are used to register and fuse point cloud images of different radars using a point cloud registration algorithm. The point cloud images of different radars are registered, and the registration adopts coarse registration and fine registration, so that the laser radars in the same indoor environment automatically form a radar network and obtain a fused indoor environment point cloud; The fused indoor environment point cloud data is saved locally and in the cloud at the same time, and users can change the corresponding registration parameters or obtain the original point cloud data through the cloud. Before registering the point cloud images of different radars, the Raspberry Pi host modifies the registration parameters delta, overlap, and dist according to the area of ​​the indoor environment of the point cloud after data processing; first find the point A (x1, y1) with the largest x and y in the 2D point cloud, the point B (x2, y2) with the largest x and the smallest y, the point C (x3, y3) with the smallest x and the largest y, and the point D (x4, y4) with the smallest x and the smallest y; calculate the area of ​​the point cloud based on these four points: Modify delta, overlap and dist according to the size of Area; the area of ​​the point cloud, in m 2 ; Specific parameter settings: If the Area is less than 10m 2 , it is considered a small room, delta is set to 0.02m, overlap is set to 0.65, and dist is set to 0.15; if Area is less than 20m 2 , it is considered a medium room, delta is set to 0.03m, overlap is set to 0.6, and dist is set to 0.25; if Area is less than 30m 2 , it is considered a large room, delta is set to 0.05m, overlap is set to 0.5, and dist is set to 0.4; if it is larger than 30m 2 , it exceeds the limitation of LiDAR; if a certain set of registration parameters is found to be better according to the actual situation, modify the parameters and upload them to the cloud; download the latest registration parameters from the cloud before registration; The point cloud registration algorithm used by the multiple laser radars includes: translating and rotating the two laser radar scanning point clouds to the same coordinate system to achieve maximum overlap; the point cloud registration includes: coarse point cloud registration based on 4PCS and fine point cloud registration based on TRICP; The coarse registration is: when the position of the point cloud cannot be determined and there is no manual calibration, the two point clouds are preliminarily registered to provide a good initial position for the fine registration; After rough registration, the target point cloud and the source point cloud are already in a good initial position; the improved ICP algorithm TrimmedICP is used for fine registration; After point cloud registration, the source point cloud needs to determine whether it is registered with the target point cloud based on the dist in the registration parameters.

2. The method for autonomously building a network of multiple laser radars based on adaptive registration according to claim 1, characterized in that: The 4PCS-based point cloud coarse registration includes: Find a coplanar four-point basis that meets the requirements of the long baseline in the target point cloud P; select a coplanar four-point basis B = {a, b, c, d}, and extract the topological information of B; according to the affine invariance of r1 and r2 in the rotation and translation of the point cloud and the formula Calculate the two proportional factors r1 and r2 between the four coplanar points B, and then use the formula Calculate the intersection position set of q1,q2∈Q, where Q represents the source point cloud, to calculate the intersection coordinates of all long baseline point pairs in Q, and compare the intersection coordinates to determine the matching set. ei≈ej means finding the corresponding consistent four points, i,j represent the i-th and j-th long baseline point pairs in Q respectively; the consistent four points corresponding to B={a,b,c,d} are represented by C={q1,q3,q4,q5}; the set of coplanar four points of all calculated point clouds P is represented by E={B1,B2,…,Bm}, where m is used Represents the number of P sets, repeat, and get the congruent four-point set D = {C1, C2, ..., Cn}, n is the total number of congruent four-point sets; finally, find the optimal congruent four-point pair in D = {C1, C2, ..., Cn}, 4PCS uses the LCP strategy to find the optimal congruent four-point match, that is, calculate the rotation and translation change parameters of the congruent four-point, apply the four-point transformation to the global point cloud transformation, record the largest consistent area match in the global registration as the optimal match, and complete the local coarse registration of the 4PCS algorithm; Where e is the intersection of the four coplanar but non-collinear points a, b, c, and d; r1 and r2 represent the ratio, q1 and q2 represent any pair of points in the target point cloud Q, and e1 and e2 represent the possible intersection of the corresponding point pairs; if the difference between e1 and e2 is within the allowable range, they are considered to be the corresponding coplanar points of a, b, c, and d.

3. The method for autonomously building a network of multiple laser radars based on adaptive registration according to claim 1, characterized in that: The improved ICP algorithm TrimmedICP includes: (1) Point cloud normalization: Select a standard and effective Cartesian coordinate system for the point cloud data to eliminate the influence of coordinate transformation before registration; According to the formula Take the center of the point cloud; Among them A i Represents the coordinates of the i-th point in the A point cloud; next, subtract this average value A from each point center , that is, to translate the point cloud so that the centroid of the point cloud moves to the origin; (2) Find the nearest point: Find the corresponding points of the two point clouds A and B. That is, after a theoretically correct translation and rotation, the point cloud B should overlap with the point cloud A. Each pair of overlapping points is a corresponding point. The point with the shortest distance is the corresponding point. Calculate the point closest to the point cloud space of A and B; Represents the x-coordinate of the i-th point in point cloud A; (3) Calculate the change matrix: Let R be the change matrix, the distance function of corresponding points be D(R, t), and the distances of all corresponding points are as follows: After opening After simplification, in Represents the x-coordinate of the i-th point in point cloud A; the rotation radian is calculated as follows in represents the x-coordinate of the i-th point in point cloud A; the translation vector t is the difference between the coordinates of the center point of A and the coordinates of the center point of B; B above is rotated, because after rotation, the distance between point cloud B and A should be smaller; (4) Transforming the point cloud by the transformation matrix: The original point clouds A and B are transformed by the transformation matrix R, that is, B′=R·B+t; R is the transformation matrix, and t is the translation vector. In fact, all points in the point cloud B are rotated by θ and then translated by vector t. At this point, an ICP operation has been completed. (5) Calculate the loss Compare the changed point cloud B' with B, and calculate the difference between the sum of the distances of the corresponding point pairs of the two point clouds. The loss is calculated as follows: A i -B′ i Represents the distance between corresponding points of two point clouds; (6) Whether to continue iteration: The number of iterations is used as the condition for determining whether to continue iteration. Specifically, for source point cloud A and target point cloud B with different numbers of points, some points in A may not have a corresponding relationship in target point cloud B. Assuming that the minimum proportion of the number of corresponding points that can be matched to all points is σ, the set of all corresponding points is N. P , the number of matching corresponding point pairs is N po =σN P , when σ=1, Trimmed ICP is equal to ICP.

4. The method for autonomously building a network of multiple laser radars based on adaptive registration according to claim 1, characterized in that: The nearest point distance and dist of the registered point cloud are used to determine whether the registration is performed and the registration score; the evaluation index dist is calculated as follows: p i and q i Represents the distance between corresponding points of two point clouds.

5. The method for autonomously building a network of multiple laser radars based on adaptive registration according to claim 1, characterized in that: The point cloud images of different radars are registered and fused, including: LiDAR data collection and processing; The registration and fusion of point cloud images of different radars include: Autonomous network building. On the basis of point cloud registration, the lidars that can be successfully registered will be added to the radar network to form a radar network. Before point cloud registration, each machine automatically updates the IP and host name of each machine in the local area network as well as the ROS system registration.

6. A multi-lidar autonomous network building system based on adaptive registration that implements the multi-lidar autonomous network building method based on adaptive registration as described in any one of claims 1 to 5, characterized in that: The multi-lidar autonomous network building system based on adaptive registration includes: The laser radar module includes the operation of the laser radar, which is used to drive the laser radar and upload the acquired data; Data processing module, used for processing and analyzing point cloud data and displaying processing results; Cloud data module, used to update the IP and host name in the LAN, upload raw data, and obtain registration parameters; Adaptive registration module, used to adjust registration parameters according to the environment, rough registration, and fine registration; Point cloud visualization display module, used to display point cloud fusion display and visualization display; Autonomous network construction module, used for automated network construction; The data acquisition module is used to obtain the laser radar point cloud data and download and acquire it using the intelligent gateway download platform.

7. A storage medium for receiving user input programs, wherein the stored computer program enables an electronic device to execute the multi-lidar autonomous network building method based on adaptive registration as described in any one of claims 1 to 5.

8. An information data processing terminal, characterized in that: The information data processing terminal includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the multi-lidar autonomous network building method based on adaptive registration as described in any one of claims 1 to 5.

Images