Mobile monitoring platform based on thermal imaging camera

Abstract

The invention discloses a mobile monitoring platform based on a thermal imaging camera. The mobile monitoring platform comprises a mobile robot mechanical chassis, an upper computer, a multi-line laser radar and the thermal imaging camera. The method is suitable for mobile robots. The problem that the monitoring platform is difficult to apply to the mobile robot is solved; secondly, path planning and navigation obstacle avoidance are difficult to carry out on the robot; then, target object identification and temperature extraction are difficult to complete; and finally, the identification work of the target object is difficult to complete by using a thermal imaging camera in an industrial scene. By adopting a modular structure, later disassembly and maintenance are facilitated, and installation and use are convenient; a pine forest four-wheel type chassis is adopted, the mechanical strength is high, and road blocks such as deceleration strips can be conveniently crossed; a radium multi-line laser radar is adopted, so that a real three-dimensional environment can be simulated; ambient light and a temperature field are measured by using a Haikangwei visual thermal imaging camera, and temperature extraction of a target object can be completed by using a multi-mode detection mode.

Description

Technical Field

[0001] The invention belongs to the technical field of mobile robots, and in particular is a mobile monitoring platform based on a thermal imaging camera. Background Art

[0002] Mobile robots, as an outstanding representative of the development of modern science and technology, have rich technical background and application prospects behind them. Mobile robots have a high degree of autonomy and flexibility, and can perform autonomous navigation, path planning, target recognition and other operations according to different environments and task requirements. At the same time, with the continuous development of artificial intelligence technology, mobile robots also have the ability to learn, adapt and evolve, and can continuously improve their performance and intelligence level. Compared with traditional temperature measurement technology, thermal imaging temperature measurement technology has the advantages of non-contact temperature measurement, rapid temperature measurement, high-precision temperature measurement, and long-distance temperature measurement. In industrial production, thermal imaging temperature measurement technology can be used to detect the operating status and temperature distribution of equipment, and improve the operating efficiency and safety of equipment. Multimodal detection technology usually includes data preprocessing, feature extraction, multimodal fusion and classification recognition. At present, there are some large multimodal models that can simultaneously process multiple types of information such as text, pictures, audio and video, showing a higher level of intelligence and flexibility.

[0003] The existing technology has the following problems: first, it is difficult to apply the monitoring platform on the mobile robot; second, it is difficult to plan the path of the robot and navigate to avoid obstacles; then, it is difficult to complete the target object recognition and temperature extraction; finally, it is difficult to complete the target object recognition work using thermal imaging cameras in industrial scenarios. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a mobile monitoring platform based on a thermal imaging camera, which is used to solve the following technical problems:

[0005] First, it is difficult to apply the monitoring platform on mobile robots. Second, it is difficult to plan the path of the robot and navigate to avoid obstacles. Then, it is difficult to complete target object recognition and temperature extraction. Finally, it is difficult to complete target object recognition using thermal imaging cameras in industrial scenarios.

[0006] To solve the above problems, the first aspect of the present invention provides a mobile monitoring platform based on a thermal imaging camera, comprising: a mobile robot mechanical chassis, a host computer, a multi-line laser radar and a thermal imaging camera; the mobile robot mechanical chassis, the multi-line laser radar and the thermal imaging camera are all connected to the host computer by wired means; the host computer serves as a central processing unit and is responsible for the system scheduling of the entire monitoring platform; the control of the robot mechanical chassis and the processing of the thermal imaging data are completed according to the data signal uploaded by the sensor; the motion planning and obstacle avoidance of the mobile robot and the identification of the target object are completed according to the processing results, including the following modules:

[0007] Data acquisition module: collects thermal imaging images, visible light images and multi-line lidar data based on sensors;

[0008] Data processing module: pre-processing of thermal imaging images and visible light images, including: denoising, image enhancement and correction; pre-processing of multi-line lidar data, including: filtering denoising, data cleaning and standardization;

[0009] Feature extraction module: extract features from processed thermal imaging images, visible light images and multi-line lidar data; generate multimodal feature vectors based on feature extraction results;

[0010] Data analysis module: adjusts the robot's motion state through control algorithms to control the mechanical chassis; builds a three-dimensional model of the environment based on multi-line laser radar and analyzes the robot's optimal path for navigation and obstacle avoidance; uses multi-modal detection methods to identify and detect targets in the temperature range.

[0011] As a further solution of the present invention, extracting features from the processed thermal imaging image, visible light image and multi-line laser radar data includes the following steps:

[0012] Extract features from thermal imaging images, including temperature distribution features, hot spot features, and edge features; extract features from visible light images, including color features, texture features, and shape features; extract features from multi-line lidar data, including three-dimensional shape features, distance features, and reflection intensity features.

[0013] As a further solution of the present invention: generating a multimodal feature vector according to the feature extraction result includes the following steps:

[0014] Normalize the features extracted from thermal imaging images, visible light images and multi-line lidar data for preprocessing;

[0015] Perform feature-level fusion based on the preprocessed features, splice the features extracted from the thermal imaging image, visible light image, and multi-line lidar data to generate a long feature vector;

[0016] The long feature vector is reduced in dimension through principal component analysis to generate multimodal feature vector.

[0017] As a further solution of the present invention: constructing a three-dimensional model of the environment according to the multi-line laser radar and analyzing the best path of the robot for navigation and obstacle avoidance, including the following steps:

[0018] After the robot is started, it uses a multi-line laser radar to navigate and avoid obstacles, and converts the data of the multi-line laser radar into point cloud data, where each point represents the intersection of a laser beam and an obstacle; the point cloud data includes: the location, shape and size of the obstacle;

[0019] Before the robot moves, the global map is simulated to build a three-dimensional model of the environment; by processing the point cloud data, the ground point cloud and non-ground point cloud are extracted; mapping is performed based on the three-dimensional point cloud map to obtain a two-dimensional grid map;

[0020] Based on the extracted ground point cloud and non-ground point cloud, the robot is path-planned; using the path planning algorithm, the best path from the current position to the target position while avoiding obstacles is searched in the two-dimensional grid map;

[0021] According to the two-dimensional grid map mapped from the three-dimensional point cloud map, the obstacle avoidance effect in the three-dimensional environment is simulated. All point clouds within the robot's height range will be regarded as obstacles, forming the robot's forward motion channel; in the two-dimensional grid map, all point clouds within the robot's height range will be marked as obstacles;

[0022] By analyzing the formula: Y = {(x,y)|h(x,y)≤H}

[0023] Obtain the obstacle point set Y, where H represents the height of the robot and h(x,y) represents the height of each point (x,y) in the two-dimensional grid map.

[0024] As a further solution of the present invention: including: the sensors used all support working in a high temperature environment.

[0025] As a further solution of the present invention: including: the internal wiring of the platform adopts the USB-CAN wiring method.

[0026] As a further solution of the present invention: a multi-modal detection method is used to identify and detect targets in the temperature region, including the following steps:

[0027] According to the collected and processed visible light images, the target objects in the temperature area are identified or classified by using the target detection model;

[0028] Use the COCO dataset to annotate objects in visible light images. The annotations include the category of the object and the X and Y coordinates of the object annotation box. Input the annotated dataset into the object detection model for training.

[0029] By collecting and processing visible light images in real time, the real-time visible light images are input into the trained target detection model, the targets in the visible light images are identified or classified in real time, and the X coordinates and Y coordinates of the target annotation box in the visible light image are output; after the target is identified and detected in the visible light channel, the X coordinates and Y coordinates of the target annotation box are synchronized to the thermal imaging channel; by synchronizing the annotations of the visible light and thermal imaging channels, the target is identified in the thermal imaging channel;

[0030] Based on the recognition results of the target object, the temperature value of the target object is predicted using the deep learning model;

[0031] Labeling multimodal feature vectors, and inputting the labeled multimodal feature vectors into a deep learning model for training;

[0032] According to the trained deep learning model, real-time multi-modal feature vectors are extracted by real-time acquisition and processing of thermal imaging images, visible light images and multi-line lidar data; the real-time multi-modal feature vectors are input into the trained deep learning model to predict the temperature value of the target object and output the result;

[0033] According to the recognition result of the target object, the actual temperature value of the target object is read in real time by a thermal imaging camera; the actual temperature value is compared and corrected with the predicted temperature value to obtain the temperature value of the target object.

[0034] As a further solution of the present invention: the mobile monitoring platform has a DC power supply provided in its mechanical structure to directly supply power to each sensor.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] The present invention adopts a modular structure, which is convenient for later disassembly and maintenance, and is easy to install and use. The Songlin four-wheel chassis has high mechanical strength and is easy to cross obstacles such as speed bumps. The Rayshen multi-line laser radar can simulate a real three-dimensional environment. The Hikvision thermal imaging camera is used to measure the ambient light and temperature field, and the temperature of the target object can be extracted using a multi-modal detection method at the same resolution. The USB-CAN wiring method is used to avoid errors caused by various electromagnetic interferences in industrial scenarios. The use of mobile power supply avoids the complicated application procedures caused by wiring in the factory area, and can also flexibly select the measurement location, which also improves safety to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 It is a flow chart of mobile monitoring of the present invention;

[0039] Figure 2 It is a structural schematic diagram of the present invention;

[0040] Figure 3 It is an example diagram of the monitoring results of the present invention. DETAILED DESCRIPTION

[0041] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] See also Figure 1-Figure 3 As shown, the first embodiment of the present invention provides a mobile monitoring platform based on a thermal imaging camera, including: a mobile robot mechanical chassis, a host computer, a multi-line laser radar and a thermal imaging camera; the mobile robot mechanical chassis, the multi-line laser radar and the thermal imaging camera are all connected to the host computer by wired means; the host computer is used as a central processing unit and is responsible for the system scheduling of the entire monitoring platform; the robot mechanical chassis is controlled and the thermal imaging data is processed according to the data signal uploaded by the sensor; the motion planning and obstacle avoidance of the mobile robot and the identification of the target object are completed according to the processing result, including the following modules:

[0043] Data acquisition module: collects thermal imaging images, visible light images and multi-line lidar data based on sensors;

[0044] Data processing module: pre-processing of thermal imaging images and visible light images, including: denoising, image enhancement and correction; pre-processing of multi-line lidar data, including: filtering denoising, data cleaning and standardization;

[0045] Feature extraction module: extract features from processed thermal imaging images, visible light images and multi-line lidar data; generate multimodal feature vectors based on feature extraction results;

[0046] Data analysis module: adjusts the robot's motion state through control algorithms to control the mechanical chassis; builds a three-dimensional model of the environment based on multi-line laser radar and analyzes the robot's optimal path for navigation and obstacle avoidance; uses multi-modal detection methods to identify and detect targets in the temperature range.

[0047] Specifically, the power of the Songlin four-wheel chassis of the mobile robot, the host computer, the Leishen multi-line laser radar and the Hikvision thermal imaging camera is turned on, the host computer is started, and the monitoring platform is loaded and run. The target point is set in the host computer. The working parameters of the thermal imaging camera, visible light camera and multi-line laser radar are configured. The multi-line laser radar starts to scan the surrounding environment and generates a three-dimensional model of the environment. The host computer preprocesses the laser radar data through the data processing module, including filtering and denoising, data cleaning and standardization. According to the three-dimensional model, the data analysis module analyzes and calculates the best path from the current position to the target point. The host computer adjusts the motion state of the robot through the control algorithm and controls the mechanical chassis to move forward along the planned path. During the movement, the environment model is continuously updated through the laser radar data, and the path is adjusted to avoid new obstacles. When the robot reaches the target point, the host computer receives a position confirmation signal. The host computer controls the camera's pan / tilt to lift and adjusts the camera angle to cover the target area. The dual-channel data acquisition function of ambient light and temperature field is started. The thermal imaging camera and the visible light camera start to collect image data at the same time. The data acquisition module uploads the collected thermal imaging images, visible light images and multi-line laser radar data to the host computer. The data processing module performs preprocessing operations such as denoising, image enhancement and correction on the image data. The laser radar data is further filtered, denoised and cleaned. The feature extraction module extracts features from the processed thermal imaging images, visible light images and multi-line laser radar data and generates multimodal feature vectors. The data analysis module uses a multimodal detection method to identify and detect targets in the temperature area. Target detection is performed using deep learning algorithms or image processing algorithms. After the target detection is completed, the camera reading ends, and the host computer can plan the return path, control the robot to return to the initial target point and save the monitoring data to the folder of the current date.

[0048] In one embodiment of the present invention, extracting features based on processed thermal imaging images, visible light images, and multi-line laser radar data includes the following steps:

[0049] Extract features from thermal imaging images, including temperature distribution features, hot spot features, and edge features; extract features from visible light images, including color features, texture features, and shape features; extract features from multi-line lidar data, including three-dimensional shape features, distance features, and reflection intensity features.

[0050] Specifically, features are extracted from thermal images, including but not limited to: temperature distribution features, hot spot features and edge features; by using image processing software or algorithms, the temperature field of thermal images is analyzed, and the temperature distribution features of different areas in the image are obtained through color mapping or numerical representation. The hot spot area is identified and extracted by using the segmentation algorithm in image processing. The edge is the boundary area where the temperature changes in the thermal imaging image, and the edge detection algorithm is used to extract the edge features in the image. Features are extracted from visible light images, including but not limited to: color features, texture features and shape features; the visible light image is converted from the RGB color space to a color space more suitable for color feature extraction, including: HSV or Lab color space. The color features in the image are extracted using methods such as color histogram, color moment or color aggregation vector. Texture features in the image are extracted using texture analysis algorithms, including: gray level co-occurrence matrix, local binary pattern, etc. Shape features in the image are extracted using shape analysis algorithms, including: contour extraction, shape matching or shape descriptor, etc. Features are extracted from multi-line laser radar data, including but not limited to: three-dimensional shape features, distance features and reflection intensity features. Use the point cloud data of the LiDAR to build a 3D model or mesh. Use 3D shape analysis algorithms, including 3D reconstruction, surface fitting or shape matching, to extract 3D shape features. LiDAR can directly measure the distance between an object and the sensor. By processing the point cloud data of the LiDAR, the distance features between different points can be extracted. The reflection intensity of the LiDAR is related to the surface material and roughness of the object. The point cloud data of the LiDAR is processed to extract the reflection intensity features of each point.

[0051] In one embodiment of the present invention, generating a multimodal feature vector according to the feature extraction result includes the following steps:

[0052] Normalize the features extracted from thermal imaging images, visible light images and multi-line lidar data for preprocessing;

[0053] Perform feature-level fusion based on the preprocessed features, splice the features extracted from the thermal imaging image, visible light image, and multi-line lidar data to generate a long feature vector;

[0054] The long feature vector is reduced in dimension through principal component analysis to generate multimodal feature vector.

[0055] Specifically, the extracted features are normalized to eliminate the dimensional differences and numerical range differences between different features. The normalized thermal imaging image features, visible light image features, and multi-line lidar data features are spliced ​​to generate a long feature vector. The spliced ​​long feature vector is used as the input data set for principal component analysis. The data set is subjected to principal component analysis to find the main components or feature directions in the data. The number of principal components to be retained is determined by calculating the variance contribution rate of each principal component. The principal components with higher variance contribution rates can be selected to retain most of the information of the original data. According to the number of principal components to be retained, the long feature vector is subjected to dimensionality reduction to generate a multimodal feature vector.

[0056] In one embodiment of the present invention, constructing a three-dimensional model of the environment based on a multi-line laser radar and analyzing the optimal path of the robot for navigation and obstacle avoidance includes the following steps:

[0057] After the robot is started, it uses a multi-line laser radar to navigate and avoid obstacles, and converts the data of the multi-line laser radar into point cloud data, where each point represents the intersection of a laser beam and an obstacle; the point cloud data includes: the location, shape and size of the obstacle;

[0058] Before the robot moves, the global map is simulated to build a three-dimensional model of the environment; by processing the point cloud data, the ground point cloud and non-ground point cloud are extracted; mapping is performed based on the three-dimensional point cloud map to obtain a two-dimensional grid map;

[0059] Based on the extracted ground point cloud and non-ground point cloud, the robot is path-planned; using the path planning algorithm, the best path from the current position to the target position while avoiding obstacles is searched in the two-dimensional grid map;

[0060] According to the two-dimensional grid map mapped from the three-dimensional point cloud map, the obstacle avoidance effect in the three-dimensional environment is simulated. All point clouds within the robot's height range will be regarded as obstacles, forming the robot's forward motion channel; in the two-dimensional grid map, all point clouds within the robot's height range will be marked as obstacles;

[0061] By analyzing the formula: Y = {(x,y)|h(x,y)≤H}

[0062] Obtain the obstacle point set Y, where H represents the height of the robot and h(x,y) represents the height of each point (x,y) in the two-dimensional grid map.

[0063] Specifically, the global map needs to be built before the robot moves. The SLAM part uses the open source Lego Loam algorithm. Before the algorithm is compiled and run, the laser radar and inertial navigation need to be jointly calibrated. Here, lidar_a l ign is used for external parameter calibration. After Lego Loam mapping, the ground point cloud and non-ground point cloud can be clearly extracted; the path planning part uses the DWA path planning algorithm combined with the NDT positioning algorithm. The two-dimensional grid map mapped from the three-dimensional point cloud map can achieve the obstacle avoidance effect in a simulated three-dimensional environment. During the mapping process, the octmap function package is used to perform a straight-through filter on the height of the robot. The point clouds above the robot will be filtered out, and the remaining point clouds will be mapped downward. The point clouds within the height range of the robot will be regarded as obstacles, thus forming a forward motion channel for the robot. As a positioning algorithm for the robot's forward movement, the NDT algorithm can have more position information than the two-dimensional positioning algorithm.

[0064] In one embodiment of the present invention, the sensors used all support working in a high temperature environment.

[0065] Specifically, the sensors used all support working in high temperature environments and can adapt to most factory environments.

[0066] In one embodiment of the present invention, the internal wiring of the platform adopts a USB-CAN wiring method.

[0067] Specifically, the USB-CAN wiring method is adopted to avoid the errors caused by various electromagnetic interferences in the measurement in industrial scenarios.

[0068] In one embodiment of the present invention, a multi-modal detection method is used to identify and detect targets in a temperature region, including the following steps:

[0069] According to the collected and processed visible light images, the target objects in the temperature area are identified or classified by using the target detection model;

[0070] Use the COCO dataset to annotate objects in visible light images. The annotations include the category of the object and the X and Y coordinates of the object annotation box. Input the annotated dataset into the object detection model for training.

[0071] By collecting and processing visible light images in real time, the real-time visible light images are input into the trained target detection model, the targets in the visible light images are identified or classified in real time, and the X coordinates and Y coordinates of the target annotation box in the visible light image are output; after the target is identified and detected in the visible light channel, the X coordinates and Y coordinates of the target annotation box are synchronized to the thermal imaging channel; by synchronizing the annotations of the visible light and thermal imaging channels, the target is identified in the thermal imaging channel;

[0072] Based on the recognition results of the target object, the temperature value of the target object is predicted using the deep learning model;

[0073] Labeling multimodal feature vectors, and inputting the labeled multimodal feature vectors into a deep learning model for training;

[0074] According to the trained deep learning model, real-time multi-modal feature vectors are extracted by real-time acquisition and processing of thermal imaging images, visible light images and multi-line lidar data; the real-time multi-modal feature vectors are input into the trained deep learning model to predict the temperature value of the target object and output the result;

[0075] According to the recognition result of the target object, the actual temperature value of the target object is read in real time by a thermal imaging camera; the actual temperature value is compared and corrected with the predicted temperature value to obtain the temperature value of the target object.

[0076] Specifically, the COCO dataset is used to annotate the target in the visible light image after the visible light image is collected and processed. The annotation includes: the category of the target and the X coordinate and Y coordinate of the target object label box. The lightweight framework Darknet and YoloV3 detection algorithm are used to train the annotated data set to obtain the target detection model. The visible light image is preprocessed, such as denoising and enhancement, to improve the target detection effect. The real-time collected visible light image is input into the trained target detection model to identify and classify the target. The category of the target and the X coordinate and Y coordinate of the label box are output. The X coordinate and Y coordinate of the target label box identified in the visible light channel are synchronized to the thermal imaging channel to realize the recognition of the target in the thermal imaging image. According to the recognition result and multimodal feature vector of the target, the temperature value of the target is predicted by the deep learning model. At the same time, the actual temperature value of the target is read in real time by using Hikvision's hemispherical thermal imaging camera. The predicted temperature value is compared and corrected with the actual temperature value to improve the accuracy of temperature measurement.

[0077] In one embodiment of the present invention, the mobile monitoring platform includes a DC power supply provided in its mechanical structure, which directly supplies power to each sensor.

[0078] Specifically, the DC power supply of the mechanical structure of the mobile monitoring platform can directly power each sensor, avoiding the need for wiring, thereby improving safety and convenience to a certain extent.

[0079] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A mobile monitoring platform based on thermal imaging camera, comprising: Mobile robot mechanical chassis, host computer, multi-line laser radar and thermal imaging camera; the mobile robot mechanical chassis, multi-line laser radar and thermal imaging camera are all connected to the host computer by wired means; the host computer is used as a central processing unit and is responsible for the system scheduling of the entire monitoring platform; the robot mechanical chassis is controlled and the thermal imaging data is processed according to the data signal uploaded by the sensor; the motion planning and obstacle avoidance of the mobile robot and the identification of the target object are completed according to the processing results, and it is characterized in that it includes the following modules: Data acquisition module: collects thermal imaging images, visible light images and multi-line lidar data based on sensors; Data processing module: pre-processing of thermal imaging images and visible light images, including: denoising, image enhancement and correction; pre-processing of multi-line lidar data, including: filtering denoising, data cleaning and standardization; Feature extraction module: extract features from processed thermal imaging images, visible light images and multi-line lidar data; generate multimodal feature vectors based on feature extraction results; Data analysis module: adjusts the robot's motion state through control algorithms to control the mechanical chassis; builds a three-dimensional model of the environment based on multi-line laser radar and analyzes the robot's optimal path for navigation and obstacle avoidance; uses multi-modal detection methods to identify and detect targets in the temperature range.

2. A mobile monitoring platform based on a thermal imaging camera according to claim 1, characterized in that: Extracting features from processed thermal imaging images, visible light images, and multi-line lidar data includes the following steps: Extract features from thermal imaging images, including temperature distribution features, hot spot features, and edge features; extract features from visible light images, including color features, texture features, and shape features; extract features from multi-line lidar data, including three-dimensional shape features, distance features, and reflection intensity features.

3. A mobile monitoring platform based on a thermal imaging camera according to claim 2, characterized in that: Generating a multimodal feature vector based on the feature extraction results includes the following steps: Normalize the features extracted from thermal imaging images, visible light images and multi-line lidar data for preprocessing; Perform feature-level fusion based on the preprocessed features, splice the features extracted from the thermal imaging image, visible light image, and multi-line lidar data to generate a long feature vector; The long feature vector is reduced in dimension through principal component analysis to generate multimodal feature vector.

4. The mobile monitoring platform based on thermal imaging camera according to claim 1, characterized in that: The three-dimensional model of the environment is constructed based on the multi-line laser radar and the optimal path of the robot is analyzed for navigation and obstacle avoidance, including the following steps: After the robot is started, it uses a multi-line laser radar to navigate and avoid obstacles, and converts the data of the multi-line laser radar into point cloud data, where each point represents the intersection of a laser beam and an obstacle; the point cloud data includes: the location, shape and size of the obstacle; Before the robot moves, the global map is simulated to build a three-dimensional model of the environment; by processing the point cloud data, the ground point cloud and non-ground point cloud are extracted; mapping is performed based on the three-dimensional point cloud map to obtain a two-dimensional grid map; Based on the extracted ground point cloud and non-ground point cloud, the robot is path-planned; using the path planning algorithm, the best path from the current position to the target position while avoiding obstacles is searched in the two-dimensional grid map; According to the two-dimensional grid map mapped from the three-dimensional point cloud map, the obstacle avoidance effect in the three-dimensional environment is simulated. All point clouds within the robot's height range will be regarded as obstacles, forming the robot's forward motion channel; in the two-dimensional grid map, all point clouds within the robot's height range will be marked as obstacles; By analyzing the formula: Y = {(x,y)|h(x,y)≤H} Obtain the obstacle point set Y, where H represents the height of the robot and h(x,y) represents the height of each point (x,y) in the two-dimensional grid map.

5. The mobile monitoring platform based on thermal imaging camera according to claim 1, characterized in that: include: The sensors used all support working in high temperature environments.

6. The mobile monitoring platform based on thermal imaging camera according to claim 1, characterized in that: include: The internal wiring of the platform adopts USB-CAN wiring method.

7. The mobile monitoring platform based on thermal imaging camera according to claim 1, characterized in that: A multi-modal detection method is used to identify and detect targets in the temperature area, including the following steps: According to the collected and processed visible light images, the target objects in the temperature area are identified or classified by using the target detection model; Use the COCO dataset to annotate objects in visible light images. The annotations include the category of the object and the X and Y coordinates of the object annotation box. Input the annotated dataset into the object detection model for training. By collecting and processing visible light images in real time, the real-time visible light images are input into the trained target detection model, the targets in the visible light images are identified or classified in real time, and the X coordinates and Y coordinates of the target annotation box in the visible light image are output; after the target is identified and detected in the visible light channel, the X coordinates and Y coordinates of the target annotation box are synchronized to the thermal imaging channel; by synchronizing the annotations of the visible light and thermal imaging channels, the target is identified in the thermal imaging channel; Based on the recognition results of the target object, the temperature value of the target object is predicted using the deep learning model; Labeling multimodal feature vectors, and inputting the labeled multimodal feature vectors into a deep learning model for training; According to the trained deep learning model, real-time multi-modal feature vectors are extracted by real-time acquisition and processing of thermal imaging images, visible light images and multi-line lidar data; the real-time multi-modal feature vectors are input into the trained deep learning model to predict the temperature value of the target object and output the result; According to the recognition result of the target object, the actual temperature value of the target object is read in real time by a thermal imaging camera; the actual temperature value is compared and corrected with the predicted temperature value to obtain the temperature value of the target object.

8. The mobile monitoring platform based on thermal imaging camera according to claim 1, characterized in that: include: The mobile monitoring platform has its own DC power supply in its mechanical structure, which directly powers each sensor.

Images