A wide-area environmental dynamic autonomous monitoring system based on multimodal fusion

Abstract

The application discloses a kind of wide-area environment dynamic autonomous monitoring system based on multi-modal fusion, including mobile platform, the automatic cruise unit is installed on the mobile platform, the lower end of the mobile platform is provided with dielectric constant detection device, one side of the mobile platform is provided with microwave detection module and image capture camera, the dielectric constant detection device, microwave detection module and image capture camera are connected with main control module, the detection mechanism can be autonomously moved in the indoor environment to be detected by using SLAM technology, to prevent the generation of detection dead angle.Simultaneously, the application adopts novel microwave detection and dielectric constant detection, which improves the accuracy of detection, and can feed back the material of impurities, so as to facilitate the cleaning personnel to carry corresponding equipment for cleaning.

Description

Technical Field

[0001] This invention relates to the field of intelligent ground debris detection technology, specifically to a wide-area environmental dynamic autonomous monitoring system based on multimodal fusion. Background Technology

[0002] Currently, fixed indoor monitoring technology can provide feedback on certain environmental parameters (such as dust and PM2.5) at fixed indoor locations, and can also detect litter on the ground using cameras. However, in traditional monitoring technologies, the detection devices are fixed. To achieve full indoor sensor coverage, a large number of monitoring facilities need to be deployed, which is costly and may result in blind spots. Summary of the Invention

[0003] The purpose of this invention is to design a wide-area dynamic autonomous monitoring system based on multimodal fusion. Employing SLAM technology, the system enables the detection mechanism to move autonomously within the indoor environment to be monitored, preventing blind spots. Furthermore, this invention utilizes novel microwave detection and dielectric constant detection, which improves detection accuracy and provides information on the material composition of debris, facilitating cleaning personnel to carry the appropriate equipment for cleaning.

[0004] The technical solution adopted in this invention is:

[0005] A wide-area environmental dynamic autonomous monitoring system based on multimodal fusion includes a mobile platform. An automatic cruise unit is installed on the mobile platform. A dielectric constant detection device is installed at the lower end of the mobile platform. A microwave detection module and an image capture camera are installed on one side of the mobile platform. The automatic cruise unit, dielectric constant detection device, microwave detection module, and image capture camera are all connected to a main control module.

[0006] The microwave detection module includes a microwave signal generator, a signal transmitting antenna, and a signal receiving antenna. The microwave signal generator is connected to the signal transmitting antenna to emit a microwave signal that is perpendicularly incident on the test sample. The signal receiving antenna receives the microwave signal transmitted through the test sample. A microwave detector is connected to each end of the signal transmitting antenna and the signal receiving antenna. The microwave detector is used to convert the microwave signal on the corresponding antenna into a voltage signal.

[0007] The main control module is configured as follows:

[0008] The dielectric constant detected by the dielectric constant detection device is obtained, and the type of ground debris is determined based on the change in the value of the dielectric constant.

[0009] The first microwave signal on the signal transmitting antenna and the second microwave signal on the signal receiving antenna in the microwave detection module are acquired, and the type of ground debris is determined based on the first microwave signal and the second microwave signal.

[0010] The system acquires image data captured by the image capture camera and identifies the types of ground debris in the image data.

[0011] Furthermore, the automatic cruise unit includes a drive unit, an odometer, a lidar, and an infrared ranging sensor. The drive unit includes casters, a differential, and a motor. The casters are connected to the motor via the differential. The motor and the differential are electrically connected to the main control module. The odometer and the lidar are used for real-time mapping and positioning, and feed the mapping and positioning signals to the main control module. The infrared ranging sensor is located on one side of the mobile platform and connected to the main control module to feed distance signals to the main control module. The main control module is configured to control the motor and the differential based on the mapping and positioning signals and the distance signals to achieve intelligent movement of the mobile platform.

[0012] Furthermore, a dust sensor is installed at the upper end of the mobile platform. The dust sensor is signal-connected to the main control module. The main control module is configured to acquire the dust signal from the dust sensor and combine it with the mapping and positioning signal at the same time to form an aerial dust positioning dataset.

[0013] Furthermore, the main control module is equipped with a first identification module, which is used to determine the type of ground debris based on the change in the dielectric constant. The first identification module obtains the information through the following method:

[0014] Construct the first recognition model;

[0015] The test samples are determined, including training samples and test samples, which are ground debris of various preset materials;

[0016] The first recognition model is trained using the detected samples:

[0017] The mobile platform is controlled to pass over each training sample, and the first numerical difference of the dielectric constant detected by the dielectric constant detection device before and after passing the training sample is obtained. The category corresponding to the first numerical difference is determined according to the category of the current training sample.

[0018] The mobile platform is controlled to pass over each test sample, and the category of the current test sample is determined based on the first numerical difference of the dielectric constant before and after passing the training sample, as detected by the dielectric constant detection device.

[0019] If the accuracy of the test sample reaches 98% or more, the trained first recognition model will be used as the first recognition module.

[0020] If the accuracy of the test samples does not reach 98%, then training samples are added, and the first recognition model is retrained and tested until the accuracy of the test samples reaches 98% or higher.

[0021] Furthermore, the main control module is equipped with a second identification module, which is used to determine the type of ground debris based on the first microwave signal and the second microwave signal. The second identification module obtains the information through the following method:

[0022] Construct a second recognition model;

[0023] The test samples are determined, including training samples and test samples, which are ground debris of various preset materials;

[0024] The second recognition model is trained using the detected samples:

[0025] The mobile platform is controlled to pass over each training sample to obtain the second numerical difference between the first microwave signal and the second microwave signal, and the category corresponding to the second numerical difference is determined according to the category of the current training sample.

[0026] The mobile platform is controlled to pass over each test sample, and the category of the current test sample is determined based on the second numerical difference between the first microwave signal and the second microwave signal.

[0027] If the accuracy of the test sample reaches 98% or more, the trained second recognition model will be used as the second recognition module.

[0028] If the accuracy of the test samples does not reach 98%, then training samples are added, and the second recognition model is retrained and tested until the accuracy of the test samples reaches 98% or higher.

[0029] The present invention has at least the following beneficial effects:

[0030] After studying the sanitation conditions of large public areas and neglected corners, it was found that the current environmental sanitation monitoring in these areas suffers from problems such as small detection range, difficulty in data quantification, poor real-time performance, and limited detection content. To address these issues, this invention combines SLAM positioning and navigation technology from intelligent inspection technology with image recognition, machine learning, and other technologies widely used in product design to design a dynamic autonomous monitoring platform for public environmental sanitation in various scenarios. To improve the accuracy of identification and detection, this invention employs novel microwave detection and dielectric constant detection methods, further enriching current environmental monitoring solutions while reducing the workload of sanitation workers. Through large-scale deployment of the autonomous monitoring platform, sanitation workers can accurately control the environmental sanitation situation, thereby transforming traditional sanitation work into a more precise and efficient process. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0032] Figure 1 A structural diagram of a wide-area environmental dynamic autonomous monitoring system based on multimodal fusion according to an embodiment of the present invention is shown;

[0033] Figure 2 A block diagram illustrating the automatic control principle of a wide-area environmental dynamic autonomous monitoring system based on multimodal fusion according to an embodiment of the present invention is shown.

[0034] Figure 3 A flowchart illustrating a method for obtaining a first identification module according to an embodiment of the present invention is shown.

[0035] Figure 4 A structural block diagram of a microwave detection module of a wide-area environmental dynamic autonomous monitoring system based on multimodal fusion according to an embodiment of the present invention is shown.

[0036] Figure 5 A flowchart illustrating a method for obtaining a second identification module according to an embodiment of the present invention is shown.

[0037] Figure 6 A structural block diagram of an automatic cruise unit of a wide-area environmental dynamic autonomous monitoring system based on multimodal fusion according to an embodiment of the present invention is shown.

[0038] Figure 7A block diagram illustrating the automatic control principle of a wide-area environmental dynamic autonomous monitoring system based on multimodal fusion according to an embodiment of the present invention is shown when a micro-dust sensor is installed. Detailed Implementation

[0039] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0040] In the description of this invention, unless otherwise stated, "a plurality of" means two or more; the terms "upper," "lower," "left," "right," "inner," "outer," "front end," "rear end," "head," "tail," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0041] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "connected" and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0042] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0043] This invention discloses a wide-area environmental dynamic autonomous monitoring system based on multimodal fusion. For example... Figure 1 and Figure 2As shown, the wide-area environment dynamic autonomous monitoring system based on multimodal fusion includes a mobile platform 100, an automatic cruise unit 200 installed on the mobile platform 100, a dielectric constant detection device 400 at the lower end of the mobile platform 100, a microwave detection module 500 and an image capture camera 600 on one side of the mobile platform 100, and the automatic cruise unit 200, the dielectric constant detection device 400, the microwave detection module 500 and the image capture camera 600 are all connected to the main control module 300.

[0044] In practical use, this embodiment utilizes SLAM positioning and navigation technology to control autonomous movement direction and plan movement routes. Specifically, the main control module 300 is equipped with relevant software based on SLAM positioning and navigation technology to control the automatic patrol unit 200, enabling intelligent unmanned patrol of the mobile platform 100. During unmanned patrol, it detects and locates debris on the ground. After all locations within the designated area have been detected, the automatic patrol vehicle returns to the earliest recorded location to re-record, ensuring data real-time performance.

[0045] This invention intelligently identifies ground debris through three aspects: dielectric constant detection, microwave signal detection, and image recognition. A core technical aspect of this invention is the application of dielectric constant detection and microwave signal detection to ground debris identification. Specifically, the image recognition module 300 can be configured with an existing image recognition model. Based on the image data collected by the image capture camera 600, the model identifies the types of ground debris in the image data.

[0046] The following embodiments of the present invention will specifically introduce the application of dielectric constant detection and microwave signal detection in the identification of ground debris.

[0047] After the control module 700 obtains the dielectric constant detected by the dielectric constant detection device 400, it determines the type of ground debris based on the change in the value of the dielectric constant.

[0048] In one specific embodiment, the main control module is equipped with a first identification module, which is used to determine the type of ground debris based on changes in the dielectric constant, such as... Figure 3 As shown, the first identification module obtains the information through the following method:

[0049] Step S1001: Construct the first recognition model;

[0050] Step S1002: Determine the test samples, which include training samples and test samples. The training samples and test samples are ground debris of various preset materials.

[0051] Step S1003: Train the first recognition model using the detected samples:

[0052] Step S1004: Control the mobile platform to pass over each training sample, obtain the first numerical difference of the dielectric constant detected by the dielectric constant detection device before and after passing the training sample, and determine the category corresponding to the first numerical difference according to the category of the current training sample.

[0053] Step S1005: Control the mobile platform to pass over each test sample, and determine the category of the current test sample based on the first numerical difference of the dielectric constant detected by the dielectric constant detection device before and after passing the training sample.

[0054] Step S1006: If the accuracy of the test sample reaches 98% or more, then the trained first recognition model is used as the first recognition module.

[0055] Step S1007: If the accuracy of the test sample does not reach 98%, then increase the training samples and retrain and test the first recognition model until the accuracy of the test sample reaches 98% or more.

[0056] like Figure 4 As shown, the microwave detection module 500 includes a microwave signal generator 501, a signal transmitting antenna 502, and a signal receiving antenna 503. The microwave signal generator 501 is connected to the signal transmitting antenna 502 to emit a microwave signal that is perpendicularly incident on the detection sample. The signal receiving antenna 503 receives the microwave signal transmitted through the detection sample. A microwave detector 504 is connected to each end of the signal transmitting antenna 502 and the signal receiving antenna 503. The microwave detector 504 is used to convert the microwave signal on the corresponding antenna into a voltage signal. The microwave detector 504 is signal-connected to the main control module 300, so that the main control module 300 can obtain the first microwave signal from the signal transmitting antenna 502 and the second microwave signal from the signal receiving antenna 503.

[0057] After the main control module 300 acquires the first microwave signal on the signal transmitting antenna 501 and the second microwave signal on the signal receiving antenna 502 in the microwave detection module 500, it determines the type of ground debris based on the first microwave signal and the second microwave signal.

[0058] In one specific embodiment, the main control module is equipped with a second identification module, which is used to determine the type of ground debris based on a first microwave signal and a second microwave signal, such as... Figure 5 As shown, the second identification module obtains the information through the following method:

[0059] Step S2001: Construct the second recognition model;

[0060] Step S2002: Determine the test samples, which include training samples and test samples. The training samples and test samples are ground debris of various preset materials.

[0061] Step S2003: Train the second recognition model using the detected samples:

[0062] Step S2004: Control the mobile platform to pass over each training sample, obtain the second numerical difference between the first microwave signal and the second microwave signal, and determine the category corresponding to the second numerical difference according to the category of the current training sample.

[0063] Step S2005: Control the mobile platform to pass over each test sample, and determine the category of the current test sample based on the second numerical difference between the first microwave signal and the second microwave signal;

[0064] Step S2006: If the accuracy of the test sample reaches 98% or more, then the trained second recognition model is used as the second recognition module.

[0065] Step S2007: If the accuracy of the test sample does not reach 98%, then increase the training samples and retrain and test the second recognition model until the accuracy of the test sample reaches 98% or more.

[0066] In one specific embodiment, the structure of the automatic cruise unit was specifically designed, such as... Figure 6 As shown, the automatic cruise unit 200 includes a drive unit, an odometer 201, a lidar 202, and an infrared ranging sensor 203. The drive unit includes casters 204, a differential 205, and a motor 206. The casters 204 are connected to the motor via the differential 205, which is designed to facilitate steering. The motor 206 and the differential 205 are electrically connected to the main control module 300. The odometer 201 and the lidar 202 are used for real-time mapping and positioning, and feed the mapping and positioning signals to the main control module 300. The infrared ranging sensor 203 is located on one side of the mobile platform 100 and connected to the main control module 300 to feed distance signals to the main control module 300. The main control module 300 is configured to control the motor 206 and the differential 205 based on the mapping and positioning signals and the distance signals to achieve intelligent movement of the mobile platform 100.

[0067] In a specific embodiment, such as Figure 1 and Figure 7As shown, a dust sensor 700 is provided at the upper end of the mobile platform 100. The dust sensor 700 is connected to the main control module 300. The main control module 300 is configured to acquire the dust signal from the dust sensor 700 and combine it with the mapping and positioning signal at the same time to form an aerial dust positioning dataset.

[0068] For indoor floor dust, based on the Doppler principle, a microwave detection module is used for real-time quantitative analysis of moving dust. At the same time, sensors such as S7-L Smart-Dust or GP2Y1010AU (micro dust sensor 700) are integrated to detect and analyze scattered floating dust outside the microwave detection module in real time, thus forming a comprehensive dust detection system.

[0069] In summary, this invention offers higher detection accuracy and coverage compared to traditional fixed sensor solutions. Furthermore, due to the smaller number of sensors required, it is less expensive. Compared to current manual detection methods, it is more efficient and real-time, effectively improving indoor cleaning efficiency. For environments with higher hygiene requirements, such as office buildings or hotel lobbies, this invention can achieve high cleaning efficiency at a low cost.

[0070] The above embodiments are only used to illustrate the present invention and are not intended to limit the present invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all equivalent technical solutions also fall within the scope of the present invention, and the patent protection scope of the present invention should be defined by the claims.

Claims

1. A wide-area environment dynamic autonomous monitoring system based on multi-modal fusion, characterized in that: The mobile platform is provided with an automatic cruise unit, a dielectric constant detection device at the lower end of the mobile platform, a microwave detection module and an image capture camera at one side of the mobile platform, and the automatic cruise unit, the dielectric constant detection device, the microwave detection module and the image capture camera are connected with a main control module, The microwave detection module comprises a microwave signal generator, a signal transmitting antenna and a signal receiving antenna, the microwave signal generator is connected with the signal transmitting antenna to emit a microwave signal vertically incident on a detection sample through the signal transmitting antenna, the signal receiving antenna receives a microwave signal transmitted through the detection sample, and two ends of the signal transmitting antenna and the signal receiving antenna are respectively connected with a microwave detector, which is used to convert the microwave signal on the corresponding antenna into a voltage signal; The main control module is configured to: acquire the dielectric constant detected by the dielectric constant detection device, and determine the type of ground debris according to the numerical change of the dielectric constant; acquire a first microwave signal on the signal transmitting antenna and a second microwave signal on the signal receiving antenna in the microwave detection module, and determine the type of ground debris according to the first microwave signal and the second microwave signal; acquire image data captured by the image capture camera, and identify the type of ground debris in the image data; The automatic cruise unit comprises a driving part, an odometer, a laser radar and an infrared distance sensor, the driving part comprises a universal wheel, a differential and a motor, the universal wheel is connected with the motor through the differential, the motor and the differential are electrically connected with the main control module, the odometer and the laser radar are used for real-time mapping and positioning, and mapping and positioning signals are fed to the main control module, the infrared distance sensor is arranged on one side of the mobile platform and connected with the main control module to feed distance signals to the main control module, and the main control module is configured to control the motor and the differential based on the mapping and positioning signals and the distance signals to realize intelligent walking of the mobile platform.

2. The multi-modal fusion based wide-area environment dynamic autonomous monitoring system according to claim 1, wherein: A fine dust sensor is arranged at the upper end of the mobile platform, the fine dust sensor is signal-connected with the main control module, the main control module is configured to acquire fine dust signals of the fine dust sensor, and combine the fine dust signals with mapping and positioning signals at the same time to form an air fine dust positioning data set.

3. The multi-modal fusion based wide-area environment dynamic autonomous monitoring system according to claim 1, wherein: A first identification module is arranged on the main control module, the first identification module is used to determine the type of ground debris according to the numerical change of the dielectric constant, and the first identification module is obtained by the following method: a first identification model is constructed; detection samples are determined, the detection samples comprise training samples and test samples, and the training samples and the test samples are ground debris of various preset materials; the first identification model is trained by using the detection samples: the mobile platform is controlled to pass above each training sample, a first numerical difference of the dielectric constant detected by the dielectric constant detection device before and after passing the training sample is acquired, and the first numerical difference corresponds to the category of the current training sample is determined. The mobile platform is controlled to pass above each test sample, and the category of the current test sample is determined according to a first numerical difference between the dielectric constant detected by the dielectric constant detection device before and after passing the training sample; If the accuracy of the test sample reaches 98% or above, the trained first recognition model is taken as the first recognition module; If the accuracy of the test sample does not reach 98%, the training sample is increased, and the first recognition model is retrained and tested until the accuracy of the test sample reaches 98% or above.

4. The multi-modal fusion based wide-area environment dynamic autonomous monitoring system according to claim 1, wherein: The main control module is configured with a second recognition module, which is used to determine the category of the ground debris according to the first microwave signal and the second microwave signal, and the second recognition module is obtained by the following method: A second recognition model is constructed; A detection sample is determined, which includes a training sample and a test sample, and the training sample and the test sample are ground debris of various preset materials; The detection sample is used to train the second recognition model: The mobile platform is controlled to pass above each training sample, a second numerical difference between the first microwave signal and the second microwave signal is obtained, and the category corresponding to the second numerical difference is determined according to the category of the current training sample; The mobile platform is controlled to pass above each test sample, and the category of the current test sample is determined according to the second numerical difference between the first microwave signal and the second microwave signal; If the accuracy of the test sample reaches 98% or above, the trained second recognition model is taken as the second recognition module; If the accuracy of the test sample does not reach 98%, the training sample is increased, and the second recognition model is retrained and tested until the accuracy of the test sample reaches 98% or above.

Images