Multi-Sensor Data Fusion Method and System for Unmanned Patrol Machines in Industrial Parks

Abstract

The present invention discloses a multi-sensor data fusion method and system for unmanned inspection machines in industrial parks, including: determining the current event type and the scene environment in which it is located based on various monitoring data; determining the impact area of ​​the current event based on the monitoring data, scene environment and event type; obtaining and storing the correlation between each position coordinate in the current event impact area and the weight configuration of each monitoring data, and the correlation between the scene environment in which the current event is located and the weight configuration of each monitoring data; obtaining the sampling position coordinates of each current monitoring data, and pre-processing each monitoring data based on a fusion formula to obtain at least one fusion value. The event type and scene environment are determined by the data characteristics of each monitoring data, and then different weights are obtained and assigned to each type of monitoring data according to the event impact area corresponding to the event, so that the data obtained after fusion is more accurate and the accuracy of the later warning is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-source data fusion processing, and more specifically, to a multi-sensor data fusion method and system for unmanned inspection machines in industrial parks. Background Art

[0002] As the scale of industrial parks continues to expand, various inspection tasks within the parks are gradually transitioning from traditional manual inspections to unmanned machine inspections.

[0003] Compared with manual inspection, unmanned inspection machines equipped with various sensor devices, such as drones and self-propelled inspection vehicles, have the advantages of low operating costs and the ability to perform inspections around the clock. However, they also have obvious disadvantages, the most prominent of which include:

[0004] (1) Reliability of inspection data: Unmanned inspection machines are usually equipped with various types of sensors to detect and inspect relevant inspection areas. Factors that affect the reliability of inspection data mainly include the operating status of the sensor itself and the interference of the external environment on the detection results. For example, when the thermal infrared sensor for collecting temperature fails, the unmanned inspection machine will not be able to accurately detect the temperature of the set area; and when the ambient airflow is too large, the detection accuracy of the gas concentration sensor on the unmanned inspection machine will be greatly affected.

[0005] (2) Comprehensiveness of inspection data: Unmanned inspection machines usually collect relevant data along fixed routes or in fixed modes. When encountering emergencies during the inspection process, the unmanned inspection machines cannot fully and specifically adjust the inspection strategy and collect relevant data, which affects the background judgment of the nature of the emergency or the tracing of the cause of the emergency. For example, when a vehicle collision is detected in the park, it is necessary to further determine the vehicle location, whether the vehicle is on fire, whether there is a leak of hazardous chemicals, whether there are vehicles or personnel to be evacuated, etc. The current unmanned inspection machines cannot fully and autonomously collect the above-mentioned relevant data, which greatly reduces the reliability of accident warning.

[0006] (3) Problems with accurate processing of inspection data: Unmanned inspection machines are usually equipped with multiple types of sensors, such as smoke sensors, image collectors, temperature sensors, etc. As mentioned above, the reliability of monitoring data collected by various types of sensors in different scene environments and event types is not the same. Even if a large amount of monitoring data is collected, if there is no data fusion processing method that is accurately adapted to the event itself, the recognition and early warning effect of subsequent events will be greatly reduced.

[0007] Obviously, how to overcome the above problems is the key to improving the reliability of inspection results and warnings of unmanned inspection machines. Summary of the invention

[0008] In order to solve the problem of incomplete inspection data and low early warning reliability of unmanned inspection machines in actual use, the first purpose of this application is to propose a multi-sensor data fusion method suitable for unmanned inspection machines in industrial parks. It is based on a large model of events and scene environments, and adjusts the weights of various monitoring data corresponding to the events according to the type and characteristics of the events themselves, so that the monitoring data collected by multiple sensors can better reflect the characteristics of the events after fusion, which is convenient for the background to judge the data, and makes the early warning analysis based on the monitoring data in the later stage more reliable. The second purpose of this application is to propose a multi-sensor data collection method suitable for unmanned inspection machines in industrial parks. The entire unmanned inspection machine can autonomously adjust and improve the data collection method and type, and the inspection data can accurately reflect the status of the current inspection area, thereby improving the accuracy of the later data fusion results and the reliability of early warnings. In order to realize the above data fusion method, the third purpose of this application is to provide a multi-sensor data fusion system suitable for unmanned inspection machines in industrial parks. The specific plan is as follows:

[0009] A multi-sensor data fusion method for an unmanned inspection machine, comprising:

[0010] Determine the current event type and its scene environment based on various monitoring data;

[0011] Determine the impact area of ​​the current event based on the monitoring data, scene environment and event type;

[0012] Obtain and store the correlation between the coordinates of each location in the area affected by the current event and the weight configuration of each monitoring data, as well as the correlation between the scene environment of the current event and the weight configuration of each monitoring data;

[0013] Get the sampling location coordinates of each current monitoring data, and pre-process each monitoring data based on the following fusion formula to obtain at least one fusion value:

[0014]

[0015] Among them, T in the formula represents the fusion value obtained after processing;

[0016] A, B, and C represent the types of each monitoring data; α, β, and γ represent the weights of the sampling values ​​of the monitoring data at each location coordinate in the event impact area;

[0017] A1, B1, C1 represent the sampling values ​​of each type of monitoring data at the first position coordinate; α1, β1, γ1 represent the weight values ​​of the above sampling values ​​at the first position coordinate;

[0018] X, Y, and Z represent the acceptance weights of different types of monitoring data in the same scenario environment.

[0019] Through the above technical solution, the event type and the scenario environment at the time of the event occurrence are first determined based on the data characteristics of the monitoring data items, and then the event impact area corresponding to the above event is obtained. Different weights are assigned to the various types of monitoring data collected at different position coordinates within the event impact area. Since different types of events have different event impact areas under different scenario conditions, the weight values configured for the monitoring data items in the above solution change with the event type, scenario environment, and sampling position, and the weight corresponding to the sampling position itself is associated with the event impact area. As a result, the fused monitoring data can better reflect the true change situation of the event, making the judgment of the event by the background more accurate and the subsequent early warning more accurate and reliable.

[0020] Furthermore, the data fusion method further includes:

[0021] Determining and generating multiple event types and their corresponding scenario environments based on the monitoring data items;

[0022] Ranking the reliability of the multiple event types and scenario models according to the matching degree between the data characteristics of the multiple event types and scenario environments and the corresponding determination models, and storing them;

[0023] Obtaining and storing the theoretical change trend of the fusion values corresponding to the multiple event types, or the theoretical change trend of the sampling values of a specific type of monitoring data on the same sampling path, and the theoretical sampling data volume N on the sampling path;

[0024] Based on the currently calculated fusion values T1 - T n 、or the sampling values A1 - A of a specific category of monitoring data currently obtained from the sampling path n , generating the actual change trend of the current fusion value or the sampling value of a specific type of monitoring data;

[0025] Comparing the actual change trend with the theoretical change trends corresponding to the multiple event types, and selecting the event type with the smallest deviation between the theoretical change trend and the actual change trend as the current event type, re-determining the event impact area and calculating the fusion value;

[0026] Wherein, the currently sampled data quantity n is not less than 2N / 3.

[0027] Through the above technical solution, when the event type cannot be accurately determined in the initial stage, multiple event types can be retained according to the determination result and sorted according to the reliability, and then by comparing the fusion value or the actual change trend of the monitoring data of a specific type with the theoretical change trend, the event type is finally determined. That is, in the data fusion process, the change trend of each monitoring data is also used as the basis for determining the event type, thereby improving the accuracy of event type determination and the accuracy of subsequent data fusion results.

[0028] Further, determining the current event type and its corresponding scenario environment based on each item of monitoring data includes:

[0029] Obtain the data characteristics corresponding to each event type and scenario environment, and store the association between each event type and scenario environment and its corresponding data characteristics to form an event determination model and a scenario determination model;

[0030] Obtain the current monitoring data, extract the data characteristics, and based on the event determination model and the scenario determination model, determine the current event type and scenario environment;

[0031] Among them, the data characteristics include multiple monitoring data values and / or their combinations.

[0032] Through the above technical solution, a determination model for identifying various events and scenario environments can be stored in advance. When receiving the monitoring data output by multiple sensors, it is only necessary to extract the data characteristics of the above monitoring data and input them into the determination model, thereby quickly determining the event type and its corresponding scenario environment.

[0033] Further, determining the current event impact area according to the monitoring data, scenario environment and event type includes:

[0034] Analyze and obtain the law of change of each item of monitoring data corresponding to each event in different scenario environments with the spatial position where the sampling position coordinates are located, and store it as an event dispersion model;

[0035] Parse the monitoring data to obtain the initial sampling position coordinates corresponding to the monitoring data;

[0036] Generate an event impact area based on the initial sampling position coordinates and according to the event dispersion model;

[0037] Among them, different event impact areas corresponding to different types of monitoring data in different event types and different scenario environments are stored in the event dispersion model; the event impact area generated according to the event dispersion model is one or more. When multiple event impact areas are generated, they are divided according to the monitoring data type.

[0038] Further, obtaining the correlation between the position coordinates at each location within the event impact area and the weight configuration of each monitoring data, including:

[0039] Generating the distribution law of each monitoring data within the event impact area according to the event dispersion model;

[0040] Dividing the event impact area into multiple blocks according to the above distribution law and marking the position coordinates, and establishing the weight relationship between each block and the monitoring data according to the relative position relationship between the block and the event impact area, and storing it as a data weighting model.

[0041] Through the above technical solution, according to the dispersion laws of relevant characteristic factors such as temperature, light brightness, and sound when various events occur, the event impact area can be determined, and then different weights can be configured for different monitoring data according to the above event impact area. When fusing the data collected by multiple sensors, interference can be effectively excluded, and at the same time, the reliability of the data collected by the unmanned inspection machine can be significantly improved.

[0042] Further, the method further includes:

[0043] Grouping the monitoring data collected from the same position coordinate according to a set rule;

[0044] Substituting each group of monitoring data into the fusion formula respectively to obtain multiple fusion values T, and forming a fusion array of the current position coordinate;

[0045] Storing the fusion data associated with each position coordinate, forming a fusion matrix and outputting it.

[0046] Through the above technical solution, the change trend of each monitoring data at each position coordinate can be clearly reflected via the fusion matrix, which is convenient for grouping calculations of different types of monitoring data, helps with the later analysis of the monitoring data, and improves the data processing efficiency and warning accuracy.

[0047] A multi-sensor data acquisition method for an unmanned inspection machine, including:

[0048] Obtaining the data characteristics corresponding to each event and scene environment, and respectively associating and storing each event and scene environment with its corresponding data characteristics as an event determination model and a scene determination model;

[0049] Analyzing and obtaining the law of change of each monitoring data corresponding to each event with the change of the spatial position where the sampling position coordinate is located in different scene environments, and storing it as an event dispersion model;

[0050] Obtaining and storing the park map data;

[0051] Obtain the current monitoring data, and based on the event determination model and the scenario determination model, determine the current event and the scenario environment. Parse the sampling position coordinates of the monitoring data from the current monitoring data, and combine the map data and the event dispersion model to determine the event impact area;

[0052] According to the event impact area and the park map data, determine the monitoring data sampling plan and collect the corresponding data according to the above monitoring data sampling plan;

[0053] Among them, the data characteristics include multiple monitoring data values and / or their combinations;

[0054] Determining the monitoring data sampling plan includes determining one or more combinations of the sampling data type, sampling route and sites, sampling time, sampling frequency, and the sampling timing relationship and position relationship between various types of sampling data.

[0055] Through the above technical solutions, it is possible to generate the data collection route and sites more quickly and reasonably according to the influence range of various events in combination with the park map data, so that the collected data is more appropriate, accurate and reliable.

[0056] Further, the monitoring sites corresponding to the monitoring data are configured as fixed monitoring sites and mobile monitoring sites;

[0057] The monitoring data is configured as basic monitoring data and supplementary monitoring data;

[0058] The data collection method further includes:

[0059] Store the position coordinates of the fixed monitoring sites;

[0060] Establish and store the weight configuration model of the basic data and the supplementary data in different scenario environments, different events and different position coordinates;

[0061] Obtain the first monitoring data and the second monitoring data from the fixed monitoring site and the mobile monitoring site respectively;

[0062] Extract the data characteristics from the first monitoring data and determine the event type according to the event determination model, and combine the second monitoring data to determine the current scenario environment according to the scenario determination model;

[0063] Based on the obtained event type and scenario environment, obtain the corresponding event dispersion model, and combine the position coordinates where the fixed monitoring site is located to generate the event impact area;

[0064] Obtain the current position coordinates of the mobile monitoring site;

[0065] Generate the moving sampling route and sampling method of the moving monitoring site by combining the position coordinates of the fixed monitoring site, the position coordinates of the moving monitoring site, and the event impact area, in combination with map data;

[0066] Obtain the basic monitoring data from the fixed monitoring site, obtain the supplementary monitoring data from the moving monitoring site according to the moving sampling route and sampling method, and then output, or weight the basic monitoring data and the supplementary monitoring data according to the weight configuration model and then output.

[0067] Through the above technical solution, by combining the fixed monitoring site and the moving monitoring site, continuous monitoring of the park can be achieved. When an abnormal event occurs, the moving monitoring site can collect relevant data along the generated data sampling route, thereby making the data collection more accurate and perfect, which helps the background to determine the event type and is also convenient for tracing the cause of the event later.

[0068] A multi-sensor data fusion system for an unmanned inspection machine, comprising:

[0069] A data acquisition unit configured to receive various monitoring data;

[0070] An event type determination unit, data-connected to the data acquisition unit, obtains various monitoring data and extracts data features, and determines the event type corresponding to the current monitoring data through an internally built or called event determination model;

[0071] A scene type determination unit, data-connected to the data acquisition unit, obtains various monitoring data and extracts data features, and determines the scene environment corresponding to the current monitoring data through an internally built or called scene determination model;

[0072] An impact area generation unit configured to generate the current event impact area based on the above event type and scene environment through an internally built or called event dissipation model;

[0073] A data weight configuration unit configured to be data-connected to the data acquisition unit, obtain and analyze the sampling position coordinates of various monitoring data, and perform weighted processing on various monitoring data in combination with an internally built or called data weighting model;

[0074] A data fusion processing unit configured to be data-connected to the data weight configuration unit, obtain various weighted monitoring data, and perform fusion processing on various data based on the fusion formula as described above to obtain at least one fusion value;

[0075] Among them, the event determination model is configured to be used to associatively store the data features of each event type and its corresponding monitoring data, so as to identify the event type according to the data features;

[0076] The scenario determination model is configured to associate and store data features of each scenario environment and its corresponding monitoring data, and use them to identify the scenario environment based on the data features;

[0077] The above data features include multiple monitoring data values and / or their combinations;

[0078] The event dispersion model is configured to analyze and obtain the distribution law of the monitoring data corresponding to each event changing with the spatial position of the sampling site in different scenario environments, and use it to determine the area affected by the relevant monitoring data according to the current event and its scenario environment;

[0079] The data weighting model is configured to divide the event impact area into multiple blocks and mark the position coordinates according to the above distribution law, and establish the weight relationship between each block and the monitoring data according to the relative position relationship between the block and the event impact area, and use it to assign corresponding weight values to each item of monitoring data according to the position coordinates where the monitoring data is collected.

[0080] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0081] By determining the event type and scenario environment through the data features of each item of monitoring data, then obtaining and assigning different weights to each type of monitoring data according to the event impact area corresponding to the event. Since different types of events have different event impact areas under different scenario conditions, the weight values configured for each item of monitoring data in the above solution change with the event type, scenario environment, and sampling position. Therefore, the fused monitoring data can better reflect the real change situation of the event, making the judgment of the event by the background more accurate and the later warning more accurate and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 It is the overall schematic diagram of the multi-sensor data fusion method of the present invention;

[0083] Figure 2 It is the schematic diagram of the method for verifying the event determination result based on the fusion value;

[0084] Figure 3 It is the schematic diagram of the monitoring data acquisition path and sites of the fire event;

[0085] Figure 4 It is the schematic diagram of the multi-sensor data acquisition method;

[0086] Figure 5 It is the schematic diagram of the method for acquiring multi-sensor data based on fixed monitoring sites and mobile monitoring sites;

[0087] Figure 6 It is the schematic diagram of the functional modules of the multi-sensor data fusion system of the present invention.

[0088] Figure numerals: 1. Data acquisition unit; 2. Event type determination unit; 3. Scene type determination unit; 4. Impact area generation unit; 5. Data weight configuration unit; 6. Data fusion processing unit; 7. Data storage device. DETAILED DESCRIPTION

[0089] The present application is further described in detail below in conjunction with embodiments and drawings, but the implementation methods of the present application are not limited thereto.

[0090] A multi-sensor data fusion method for unmanned inspection machines, such as Figure 1 As shown, it mainly includes the following steps:

[0091] S100, determining the current event type and the scene environment in which it is located based on various monitoring data;

[0092] S200, determining the impact area of ​​the current event according to the monitoring data, scene environment and event type;

[0093] S300, obtaining and storing the association relationship between each position coordinate in the current event impact area and each monitoring data weight configuration, and the association relationship between the scene environment of the current event and each monitoring data weight configuration;

[0094] S400, obtaining sampling position coordinates of each current monitoring data, and preprocessing each monitoring data based on the following fusion formula to obtain at least one fusion value:

[0095]

[0096] Among them, T in the formula represents the fusion value obtained after processing;

[0097] A, B, and C represent the types of each monitoring data; α, β, and γ represent the weights of the sampling values ​​of the monitoring data at each location coordinate in the event impact area;

[0098] A1, B1, C1 represent the sampling values ​​of each type of monitoring data at the first position coordinate; α1, β1, γ1 represent the weight values ​​of the above sampling values ​​at the first position coordinate;

[0099] X, Y, and Z represent the acceptance weights of different types of monitoring data in the same scenario environment.

[0100] In the implementation manner of the present application, the unmanned inspection machines used to collect various monitoring data include but are not limited to unmanned inspection vehicles equipped with various types of monitoring sensors, drones, or smart monitoring stations fixed at specific locations in the park.

[0101] The monitoring data in step S100 includes, but is not limited to: temperature monitoring data, humidity monitoring data, wind force and direction monitoring data, sound monitoring data, specific gas monitoring data, human body infrared monitoring data, rain and snow monitoring data, distance monitoring data, position coordinate monitoring data, and audio-visual data. The above data are respectively collected by such as infrared temperature monitoring sensors, humidity sensors, acoustic wave sensors, small gas component analyzers, and cameras.

[0102] The event types described in step S100 include event names or natures, such as vehicle collisions, vehicle fires, illegal occupation of public areas, etc. The scene environment refers to the scene where the event occurs and the surrounding environment, such as rainy and foggy environments, high-temperature and sunny environments, windless and quiet environments, etc.

[0103] Step S100, based on the monitoring data, determines the current event type and its scene environment. Specifically, it includes:

[0104] S110, obtains the data characteristics corresponding to each event type and scene environment, and stores the association between each event type and scene environment and its corresponding data characteristics to form an event determination model and a scene determination model. The above data characteristics include multiple monitoring data values and / or their combinations. For example, if the temperature value exceeds 800°C, it can be determined as a fire event. If it is simultaneously detected that the sound exceeds 120 decibels, the combination of temperature and sound data can be determined as an explosion and combustion event.

[0105] S120, obtains the current monitoring data, extracts the data characteristics, and based on the event determination model and the scene determination model, determines the current event type and scene environment. For example, associate a fire with specific image data and temperature data. Thus, when the set image and temperature values are detected, the event type can be initially determined as a fire. Further, by combining sound data, temperature data, and image data, events such as vehicle collisions and item explosions can be determined. Similarly, by combining wind force data, temperature and humidity data, etc., scenes such as weather and park environments can be determined.

[0106] The technical solution in the above step S100 can pre-store a determination model for identifying various events and scene environments. When receiving the monitoring data output by multiple sensors, it only needs to extract the data characteristics of the above monitoring data and input them into the determination model. Thus, the event type and its scene environment can be quickly determined.

[0107] In step S200, based on the monitoring data, scene environment, and event type, determine the current event impact area, which further includes:

[0108] S210. Analyze and obtain the rule that the monitoring data corresponding to each event changes with the spatial position of the sampling position coordinates in different scenario environments, and store it as an event dispersion model.

[0109] S220. Analyze the monitoring data to obtain the initial sampling position coordinates corresponding to the monitoring data.

[0110] S230. Generate an event impact area based on the initial sampling position coordinates and according to the event dispersion model.

[0111] In the embodiments of the present application, the above event dispersion model can be obtained through theoretical calculation or through big data analysis of historical monitoring data. The event dispersion model stores different event impact areas corresponding to different types of monitoring data in different event types and different scenario environments. For example, see Figure 3 , when the event type is an item catching fire and burning, the scenario environment is an open area, the light intensity is 60,000 lx, and the southwest wind is level 2, then for the smoke monitoring data, its event impact area extends and spreads a set range in the southwest direction from the combustion point position coordinates (which can be obtained by associative calculation of the above initial position coordinates combined with ranging data, etc.), and the specific area size of the extension and spread is calculated by the event dispersion model combined with wind force data and smoke image data.

[0112] The event impact area generated according to the event dispersion model is one or more. For example, the above smoke dispersion area can be generated with reference to the smoke monitoring data. Similarly, if the temperature monitoring data is used as a reference, another temperature dispersion area can be generated. Thus, according to the classification of different monitoring data types, multiple event impact areas can be generated. Of course, multiple monitoring data can also be used as references simultaneously to generate a composite event impact area, that is, the monitoring values of the monitoring data in this area will change accordingly with the change of the sampling position coordinates. For example Figure 3 the area where the smoke dispersion area coincides with the temperature dispersion area in

[0113] In the above step S300, obtain the correlation between the position coordinates at each position in the event impact area and the weight configuration of each monitoring data, including:

[0114] S310. Generate the distribution rule of each monitoring data in the event impact area according to the event dispersion model. The above distribution rule includes the attenuation rule of relevant monitoring data, the change rule of things, etc. For example, the smoke concentration will gradually attenuate as the event impact area expands, and after a specific chemical reagent leaks, it will gradually react with the air in the event impact area, thereby changing the reagent composition.

[0115] S320. Divide the event impact area into multiple blocks according to the above distribution law and mark the position coordinates. Based on the relative position relationship between the blocks and the event impact area, establish the weight relationship between each block and the monitoring data, and store it as a data weighting model. The reference benchmark of the above position coordinates can be a three-dimensional coordinate system determined by temporarily selecting a certain reference point as the origin and combining with the spatial direction, or a position coordinate system generated based on the park map data.

[0116] In the above step S320, the relative position relationship between the area and the event impact area refers to the position where the block itself is located in the event impact area. For example, if the block is located in the center of the event impact area and the probability of deviation in the relevant monitoring data here is very low, then the weight value configured for this block is higher. On the contrary, if the block is closer to the edge of the event impact area, its corresponding weight value is lower.

[0117] The above technical solution can determine the event impact area according to the dissipation laws of relevant characteristic factors such as temperature, light brightness, sound, smoke concentration, etc. when various events occur. Then, different weights are configured for different monitoring data according to the above event impact area. When fusing the data collected by multiple sensors, it can effectively eliminate interference and significantly improve the reliability of the monitoring data collected by the unmanned patrol machine.

[0118] Combined with steps S100 - S400, it can be seen that in the data fusion method of the embodiment of the present application, first, an event type and a scenario environment are initially determined, and then the fusion processing of relevant monitoring data is carried out. However, in actual applications, the determination of the event type and the scenario environment cannot be completely accurate. Therefore, in order to improve the accuracy of event type determination and also improve the accuracy of subsequent data fusion results, the multi-sensor data fusion method disclosed in the embodiment of the present application, such as Figure 2 shown, further includes:

[0119] A100. Based on each item of monitoring data, determine and generate multiple event types and their corresponding scenario environments;

[0120] A110. According to the matching degree between the data characteristics of the multiple event types and scenario environments and the corresponding determination models, sort the reliability of the multiple event types and scenario models and store them;

[0121] A120. Obtain and store the theoretical change trend of the fusion values corresponding to the above multiple event types, or the theoretical change trend of the sampled values of specific types of monitoring data on the same sampling path, and the theoretical sampling data volume N on the sampling path;

[0122] A130. Based on the currently calculated fusion values T1 - T nor the sampling values A1 - A of the specific category of monitoring data already obtained on the current self - sampling path n , generate the actual change trend of the current fusion value or the sampling value of the specific type of monitoring data;

[0123] A140. Compare the actual change trend with the theoretical change trends corresponding to multiple event types, and select the event type with the smallest deviation between the theoretical change trend and the actual change trend as the current event type, re - determine the event - affected area and calculate the fusion value.

[0124] In the above step A120, the fusion values corresponding to different event types have different change trends. Similarly, for the monitoring data of a specific type, the sampling values along the sampling path will also show specific change trends. For example, in a combustion event, if the monitoring data type is smoke data, then as the sampling path gradually approaches the event location, the smoke concentration value will gradually increase. Another example is in a gas leakage event, if the monitoring data type is sound data, then as the sampling path gradually approaches the event location, the amplitude of the leakage sound will gradually increase. Through the change trends of the above - mentioned fusion values or sampling values, it is possible to further determine whether the determination of the current event type is accurate.

[0125] To reduce data interference and to obtain a more accurate data change trend, the number of data n that has been sampled for change trend comparison is not less than 2 / 3 of the theoretical sampling data volume N.

[0126] In the above - mentioned technical solution, when the event type cannot be accurately determined in the initial stage, multiple event types can be retained according to the determination result and sorted according to their reliability, and then by comparing the actual change trend of the fusion value or the monitoring data of a specific type with the theoretical change trend, the event type is finally determined. That is, in the data fusion process, the change trends of each monitoring data are also used as the basis for determining the event type.

[0127] The fusion value obtained based on the above - mentioned method steps is actually a single value that changes with time or sampling position, and it cannot well reflect the change laws of each monitoring data during the later warning analysis. Optimally, the data fusion method described in the embodiment of the present application further includes:

[0128] S510. Group the monitoring data collected from the same position coordinates according to the set rules;

[0129] S520. Substitute each group of monitoring data into the fusion formula to obtain multiple fusion values T, and form a fusion array of the current position coordinates;

[0130] S530. Store the fusion data associated with each position coordinate, form a fusion matrix and output it.

[0131] The above technical solution can clearly reflect the change trends of each monitoring data at each position coordinate through the fusion matrix, facilitating the grouped calculation of different types of monitoring data, helping to exclude the interference of irrelevant data, and helping to improve the efficiency of later monitoring data analysis and processing.

[0132] Regarding the multi-sensor data fusion method of the above-mentioned unmanned inspection machine, in order to better collect relevant monitoring data, the embodiment of the present application also discloses a multi-sensor data collection method for an unmanned inspection machine, as Figure 4 shown, which mainly includes the following steps:

[0133] B100, obtaining the data characteristics corresponding to each event and scene environment, and respectively associating and storing each event and scene environment with its corresponding data characteristics as an event determination model and a scene determination model. Obtaining the data characteristics corresponding to each event and scene environment can be completed by manual input, or by analyzing historical monitoring data and combining relevant data processing algorithms. The data characteristics include multiple monitoring data values and / or their combinations.

[0134] B200, analyzing and obtaining the law that each item of monitoring data corresponding to each event changes with the spatial position of the sampling position coordinate in different scene environments, and storing it as an event dispersion model.

[0135] B300, obtaining and storing the park map data.

[0136] B400, obtaining the current monitoring data and based on the event determination model and the scene determination model, determining the current event and scene environment, parsing the monitoring data sampling position coordinate from the current monitoring data to obtain the position coordinate, and combining the map data and the event dispersion model to determine the event impact area.

[0137] B500, determining the monitoring data sampling plan according to the event impact area and the park map data, and collecting the corresponding data according to the above monitoring data sampling plan.

[0138] In the above step B500, determining the monitoring data sampling plan includes determining one or more combinations of the sampling data type, sampling route and site, sampling time, sampling frequency, and the sampling timing relationship and position relationship between various types of sampling data.

[0139] In order to further improve the accuracy of monitoring data collection, the monitoring sites corresponding to the monitoring data are configured as fixed monitoring sites and mobile monitoring sites. The fixed monitoring sites are set on charging piles, smart street lights, etc., and the mobile monitoring sites are set on unmanned inspection vehicles or drones. Correspondingly, the monitoring data is divided into basic monitoring data and reinforcement monitoring data according to the type of monitoring site. As Figure 5 shown, the foregoing data collection method further includes:

[0140] C100. Based on the park map data, establish a three-dimensional coordinate system and store the position coordinates of each fixed monitoring site.

[0141] C200. Establish and store a weight configuration model for basic data and reinforcement data in different scenario environments, different events, and different position coordinates, that is, for the same type of monitoring data in the same event, configure different weight values for the monitoring data collected from fixed monitoring sites and mobile monitoring sites.

[0142] C300. Obtain first monitoring data and second monitoring data from the fixed monitoring site and the mobile monitoring site respectively. The first monitoring data and the second monitoring data in this step include multiple types of multiple data and are stored as a monitoring data group.

[0143] C400. Extract data features from the first monitoring data and determine the event type according to the event determination model, and combine the second monitoring data to determine the current scenario environment according to the scenario determination model.

[0144] C500. Based on the obtained event type and scenario environment, obtain the corresponding event dispersion model, and combine the position coordinates of the fixed monitoring site to generate an event impact area.

[0145] C600. Obtain the current position coordinates of the mobile monitoring site.

[0146] C700. Based on the position coordinates of the fixed monitoring site, the position coordinates of the mobile monitoring site, and the event impact area, combine the map data to generate the mobile sampling route and sampling method of the mobile monitoring site. In this step, after determining the event impact area, integrate and analyze the above event impact area and the map data to obtain a sampling path for the mobile monitoring site to move within the event impact area, such as a park road within the event impact area, and thus determine the mobile sampling route and sampling method.

[0147] C800. Directly output the basic monitoring data obtained from the fixed monitoring site and the reinforcement monitoring data obtained from the mobile monitoring site according to the mobile sampling route and sampling method, and perform relevant weighted fusion calculations by the background; or in a specific implementation manner, directly weight the basic monitoring data and the reinforcement monitoring data according to the weight configuration model in step C200 and then output, which is convenient for improving the background's data analysis and processing efficiency.

[0148] The above technical solution combines fixed monitoring sites with mobile monitoring sites, enabling continuous monitoring of the park. When an abnormal event occurs, the mobile monitoring site can collect relevant data along the generated data sampling route, making the data collection more accurate and complete, facilitating the determination of the event type by the background and the tracing of the event cause in the later stage.

[0149] The following is a further description of the embodiments of the present application with reference to the Figure 3 accompanying drawings by way of example:

[0150] As Figure 3 shown, the shaded part is the affected area of smoke dispersion, and the dashed circles from the inside out are the affected areas of the two dispersion factors of temperature and light intensity respectively. When a fire breaks out in the park, the unmanned patrol vehicle can move along paths A and B to the event location. It is set that the types of sensors on the unmanned patrol vehicle include: a camera for collecting image data, a smoke sensor for detecting smoke, an infrared temperature sensor for detecting temperature, a light intensity sensor for detecting light intensity, a sound sensor for detecting sound, and an airflow sensor for monitoring wind force and direction. In addition, fixed monitoring sites are also configured on the signboards to monitor emergencies within a set range, such as fires and collisions.

[0151] When the patrol vehicle is at the starting point and blocked by the signboard, at this time, only the sound sensor of the patrol vehicle (mobile monitoring site) can detect the existence of the fire event. Then, it obtains the specific location coordinates of the fire location from the fixed monitoring site. After analysis by the event dispersion model, route A is selected as the monitoring data collection route.

[0152] Set the sound monitoring value on the mobile patrol vehicle as V1 sound and the light intensity monitoring value as V1 light and the smoke monitoring value as V1 smoke and the temperature monitoring value as V1 temp and the image monitoring value as V1 image .

[0153] The sound monitoring value detected by the signboard as the fixed monitoring site is V2 sound and the light intensity monitoring value is V2 light and the smoke monitoring value is V2 smoke and the temperature monitoring value is V2 temp and the image monitoring data is V2 image .

[0154] When the patrol vehicle moves along path A to monitoring site a, the fusion value of the multi-sensor monitoring data at this time:

[0155] Ta = L * (V1 sound * 100% + V1 light*0% + V1 smoke *0% + V1 temp *0% + V1 image *0%) + M * (V2 sound *20% + V2 light *20% + V2 smoke *0% + V2 temp *10% + V2 image *50%). Where L and M are different weight values configured for collecting monitoring data at the self - moving monitoring site and the fixed monitoring site respectively. From the calculation method of the fusion value of the monitoring data at the a monitoring site above, it can be seen that due to being blocked by the signboard, the weight values configured for the monitoring data such as the image and light intensity collected at the a monitoring site are 0, thus relevant interferences can be excluded.

[0156] When the inspection trolley moves along path A to the b monitoring site, the fusion value of the multi - sensor monitoring data at this time:

[0157] Tb = L * (V1 sound *30% + V1 light *10% + V1 smoke *0% + V1 temp *0% + V1 image *60%) + M * (V2 sound *20% + V2 light *20% + V2 smoke *0% + V2 temp *10% + V2 image *50%). For simplicity of explanation, the weight values L and M configured for collecting monitoring data at the self - moving monitoring site and the fixed monitoring site do not change hereunder. At this time, it can be seen that the weight values corresponding to the various monitoring data collected on the inspection trolley at the b monitoring site have changed. The proportion of the weight value of the image monitoring data V1 image has increased from the original 0% to 60% because the image data of the fire occurrence location can be clearly obtained at the b monitoring site, and the weight values of other types of monitoring data have also changed accordingly.

[0158] When the inspection trolley moves along path A to the c monitoring site, the fusion value of the multi - sensor monitoring data at this time:

[0159] Tc = L * (V1 sound *30% + V1 light *10% + V1 smoke *40% + V1 temp *0% + V1 image *10%) + M * (V2 sound *20% + V2 light *20% + V2 smoke*0% + V2 temp *10% + V2 image *50%). From the calculation method of the fusion value of the above c monitoring site, it can be seen that when the inspection trolley is at the c monitoring site, since it is already in the area covered by smoke, the smoke monitoring value in the monitoring data is V1 smoke The weight value it occupies has increased from the original 0% to 40%, and due to the blockage of the smoke, the image monitoring data V1 image The proportion of the weight value it occupies has decreased from the original 60% to 10%, and the weight values of other types of monitoring data have also changed accordingly.

[0160] Finally, when the inspection trolley moves along path A to the d monitoring site, the fusion value of the multi-sensor monitoring data at this time:

[0161] Td = L * (V1 sound *10% + V1 light *10% + V1 smoke *40% + V1 temp *40% + V1 image *0%) + M * (V2 sound *20% + V2 light *20% + V2 smoke *0% + V2 temp *10% + V2 image *50%). When the inspection trolley is at the d monitoring site, since it is already very close to the fire occurrence point, the temperature monitoring value is V1 temp The weight value it occupies has increased significantly, and the weight values of other types of monitoring data have also changed accordingly.

[0162] After collecting data at the above four monitoring sites a, b, c, and d, four fusion values Ta, Tb, Tc, and Td are obtained. If the numerical values of each type of monitoring data at each monitoring site are arranged in a layout, an array or matrix can be obtained. By analyzing and processing the above data, it is possible to more quickly and accurately judge the event type, making the early warning output later more accurate.

[0163] To implement the multi-sensor data fusion method of the aforementioned unmanned inspection machine, an embodiment of the present application also discloses a multi-sensor data fusion system for an unmanned inspection machine, as Figure 6 shown, including: a data acquisition unit 1, an event type determination unit 2, a scene type determination unit 3, an affected area generation unit 4, a data weight configuration unit 5, and a data fusion processing unit 6.

[0164] The data acquisition unit 1 is configured to be data-connected to the data output ports of each sensor to receive various monitoring data. In practical applications, the data acquisition unit 1 may also be configured with corresponding data conversion modules as required, such as a filter module, an AD conversion module, etc. The acquired monitoring data are directly output to the relevant data processing module or temporarily stored in the data storage device 7.

[0165] The event type determination unit 2 is connected to the data acquisition unit 1 or the data storage unit 7 to acquire various monitoring data and extract data features, and then determine the event type corresponding to the current monitoring data through the built-in or called event determination model. The method of extracting data features includes extracting pixel values ​​of specific locations in image data, extracting audio of specific frequency bands in audio data, etc., and then comparing it with the reference data already stored in the event determination model.

[0166] The scene type determination unit 3 is data-connected with the data acquisition unit 1 to acquire various monitoring data and extract data features, and determine the scene environment corresponding to the current monitoring data through a built-in or called scene determination model.

[0167] The event determination model is configured to store data features of each event type and its corresponding monitoring data in association, so as to identify the event type according to the data features. The scene determination model is configured to store data features of each scene environment and its corresponding monitoring data in association, so as to identify the scene environment according to the data features.

[0168] The impact area generation unit 4 is configured to receive the determination results of the event determination model and the scene determination model, and based on the determined event type and scene environment, generate the impact area of ​​the current event through the built-in or called event dispersion model. The location coordinates of the above event impact area can be obtained in combination with the park map data. The above event dispersion model is configured to analyze and obtain the distribution law of each monitoring data corresponding to each event in different scene environments as the spatial position of the sampling site changes, so as to determine the area affected by the relevant monitoring data according to the current event and the scene environment in which it is located.

[0169] The data weight configuration unit 5 is configured to be data-connected with the data acquisition unit 1, acquire and analyze the sampling position coordinates of each monitoring data, and perform weighted processing on each monitoring data in combination with the built-in or called data weighting model. The above-mentioned data weighting model is configured to divide the event impact area into multiple blocks according to the above-mentioned distribution law and mark the position coordinates, and establish a weight relationship between each block and the monitoring data according to the relative position relationship between the block and the event impact area, so as to assign corresponding weight values ​​to each monitoring data according to the position coordinates at the time of monitoring data collection.

[0170] The data fusion processing unit 6 is configured to be data-connected to the data weight configuration unit 5, obtain the weighted monitoring data for each item, and perform fusion processing on the data for each item based on the fusion formula described above to obtain at least one fusion value.

[0171] In practical applications, the above event determination model, scenario determination model, event dissipation model, and data weighting model can all be configured in a cloud server, which is convenient for optimizing each data model and is also beneficial for the multi-sensor data fusion system in the solution of the present application to adopt a distributed architecture for configuration.

[0172] The above are only the preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the idea of the present invention belong to the protection scope of the present invention. It should be noted that for those of ordinary skill in the art, several improvements and refinements made without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A multi-sensor data collection method for unmanned inspection machines in industrial parks, characterized in that: include: Acquire data features corresponding to each event and scene environment, and associate each event and scene environment with its corresponding data features and store them as an event determination model and a scene determination model respectively; Analyze and obtain the changing rules of various monitoring data corresponding to each event in different scene environments as the spatial position of the sampling position coordinates changes, and store them as event dispersion models; Acquire and store park map data; Obtain current monitoring data and determine the current event and scene environment based on the event determination model and the scene determination model, obtain the sampling location coordinates of the monitoring data from the current monitoring data analysis, and determine the event impact area in combination with the map data and the event dispersion model; Determine a monitoring data sampling plan based on the event impact area and park map data and collect corresponding data according to the monitoring data sampling plan; Wherein, the data features include multiple monitoring data values ​​and / or their combinations; Determining the monitoring data sampling plan includes determining the sampling data type, sampling route and location, sampling time, sampling frequency, and one or more combinations of the sampling time sequence relationship and position relationship between various types of sampling data; The monitoring sites corresponding to the monitoring data are configured as fixed monitoring sites and mobile monitoring sites; The monitoring data is configured as basic monitoring data and supplementary monitoring data; The data collection method further comprises: Storing the location coordinates of the fixed monitoring site; Establish and store the weight configuration model of basic data and reinforcement data in different scene environments, different events and different location coordinates; Acquire first monitoring data and second monitoring data from the fixed monitoring site and the mobile monitoring site respectively; Extracting data features from the first monitoring data and determining the event type according to the event determination model, and combining the second monitoring data and determining the current scene environment according to the scene determination model; Based on the acquired event type and scene environment, a corresponding event dispersion model is acquired, and the event impact area is generated in combination with the position coordinates of the fixed monitoring site; Get the current location coordinates of the mobile monitoring site; Based on the location coordinates of the fixed monitoring sites, the location coordinates of the mobile monitoring sites, and the event impact area, a mobile sampling route and sampling method of the mobile monitoring sites are generated in combination with map data; The basic monitoring data is obtained from the fixed monitoring site, and the reinforced monitoring data is obtained from the mobile monitoring site according to the mobile sampling route and sampling method and then output, or the basic monitoring data and the reinforced monitoring data are weighted according to the weight configuration model and then output.

2. The multi-sensor data collection method for unmanned inspection machines according to claim 1, characterized in that: The fixed monitoring sites include charging piles, smart street lights or monitoring stations equipped with various sensors; The mobile monitoring sites include unmanned inspection vehicles and drones.

3. A multi-sensor data fusion method for unmanned inspection machines in industrial parks, characterized in that: The various monitoring data collected and acquired based on the multi-sensor data collection method for unmanned inspection machines in industrial parks as claimed in claim 1 or 2 include: Determine the current event type and its scene environment based on various monitoring data; Determine the impact area of ​​the current event based on the monitoring data, scene environment and event type; Obtain and store the correlation between the coordinates of each location in the area affected by the current event and the weight configuration of each monitoring data, as well as the correlation between the scene environment of the current event and the weight configuration of each monitoring data; Get the sampling location coordinates of each current monitoring data, and pre-process each monitoring data based on the following fusion formula to obtain at least one fusion value: Among them, T in the formula represents the fusion value obtained after processing; A, B, and C represent the types of each monitoring data; α, β, and γ represent the weights of the sampling values ​​of the monitoring data at each location coordinate in the event impact area; A1, B1, C1 represent the sampling values ​​of each type of monitoring data at the first position coordinate; α1, β1, γ1 represent the weight values ​​of the above sampling values ​​at the first position coordinate; X, Y, and Z represent the acceptance weights of different types of monitoring data in the same scenario environment; Determine the area affected by the current event based on the monitoring data, scene environment and event type, including: Analyze and obtain the changing rules of various monitoring data corresponding to each event in different scene environments as the spatial position of the sampling position coordinates changes, and store them as event dispersion models; Analyze the monitoring data to obtain the initial sampling position coordinates corresponding to the monitoring data; Generate an event impact area based on the initial sampling position coordinates and according to the event dispersion model; Among them, the event dispersion model stores different event impact areas corresponding to different types of monitoring data in different event types and different scenario environments; the event impact area generated according to the event dispersion model is one or more, and when there are multiple event impact areas generated, they are divided according to the type of monitoring data.

4. The data fusion method according to claim 3, characterized in that: The data fusion method further includes: Determine and generate multiple event types and their scene environments based on various monitoring data; According to the matching degree between the data features of the multiple event types and scene environments and the corresponding determination models, the multiple event types and scene models are sorted by reliability and stored; Obtain and store the theoretical change trend of the fusion values ​​corresponding to the above multiple event types, or the theoretical change trend of the sampling values ​​of the specific type of monitoring data on the same sampling path, and the theoretical sampling data volume N on the sampling path; Based on the currently calculated fusion value T1-T n , or the sampling value A1-A of the specific category of monitoring data currently obtained from the sampling path n , generate the actual change trend of the current fusion value or the sampling value of a specific type of monitoring data; Compare the actual change trend with the theoretical change trends corresponding to multiple event types, select the event type with the smallest deviation between the theoretical change trend and the actual change trend as the current event type, re-determine the event impact area and calculate the fusion value; Among them, the number n of data that has been sampled currently is not less than 2N / 3.

5. The data fusion method according to claim 3, characterized in that: The determination of the current event type and the scene environment in which it is located based on various monitoring data includes: Obtaining data features corresponding to each event type and scene environment, associating each event type and scene environment with its corresponding data features and storing them to form an event determination model and a scene determination model; Acquire current monitoring data, extract data features and determine the current event type and scene environment based on the event determination model and scene determination model; Wherein, the data features include multiple monitoring data values ​​and / or their combinations.

6. The data fusion method according to claim 3, characterized in that: Obtain the relationship between the coordinates of each location in the event impact area and the weight configuration of each monitoring data, including: Generate the distribution law of various monitoring data in the event impact area according to the event dispersion model; According to the above distribution law, the event impact area is divided into multiple blocks and the position coordinates are marked. According to the relative position relationship between the blocks and the event impact area, a weight relationship between each block and the monitoring data is established and stored as a data weighted model.

7. The data fusion method according to claim 3, characterized in that: The data fusion method further includes: Group the monitoring data collected from the same location coordinates according to the set rules; Substituting each set of monitoring data into the fusion formula to obtain multiple fusion values ​​T, forming a fusion array of the current position coordinates; The fused data associated with each position coordinate is stored to form a fusion matrix and output.

8. A multi-sensor data fusion system for unmanned inspection machines in industrial parks, characterized in that: The various monitoring data collected and acquired based on the multi-sensor data collection method for unmanned inspection machines in industrial parks as claimed in claim 1 or 2 include: A data acquisition unit (1) is configured to receive various monitoring data; An event type determination unit (2) is connected to the data acquisition unit (1) to acquire various monitoring data and extract data features, and determine the event type corresponding to the current monitoring data through a built-in or called event determination model; A scene type determination unit (3) is connected to the data acquisition unit (1) to acquire various monitoring data and extract data features, and determine the scene environment corresponding to the current monitoring data through a built-in or called scene determination model; An impact region generating unit (4) is configured to generate the impact region of the current event based on the above event type and scene environment through a built-in or called event dispersion model; A data weight configuration unit (5) is configured to be data-connected to the data acquisition unit (1), acquire and analyze the sampling position coordinates of each monitoring data, and perform weighted processing on each monitoring data in combination with a built-in or called data weighting model; A data fusion processing unit (6) is configured to be data-connected to the data weight configuration unit (5), obtain each weighted monitoring data, and fuse each data based on the fusion formula described in claim 1 to obtain at least one fusion value; The event determination model is configured to associate and store data features of each event type and its corresponding monitoring data, so as to identify the event type according to the data features; The scene determination model is configured to associate and store data features of each scene environment and its corresponding monitoring data, so as to identify the scene environment according to the data features; The above data features include multiple monitoring data values ​​and / or their combinations; The event dispersion model is configured to analyze and obtain the distribution law of each monitoring data corresponding to each event in different scene environments as the spatial location of the sampling site changes, so as to determine the area affected by the relevant monitoring data according to the current event and its scene environment; The data weighting model is configured to divide the event impact area into multiple blocks according to the above distribution law and mark the location coordinates. According to the relative position relationship between the block and the event impact area, a weight relationship between each block and the monitoring data is established to assign corresponding weight values ​​to each monitoring data according to the location coordinates when the monitoring data is collected.

Images