Multi-sensor data fusion method and system of industrial park unmanned inspection machine

Abstract

The invention discloses a multi-sensor data fusion method and system for an industrial park unmanned inspection machine. The method comprises the following steps: determining a current event type and a scene environment where the current event type is located based on various monitoring data; judging an influence area of the current event according to the monitoring data, the scene environment and the event type; obtaining and storing an association relationship between each position coordinate in the current event influence area and each monitoring data weight configuration, and an association relationship between the scene environment where the current event is located and each monitoring data weight configuration; acquiring sampling position coordinates of the current monitoring data, and preprocessing the monitoring data based on the fusion equation to obtain at least one fusion value; according to the method, the event type and the scene environment are judged through the data features of the monitoring data, then the event influence area corresponding to the event is obtained, and different weights are given to the monitoring data of various types according to the event, so that the data obtained after fusion is more accurate, and the accuracy of later early warning is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-source data fusion processing, and more specifically, to a multi-sensor data fusion method and system for unmanned inspection machines in industrial parks. Background Art

[0002] As the scale of industrial parks continues to expand, various inspection tasks within the parks are gradually transitioning from traditional manual inspections to unmanned machine inspections.

[0003] Compared with manual inspection, unmanned inspection machines equipped with various sensor devices, such as drones and self-propelled inspection vehicles, have the advantages of low operating costs and the ability to perform inspections around the clock. However, they also have obvious disadvantages, the most prominent of which include: (1) Reliability of inspection data: Unmanned inspection machines are usually equipped with various types of sensors to detect and inspect relevant inspection areas. Factors that affect the reliability of inspection data mainly include the operating status of the sensor itself and the interference of the external environment on the detection results. For example, when the thermal infrared sensor for collecting temperature fails, the unmanned inspection machine will not be able to accurately detect the temperature of the set area; and when the ambient airflow is too large, the detection accuracy of the gas concentration sensor on the unmanned inspection machine will be greatly affected.

[0004] (2) Comprehensiveness of inspection data: Unmanned inspection machines usually collect relevant data along fixed routes or in fixed modes. When encountering emergencies during the inspection process, the unmanned inspection machines cannot fully and specifically adjust the inspection strategy and collect relevant data, which affects the background judgment of the nature of the emergency or the tracing of the cause of the emergency. For example, when a vehicle collision is detected in the park, it is necessary to further determine the vehicle location, whether the vehicle is on fire, whether there is a leak of hazardous chemicals, whether there are vehicles or personnel to be evacuated, etc. The current unmanned inspection machines cannot fully and autonomously collect the above-mentioned relevant data, which greatly reduces the reliability of accident warning.

[0005] (3) Problems with accurate processing of inspection data: Unmanned inspection machines are usually equipped with multiple types of sensors, such as smoke sensors, image collectors, temperature sensors, etc. As mentioned above, the reliability of monitoring data collected by various types of sensors in different scene environments and event types is not the same. Even if a large amount of monitoring data is collected, if there is no data fusion processing method that is accurately adapted to the event itself, the recognition and early warning effect of subsequent events will be greatly reduced.

[0006] Obviously, how to overcome the above problems is the key to improving the reliability of inspection results and warnings of unmanned inspection machines. Summary of the invention

[0007] In order to solve the problem of incomplete inspection data and low early warning reliability of unmanned inspection machines in actual use, the first purpose of this application is to propose a multi-sensor data fusion method suitable for unmanned inspection machines in industrial parks. It is based on a large model of events and scene environments, and adjusts the weights of various monitoring data corresponding to the events according to the type and characteristics of the events themselves, so that the monitoring data collected by multiple sensors can better reflect the characteristics of the events after fusion, which is convenient for the background to judge the data, and makes the early warning analysis based on the monitoring data in the later stage more reliable. The second purpose of this application is to propose a multi-sensor data collection method suitable for unmanned inspection machines in industrial parks. The entire unmanned inspection machine can autonomously adjust and improve the data collection method and type, and the inspection data can accurately reflect the status of the current inspection area, thereby improving the accuracy of the later data fusion results and the reliability of early warnings. In order to realize the above data fusion method, the third purpose of this application is to provide a multi-sensor data fusion system suitable for unmanned inspection machines in industrial parks. The specific plan is as follows: A multi-sensor data fusion method for an unmanned inspection machine, comprising: Determine the current event type and its scene environment based on various monitoring data; Determine the impact area of ​​the current event based on the monitoring data, scene environment and event type; Obtain and store the correlation between the coordinates of each location in the area affected by the current event and the weight configuration of each monitoring data, as well as the correlation between the scene environment of the current event and the weight configuration of each monitoring data; Get the sampling location coordinates of each current monitoring data, and pre-process each monitoring data based on the following fusion formula to obtain at least one fusion value: Among them, T in the formula represents the fusion value obtained after processing; A, B, and C represent the types of each monitoring data; α, β, and γ represent the weights of the sampling values ​​of the monitoring data at each location coordinate in the event impact area; A 1 , B 1 , C 1 Represents the sampling value of each type of monitoring data at the first position coordinate; α 1 , β 1 , γ 1 Represents the weight value of the above sampling value at the first position coordinate; X, Y, and Z represent the acceptance weights of different types of monitoring data in the same scenario environment.

[0008] Through the above technical solution, the event type and the scene environment in which the event occurred are first determined based on the data characteristics of each monitoring data, and then the event impact area corresponding to the above event is obtained. Different weights correspond to each type of monitoring data collected at different position coordinates in the event impact area. Since different types of events have different event impact areas under different scene conditions, the weight values ​​configured for each monitoring data in the above solution vary with the event type, scene environment, and sampling location, and the weight corresponding to the sampling location itself is associated with the event impact area, so that the monitoring data obtained after fusion can better reflect the real changes of the event, making the background judgment of the event more accurate, and making the later warning more accurate and reliable.

[0009] Furthermore, the data fusion method further includes: Determine and generate multiple event types and their scene environments based on various monitoring data; According to the matching degree between the data features of the multiple event types and scene environments and the corresponding determination models, the multiple event types and scene models are sorted by reliability and stored; Obtain and store the theoretical change trend of the fusion values ​​corresponding to the above multiple event types, or the theoretical change trend of the sampling values ​​of the specific type of monitoring data on the same sampling path, and the theoretical sampling data volume N on the sampling path; Based on the currently calculated fusion value T 1 -T n , or the sampling value A of the specific category of monitoring data currently obtained from the sampling path 1 -A n , generate the actual change trend of the current fusion value or the sampling value of a specific type of monitoring data; Compare the actual change trend with the theoretical change trends corresponding to multiple event types, select the event type with the smallest deviation between the theoretical change trend and the actual change trend as the current event type, re-determine the event impact area and calculate the fusion value; Among them, the number n of data that has been sampled currently is not less than 2N / 3.

[0010] Through the above technical scheme, when the event type cannot be accurately determined in the initial stage, multiple event types can be retained according to the determination results and sorted according to reliability. Then, the event type is finally determined by comparing the actual change trend of the fused value or the specific type of monitoring data with the theoretical change trend. That is, in the data fusion process, the change trend of each monitoring data is also used as the basis for determining the event type, thereby improving the accuracy of event type determination and the accuracy of subsequent data fusion results.

[0011] Furthermore, the determination of the current event type and the scene environment in which it is located based on various monitoring data includes: Obtaining data features corresponding to each event type and scene environment, associating each event type and scene environment with its corresponding data features and storing them to form an event determination model and a scene determination model; Acquire current monitoring data, extract data features and determine the current event type and scene environment based on the event determination model and scene determination model; Wherein, the data features include multiple monitoring data values ​​and / or their combinations.

[0012] Through the above technical solution, judgment models for identifying various types of events and scene environments can be stored in advance. When monitoring data output by multiple sensors is received, it is only necessary to extract the data features of the above monitoring data and input them into the judgment model, thereby quickly determining the event type and the scene environment in which it is located.

[0013] Furthermore, according to the monitoring data, scene environment and event type, the impact area of ​​the current event is determined, including: Analyze and obtain the changing rules of various monitoring data corresponding to each event in different scene environments as the spatial position of the sampling position coordinates changes, and store them as event dispersion models; Analyze the monitoring data to obtain the initial sampling position coordinates corresponding to the monitoring data; Generate an event impact area based on the initial sampling position coordinates and according to the event dispersion model; Among them, the event dispersion model stores different event impact areas corresponding to different types of monitoring data in different event types and different scenario environments; the event impact area generated according to the event dispersion model is one or more, and when there are multiple event impact areas generated, they are divided according to the type of monitoring data.

[0014] Furthermore, the correlation between the coordinates of each location in the event impact area and the weight configuration of each monitoring data is obtained, including: Generate the distribution law of various monitoring data in the event impact area according to the event dispersion model; According to the above distribution law, the event impact area is divided into multiple blocks and the position coordinates are marked. According to the relative position relationship between the blocks and the event impact area, a weight relationship between each block and the monitoring data is established and stored as a data weighted model.

[0015] Through the above technical solution, the event impact area can be determined based on the dispersion laws of relevant characteristic factors when various events occur, such as temperature, brightness, sound, etc., and then different weights can be configured for different monitoring data based on the above event impact area. When fusing data collected by multiple sensors, interference can be effectively eliminated, and the reliability of data collected by unmanned inspection machines can be significantly improved.

[0016] Furthermore, the method further comprises: Group the monitoring data collected from the same location coordinates according to the set rules; Substituting each set of monitoring data into the fusion formula to obtain multiple fusion values ​​T, forming a fusion array of the current position coordinates; The fused data associated with each position coordinate is stored to form a fusion matrix and output.

[0017] Through the above technical solution, the changing trend of each monitoring data at each location coordinate can be clearly reflected through the fusion matrix, which is convenient for grouping and calculating different types of monitoring data, helps in the later analysis of monitoring data, and improves data processing efficiency and early warning accuracy.

[0018] A multi-sensor data collection method for an unmanned inspection machine, comprising: Acquire data features corresponding to each event and scene environment, and associate each event and scene environment with its corresponding data features and store them as an event determination model and a scene determination model respectively; Analyze and obtain the changing rules of various monitoring data corresponding to each event in different scene environments as the spatial position of the sampling position coordinates changes, and store them as event dispersion models; Acquire and store park map data; Obtain current monitoring data and determine the current event and scene environment based on the event determination model and the scene determination model, obtain the sampling location coordinates of the monitoring data from the current monitoring data analysis, and determine the event impact area in combination with the map data and the event dispersion model; Determine a monitoring data sampling plan based on the event impact area and park map data and collect corresponding data according to the monitoring data sampling plan; Wherein, the data features include multiple monitoring data values ​​and / or their combinations; Determining the monitoring data sampling plan includes determining one or more combinations of sampling data type, sampling route and location, sampling time, sampling frequency, and sampling timing relationship and position relationship between various types of sampling data.

[0019] Through the above technical solution, data collection routes and locations can be generated more quickly and reasonably according to the impact range of various events combined with park map data, making the collected data more appropriate, accurate and reliable.

[0020] Furthermore, the monitoring sites corresponding to the monitoring data are configured as fixed monitoring sites and mobile monitoring sites; The monitoring data is configured as basic monitoring data and supplementary monitoring data; The data collection method further comprises: Storing the location coordinates of the fixed monitoring site; Establish and store the weight configuration model of basic data and reinforcement data in different scene environments, different events and different location coordinates; Acquire first monitoring data and second monitoring data from the fixed monitoring site and the mobile monitoring site respectively; Extracting data features from the first monitoring data and determining the event type according to the event determination model, and combining the second monitoring data and determining the current scene environment according to the scene determination model; Based on the acquired event type and scene environment, a corresponding event dispersion model is acquired, and the event impact area is generated in combination with the position coordinates of the fixed monitoring site; Get the current location coordinates of the mobile monitoring site; Based on the location coordinates of the fixed monitoring sites, the location coordinates of the mobile monitoring sites, and the event impact area, a mobile sampling route and sampling method of the mobile monitoring sites are generated in combination with map data; The basic monitoring data is obtained from the fixed monitoring site, and the reinforced monitoring data is obtained from the mobile monitoring site according to the mobile sampling route and sampling method and then output, or the basic monitoring data and the reinforced monitoring data are weighted according to the weight configuration model and then output.

[0021] Through the above technical solution, fixed monitoring sites are combined with mobile monitoring sites to achieve uninterrupted monitoring of the park. When an abnormal event occurs, the mobile monitoring site can collect relevant data along the generated data sampling route, thereby making data collection more accurate and complete, which helps the background to determine the type of event and facilitates the tracing of the cause of the event at a later stage.

[0022] A multi-sensor data fusion system for unmanned inspection machines, comprising: A data acquisition unit, configured to receive various monitoring data; An event type determination unit is connected to the data acquisition unit, acquires various monitoring data and extracts data features, and determines the event type corresponding to the current monitoring data through a built-in or called event determination model; A scene type determination unit, connected to the data acquisition unit, acquires various monitoring data and extracts data features, and determines the scene environment corresponding to the current monitoring data through a built-in or called scene determination model; An impact area generation unit is configured to generate the current event impact area based on the above event type and scene environment through a built-in or called event dispersion model; A data weight configuration unit is configured to be connected to the data acquisition unit, acquire and analyze the sampling position coordinates of each monitoring data, and perform weighted processing on each monitoring data in combination with a built-in or called data weighting model; A data fusion processing unit is configured to be data-connected to the data weight configuration unit, obtain the weighted monitoring data, and fuse the data based on the fusion formula as described above to obtain at least one fusion value; The event determination model is configured to associate and store data features of each event type and its corresponding monitoring data, so as to identify the event type according to the data features; The scene determination model is configured to associate and store data features of each scene environment and its corresponding monitoring data, so as to identify the scene environment according to the data features; The above data features include multiple monitoring data values ​​and / or their combinations; The event dispersion model is configured to analyze and obtain the distribution law of each monitoring data corresponding to each event in different scene environments as the spatial location of the sampling site changes, so as to determine the area affected by the relevant monitoring data according to the current event and its scene environment; The data weighting model is configured to divide the event impact area into multiple blocks according to the above distribution law and mark the location coordinates. According to the relative position relationship between the block and the event impact area, a weight relationship between each block and the monitoring data is established to assign corresponding weight values ​​to each monitoring data according to the location coordinates when the monitoring data is collected.

[0023] Compared with the prior art, the present invention has the following beneficial effects: The event type and scene environment are determined by the data characteristics of each monitoring data, and then different weights are assigned to each type of monitoring data based on the event impact area corresponding to the event. Since different types of events have different event impact areas under different scene conditions, the weight value configured for each monitoring data in the above scheme changes with the event type, scene environment, and sampling location. Therefore, the monitoring data obtained after fusion can better reflect the actual changes of the event, make the background judgment of the event more accurate, and make the later warning more accurate and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1It is an overall schematic diagram of the multi-sensor data fusion method of the present invention; Figure 2 A schematic diagram of a method for verifying an event determination result based on a fusion value; Figure 3 This is a schematic diagram of the fire incident monitoring data collection path and location; Figure 4 is a schematic diagram of a multi-sensor data acquisition method; Figure 5 A schematic diagram of a method for collecting multi-sensor data based on fixed monitoring sites and mobile monitoring sites; Figure 6 It is a schematic diagram of the functional modules of the multi-sensor data fusion system of the present invention.

[0025] Figure numerals: 1. Data acquisition unit; 2. Event type determination unit; 3. Scene type determination unit; 4. Impact area generation unit; 5. Data weight configuration unit; 6. Data fusion processing unit; 7. Data storage device. DETAILED DESCRIPTION

[0026] The present application is further described in detail below in conjunction with embodiments and drawings, but the implementation methods of the present application are not limited thereto.

[0027] A multi-sensor data fusion method for unmanned inspection machines, such as Figure 1 As shown, it mainly includes the following steps: S100, determining the current event type and the scene environment in which it is located based on various monitoring data; S200, determining the impact area of ​​the current event according to the monitoring data, scene environment and event type; S300, obtaining and storing the association relationship between each position coordinate in the current event impact area and each monitoring data weight configuration, and the association relationship between the scene environment of the current event and each monitoring data weight configuration; S400, obtaining sampling position coordinates of each current monitoring data, and preprocessing each monitoring data based on the following fusion formula to obtain at least one fusion value: Among them, T in the formula represents the fusion value obtained after processing; A, B, and C represent the types of each monitoring data; α, β, and γ represent the weights of the sampling values ​​of the monitoring data at each location coordinate in the event impact area; A 1 , B 1 , C 1 Represents the sampling value of each type of monitoring data at the first position coordinate; α 1 , β 1, γ 1 Represents the weight value of the above sampling value at the first position coordinate; X, Y, and Z represent the acceptance weights of different types of monitoring data in the same scenario environment.

[0028] In the implementation manner of the present application, the unmanned inspection machines used to collect various monitoring data include but are not limited to unmanned inspection vehicles equipped with various types of monitoring sensors, drones, or smart monitoring stations fixed at specific locations in the park.

[0029] The monitoring data in step S100 include, but are not limited to: temperature monitoring data, humidity monitoring data, wind force and wind direction monitoring data, sound monitoring data, specific gas monitoring data, human infrared monitoring data, rain and snow monitoring data, distance monitoring data, location coordinate monitoring data, and audio and video data. The above data are collected by infrared temperature monitoring sensors, humidity sensors, sound wave sensors, small gas composition analyzers, and cameras.

[0030] The event type described in step S100 includes the event name or event nature, such as vehicle collision, vehicle fire, illegal occupation of public areas, etc. The scene environment refers to the scene and surrounding environment where the event occurs, such as rainy and foggy environment, high temperature and sunshine environment, windless and quiet environment, etc.

[0031] Step S100, based on various monitoring data, determines the current event type and its scene environment, the details of which include: S110, obtaining data features corresponding to each event type and scene environment, associating each event type and scene environment with its corresponding data features and storing them, forming an event determination model and a scene determination model. The above data features include multiple monitoring data values ​​and / or their combinations. For example, if the temperature value exceeds 800°C, it can be determined as a fire event. If the sound is monitored to exceed 120 decibels at the same time, the combination of temperature and sound data can be determined as an explosion and combustion event.

[0032] S120, obtain the current monitoring data, extract the data features and determine the current event type and scene environment based on the event determination model and scene determination model. For example, the fire situation is associated with specific image data and temperature data, so that when the set image and temperature values ​​are monitored, the event type can be preliminarily determined to be a fire situation; further, by combining the sound data, temperature data and image data, events such as vehicle collisions and object explosions can be determined. Similarly, combined with wind data, temperature and humidity data, scenes such as weather and park environment can be determined.

[0033] The technical solution in the above step S100 can store in advance the judgment model for identifying various types of events and scene environments. When the monitoring data output by multiple sensors is received, it is only necessary to extract the data features of the above monitoring data and input them into the judgment model, thereby quickly determining the event type and the scene environment in which it is located.

[0034] In step S200, determining the impact area of ​​the current event according to the monitoring data, scene environment and event type further includes: S210, analyzing and obtaining the changing rules of various monitoring data corresponding to each event in different scene environments as the spatial position of the sampling position coordinates changes, and storing them as an event dispersion model.

[0035] S220, parsing the monitoring data to obtain initial sampling position coordinates corresponding to the monitoring data; S230, generating an event impact area based on the initial sampling position coordinates and according to the event dispersion model; In the implementation manner of the present application, the above-mentioned event dispersion model can be obtained through theoretical calculation or through big data analysis of historical monitoring data. The event dispersion model stores different types of monitoring data corresponding to different event impact areas in different event types and different scene environments. For example, see Figure 3 , the event type is object fire, the scene environment is an open field, the light intensity is 60000lx, and the southwest wind is level 2. For the smoke monitoring data, the event impact area is the setting range extending from the position coordinates of the burning point (which can be calculated by combining the above initial position coordinates with the ranging data) to the southwest direction. The specific extension and diffusion area is calculated by the event dispersion model combined with the wind data and smoke image data.

[0036] According to the event dissipation model, there are one or more event impact areas generated. For example, the above-mentioned smoke dissipation area can be generated by taking smoke monitoring data as a reference. Similarly, if temperature monitoring data is used as a reference, another temperature dissipation area can be generated. It can be seen that according to the classification of different monitoring data types, there can be multiple event impact areas generated. Of course, multiple monitoring data can also be used as references at the same time to generate a composite event impact area, that is, the monitoring values ​​of each monitoring data in the area will change accordingly with the change of the sampling position coordinates, for example Figure 3 In the area where the smoke escape area overlaps with the temperature escape area, it is obviously most efficient to use unmanned inspection machines equipped with various sensors to collect relevant monitoring data in the above-mentioned area.

[0037] In the above step S300, the association relationship between each position coordinate in the event impact area and each monitoring data weight configuration is obtained, including: S310, generating distribution rules of various monitoring data in the event impact area according to the event dissipation model, the above distribution rules include attenuation rules of relevant monitoring data, laws of changes of things, etc. For example, the smoke concentration will gradually decay as the event impact area expands, and after a specific chemical reagent leaks, it will gradually react with the air in the event impact area, thereby causing the reagent composition to change.

[0038] S320, according to the above distribution law, the event impact area is divided into multiple blocks and the position coordinates are marked, and according to the relative position relationship between the blocks and the event impact area, a weight relationship between each block and the monitoring data is established and stored as a data weighted model. The reference base of the above position coordinates can be a three-dimensional coordinate system determined by combining a temporarily selected reference point as the origin and the spatial direction, or it can be a position coordinate system generated based on the park map data.

[0039] In the above step S320, the relative position relationship between the area and the event impact area refers to the position of the block itself in the event impact area. For example, the block is located in the center of the event impact area. The probability of deviation in the relevant monitoring data here is very low, so the weight value configured for the block is higher. Conversely, the closer the block is to the edge of the event impact area, the lower its corresponding weight value.

[0040] The above technical solution can determine the event impact area based on the dispersion patterns of relevant characteristic factors when various events occur, such as temperature, light brightness, sound, smoke concentration, etc., and then configure different weights for different monitoring data based on the above event impact area. When fusing data collected by multiple sensors, interference can be effectively eliminated, and the reliability of various monitoring data collected by unmanned inspection machines can also be significantly improved.

[0041] Combining steps S100-S400, it can be seen that the data fusion method in the implementation mode of the present application first preliminarily formulates an event type and scene environment, and then carries out the fusion processing of related monitoring data. However, in actual applications, the determination of event type and scene environment cannot be completely accurate. Therefore, in order to improve the accuracy of event type determination and the accuracy of subsequent data fusion results, the multi-sensor data fusion method disclosed in the implementation mode of the present application, such as Figure 2 As shown, it also includes: A100 determines and generates multiple event types and their scene environments based on various monitoring data; A110, ranking and storing the reliability of the plurality of event types and scene models according to the matching degree between the data features of the plurality of event types and scene environments and the corresponding determination models; A120, obtaining and storing the theoretical change trend of the fusion values ​​corresponding to the above-mentioned multiple event types, or the theoretical change trend of the sampling values ​​of the specific type of monitoring data on the same sampling path, and the theoretical sampling data volume N on the sampling path; A130, based on the currently calculated fusion value T 1 -T n , or the sampling value A of the specific category of monitoring data currently obtained from the sampling path 1 -A n , generate the actual change trend of the current fusion value or the sampling value of a specific type of monitoring data; A140, comparing the actual change trend with the theoretical change trends corresponding to multiple event types, selecting the event type with the smallest deviation between the theoretical change trend and the actual change trend as the current event type, re-determining the event impact area and calculating the fusion value.

[0042] In the above step A120, the fusion values ​​corresponding to different event types have different changing trends. Similarly, for a specific type of monitoring data, the sampling values ​​along the sampling path will also show a specific changing trend. For example, in a combustion event, the monitoring data type is smoke data. When gradually approaching the event location along the sampling path, the smoke concentration value will gradually increase. For another example, in a gas leakage event, the monitoring data type is sound data. When gradually approaching the event location along the sampling path, the amplitude of the leakage sound will gradually increase. Through the changing trend of the above fusion value or sampling value, it can be further determined whether the current event type is accurately determined.

[0043] In order to reduce data interference and obtain more accurate data change trends, the current number of sampled data n used for comparison of change trends is not less than 2 / 3 of the theoretical sampled data amount N.

[0044] In the above technical scheme, when the event type cannot be accurately determined in the initial stage, multiple event types can be retained according to the determination results and sorted according to reliability. Then, the event type is finally determined by comparing the actual change trend of the fused value or the specific type of monitoring data with the theoretical change trend. That is, in the data fusion process, the change trend of each monitoring data is also used as the basis for determining the event type.

[0045] The fusion value obtained based on the above method steps is actually a single value that changes with time or sampling position, and cannot well reflect the changing rules of various monitoring data in the later warning analysis. Optimized, the data fusion method described in the implementation mode of this application also includes: S510, grouping various monitoring data collected from the same location coordinates according to set rules; S520, respectively substituting each set of monitoring data into the fusion formula to obtain a plurality of fusion values ​​T, forming a fusion array of the current position coordinates; S530, storing the fusion data associated with each position coordinate, forming a fusion matrix and outputting it.

[0046] The above technical solution can clearly reflect the changing trend of each monitoring data at each location coordinate through the fusion matrix, facilitate the grouping calculation of different types of monitoring data, help eliminate the interference of irrelevant data, and help improve the efficiency of subsequent monitoring data analysis and processing.

[0047] In view of the multi-sensor data fusion method of the unmanned inspection machine, in order to better collect relevant monitoring data, the embodiment of the present application also discloses a multi-sensor data collection method for the unmanned inspection machine, such as Figure 4 As shown, it mainly includes the following steps: B100, obtain the data features corresponding to each event and scene environment, and associate each event and scene environment with the corresponding data features and store them as an event determination model and a scene determination model. The data features corresponding to each event and scene environment can be obtained by manual input or by analyzing historical monitoring data in combination with relevant data processing algorithms. The data features include multiple monitoring data values ​​and / or their combinations.

[0048] B200, analyzes and obtains the changing rules of various monitoring data corresponding to each event in different scene environments as the spatial position of the sampling position coordinates changes, and stores them as event dispersion models.

[0049] B300, obtains and stores park map data.

[0050] B400, obtain the current monitoring data and determine the current event and scene environment based on the event determination model and the scene determination model, obtain the location coordinates of the monitoring data sampling location coordinates from the current monitoring data analysis, and determine the event impact area in combination with the map data and the event dispersion model.

[0051] B500, determine a monitoring data sampling plan based on the event impact area and park map data, and collect corresponding data based on the above monitoring data sampling plan.

[0052] In the above step B500, determining the monitoring data sampling plan includes determining one or more combinations of sampling data type, sampling route and location, sampling time, sampling frequency, and sampling timing relationship and position relationship between various types of sampling data.

[0053] In order to further improve the accuracy of monitoring data collection, the monitoring sites corresponding to the monitoring data are configured as fixed monitoring sites and mobile monitoring sites. Fixed monitoring sites are set up on charging piles, smart street lights, and mobile monitoring sites are set up on unmanned patrol vehicles or drones. Correspondingly, the monitoring data is divided into basic monitoring data and supplementary monitoring data according to the type of monitoring site. Figure 5 As shown, the aforementioned data collection method also includes: C100,establishes a three-dimensional coordinate system based on the park map data and stores the location coordinates of each fixed monitoring point.

[0054] C200 establishes and stores weight configuration models for basic data and reinforcement data in different scene environments, different events and different location coordinates, that is, for the same type of monitoring data in the same event, different weight values ​​are configured for the monitoring data collected from fixed monitoring sites and mobile monitoring sites.

[0055] C300, respectively obtain first monitoring data and second monitoring data from the fixed monitoring site and the mobile monitoring site. The first monitoring data and the second monitoring data in this step include multiple types of multiple data, which are stored as a monitoring data group.

[0056] C400, extracting data features from the first monitoring data and determining the event type according to the event determination model, and combining the second monitoring data and determining the current scene environment according to the scene determination model.

[0057] C500, based on the acquired event type and scene environment, obtain the corresponding event dispersion model, and generate the event impact area in combination with the position coordinates of the fixed monitoring site.

[0058] C600, obtain the current location coordinates of the mobile monitoring site.

[0059] C700, based on the position coordinates of the fixed monitoring sites, the position coordinates of the mobile monitoring sites and the event impact area, combined with the map data, generate a mobile sampling route and sampling method for the mobile monitoring sites. In this step, after determining the event impact area, the event impact area and the map data are integrated and analyzed to obtain a sampling path for the mobile monitoring sites in the event impact area, such as a park road in the event impact area, thereby determining the mobile sampling route and sampling method.

[0060] C800, obtains basic monitoring data from fixed monitoring sites, obtains reinforced monitoring data from mobile monitoring sites according to the mobile sampling route and sampling method, and then directly outputs them, and the background performs relevant weighted fusion calculations; or in a specific implementation, directly weights the basic monitoring data and reinforced monitoring data according to the weight configuration model in step C200 and then outputs them, so as to improve the efficiency of data analysis and processing in the background.

[0061] The above technical solution combines fixed monitoring sites with mobile monitoring sites, which can realize uninterrupted monitoring of the park. When an abnormal event occurs, the mobile monitoring sites can collect relevant data along the generated data sampling route, thereby making the data collection more accurate and complete, which helps the background to determine the type of event and facilitates the tracing of the cause of the event at a later time.

[0062] The following is combined with Figure 3 The implementation methods of this application are further described by taking examples: like Figure 3 As shown, the shaded part is the impact area of ​​smoke escape, and the dotted circles from the inside to the outside are the impact areas of the two escape factors of temperature and light intensity. When a fire occurs in the park, the unmanned patrol car can go to the scene of the incident along path A and path B. The types of sensors set on the unmanned patrol car include: cameras for collecting image data, smoke sensors for detecting smoke, infrared temperature sensors for detecting temperature, light intensity sensors for detecting light intensity, sound sensors for detecting sound, and airflow sensors for monitoring wind force and wind direction. In addition, fixed monitoring sites are also configured on the signboard to monitor emergencies within the set range, such as fires, collisions, etc.

[0063] When the inspection car is at the starting point, it is blocked by the signboard. At this time, only the sound sensor of the inspection car (mobile monitoring point) can detect the existence of the fire event, and then obtain the specific location coordinates of the fire from the fixed monitoring point. After analyzing the event dispersion model, route A is selected as the monitoring data collection route.

[0064] Set the sound monitoring value on the mobile inspection vehicle to V1 sound , light intensity monitoring value is V1 light , the smoke monitoring value is V1 smoke , the temperature monitoring value is V1 temp , the image monitoring value is V1 image .

[0065] The sound monitoring value detected by the signboard as a fixed monitoring point is V2 sound , light intensity monitoring value is V2 light , the smoke monitoring value is V2 smoke , the temperature monitoring value is V2 temp, image monitoring data is V2 image .

[0066] When the inspection vehicle moves along path A to monitoring point a, the fusion value of the multi-sensor monitoring data is: Ta = L*(V1 sound *100%+V1 light *0%+V1 smoke *0%+V1 temp *0%+V1 image *0%)+M*(V2 sound *20%+V2 light *20%+V2 smoke *0%+V2 temp *10%+V2 image *50%). L and M are different weight values ​​configured for monitoring data collected from mobile monitoring sites and fixed monitoring sites, respectively. From the calculation method of the fusion value of monitoring site a, it can be seen that due to the obstruction of the signboard, the weight value configured for the monitoring data such as the image and light intensity collected at monitoring site a is 0, thereby eliminating related interference.

[0067] When the inspection vehicle moves along path A to monitoring point B, the fusion value of the multi-sensor monitoring data is: Tb=L*(V1 sound *30%+V1 light *10%+V1 smoke *0%+V1 temp *0%+V1 image *60%)+M*(V2 sound *20%+V2 light *20%+V2 smoke *0%+V2 temp *10%+V2 image *50%). To simplify the explanation, the weight values ​​L and M configured for the monitoring data collected at the mobile monitoring site and the fixed monitoring site remain unchanged, the same below. It can be seen that the weight values ​​corresponding to the various monitoring data collected by the inspection vehicle at the b monitoring site have changed. The image monitoring data V1 image The weight value ratio increased from the original 0% to 60%, because the image data of the fire location can be clearly obtained at the b monitoring site, and the weight of other types of monitoring data also changed accordingly.

[0068] When the inspection vehicle moves along path A to monitoring point c, the fusion value of the multi-sensor monitoring data is: Tc=L*(V1 sound *30%+V1 light *10%+V1smoke *40%+V1 temp *0%+V1 image *10%)+M*(V2 sound *20%+V2 light *20%+V2 smoke *0%+V2 temp *10%+V2 image *50%). From the calculation method of the fusion value of the above monitoring point c, we can see that when the inspection vehicle is located at the monitoring point c, since it is already in the area covered by smoke, the smoke monitoring value in the monitoring data is V1 smoke The weight value increased from 0% to 40%, and due to the obstruction of smoke, the image monitoring data V1 image The weight ratio has dropped from 60% to 10%, and the weights of other types of monitoring data have also changed accordingly.

[0069] Finally, when the inspection car moves along path A to monitoring point d, the fusion value of the multi-sensor monitoring data is: Td=L*(V1 sound *10%+V1 light *10%+V1 smoke *40%+V1 temp *40%+V1 image *0%)+M*(V2 sound *20%+V2 light *20%+V2 smoke *0%+V2 temp *10%+V2 image *50%). When the inspection vehicle is located at monitoring point d, the temperature monitoring value is V1 because it is very close to the fire point. temp The weight value has increased significantly, and the weights of other types of monitoring data have also changed accordingly.

[0070] After collecting data from the four monitoring sites a, b, c, and d, the four fusion values ​​of Ta, Tb, Tc, and Td are obtained. If the numerical values ​​of each type of monitoring data in each monitoring site are expanded and arranged, an array or matrix can be obtained. By analyzing and processing the above data, it is possible to make a faster and more accurate judgment on the event type, making the warning output later more accurate.

[0071] In order to realize the aforementioned multi-sensor data fusion method of the unmanned inspection machine, the embodiment of the present application also discloses a multi-sensor data fusion system of the unmanned inspection machine, such as Figure 6As shown, it includes: a data acquisition unit 1, an event type determination unit 2, a scene type determination unit 3, an impact area generation unit 4, a data weight configuration unit 5 and a data fusion processing unit 6.

[0072] The data acquisition unit 1 is configured to be data-connected to the data output ports of each sensor to receive various monitoring data. In practical applications, the data acquisition unit 1 may also be configured with corresponding data conversion modules as required, such as a filter module, an AD conversion module, etc. The acquired monitoring data are directly output to the relevant data processing module or temporarily stored in the data storage device 7.

[0073] The event type determination unit 2 is connected to the data acquisition unit 1 or the data storage unit 7 to acquire various monitoring data and extract data features, and then determine the event type corresponding to the current monitoring data through the built-in or called event determination model. The method of extracting data features includes extracting pixel values ​​of specific locations in image data, extracting audio of specific frequency bands in audio data, etc., and then comparing it with the reference data already stored in the event determination model.

[0074] The scene type determination unit 3 is data-connected with the data acquisition unit 1 to acquire various monitoring data and extract data features, and determine the scene environment corresponding to the current monitoring data through a built-in or called scene determination model.

[0075] The event determination model is configured to store data features of each event type and its corresponding monitoring data in association, so as to identify the event type according to the data features. The scene determination model is configured to store data features of each scene environment and its corresponding monitoring data in association, so as to identify the scene environment according to the data features.

[0076] The impact area generation unit 4 is configured to receive the determination results of the event determination model and the scene determination model, and based on the determined event type and scene environment, generate the impact area of ​​the current event through the built-in or called event dispersion model. The location coordinates of the above event impact area can be obtained in combination with the park map data. The above event dispersion model is configured to analyze and obtain the distribution law of each monitoring data corresponding to each event in different scene environments as the spatial position of the sampling site changes, so as to determine the area affected by the relevant monitoring data according to the current event and the scene environment in which it is located.

[0077] The data weight configuration unit 5 is configured to be data-connected with the data acquisition unit 1, acquire and analyze the sampling position coordinates of each monitoring data, and perform weighted processing on each monitoring data in combination with the built-in or called data weighting model. The above-mentioned data weighting model is configured to divide the event impact area into multiple blocks according to the above-mentioned distribution law and mark the position coordinates, and establish a weight relationship between each block and the monitoring data according to the relative position relationship between the block and the event impact area, so as to assign corresponding weight values ​​to each monitoring data according to the position coordinates at the time of monitoring data collection.

[0078] The data fusion processing unit 6 is configured to be data-connected with the data weight configuration unit 5, obtain each weighted monitoring data, and fuse each data based on the fusion formula as described above to obtain at least one fusion value.

[0079] In practical applications, the above-mentioned event determination model, scene determination model, event dispersion model, and data weighting model can all be configured in the cloud server, which is convenient for optimizing each data model and is also conducive to the multi-sensor data fusion system in the present application scheme to be configured with a distributed architecture.

[0080] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A multi-sensor data fusion method for unmanned inspection machines in industrial parks, characterized in that: include: Determine the current event type and its scene environment based on various monitoring data; Determine the impact area of ​​the current event based on the monitoring data, scene environment and event type; Obtain and store the correlation between the coordinates of each location in the area affected by the current event and the weight configuration of each monitoring data, as well as the correlation between the scene environment of the current event and the weight configuration of each monitoring data; Get the sampling location coordinates of each current monitoring data, and pre-process each monitoring data based on the following fusion formula to obtain at least one fusion value: Among them, T in the formula represents the fusion value obtained after processing; A, B, and C represent the types of each monitoring data; α, β, and γ represent the weights of the sampling values ​​of the monitoring data at each location coordinate in the event impact area; A1, B1, C1 represent the sampling values ​​of each type of monitoring data at the first position coordinate; α1, β1, γ1 represent the weight values ​​of the above sampling values ​​at the first position coordinate; X, Y, and Z represent the acceptance weights of different types of monitoring data in the same scenario environment.

2. The data fusion method according to claim 1, characterized in that: The data fusion method further includes: Determine and generate multiple event types and their scene environments based on various monitoring data; According to the matching degree between the data features of the multiple event types and scene environments and the corresponding determination models, the multiple event types and scene models are sorted by reliability and stored; Obtain and store the theoretical change trend of the fusion values ​​corresponding to the above multiple event types, or the theoretical change trend of the sampling values ​​of the specific type of monitoring data on the same sampling path, and the theoretical sampling data volume N on the sampling path; Based on the currently calculated fusion value T1-T n , or the sampling value A1-A of the specific category of monitoring data currently obtained from the sampling path n , generate the actual change trend of the current fusion value or the sampling value of a specific type of monitoring data; Compare the actual change trend with the theoretical change trends corresponding to multiple event types, select the event type with the smallest deviation between the theoretical change trend and the actual change trend as the current event type, re-determine the event impact area and calculate the fusion value; Among them, the number n of data that has been sampled currently is not less than 2N / 3.

3. The data fusion method according to claim 1, characterized in that: The determination of the current event type and the scene environment in which it is located based on various monitoring data includes: Obtaining data features corresponding to each event type and scene environment, associating each event type and scene environment with its corresponding data features and storing them to form an event determination model and a scene determination model; Acquire current monitoring data, extract data features and determine the current event type and scene environment based on the event determination model and scene determination model; Wherein, the data features include multiple monitoring data values ​​and / or their combinations.

4. The data fusion method according to claim 1, characterized in that: Determine the area affected by the current event based on the monitoring data, scene environment and event type, including: Analyze and obtain the changing rules of various monitoring data corresponding to each event in different scene environments as the spatial position of the sampling position coordinates changes, and store them as event dispersion models; Analyze the monitoring data to obtain the initial sampling position coordinates corresponding to the monitoring data; Generate an event impact area based on the initial sampling position coordinates and according to the event dispersion model; Among them, the event dispersion model stores different event impact areas corresponding to different types of monitoring data in different event types and different scenario environments; the event impact area generated according to the event dispersion model is one or more, and when there are multiple event impact areas generated, they are divided according to the type of monitoring data.

5. The data fusion method according to claim 4, characterized in that: Obtain the relationship between the coordinates of each location in the event impact area and the weight configuration of each monitoring data, including: Generate the distribution law of various monitoring data in the event impact area according to the event dispersion model; According to the above distribution law, the event impact area is divided into multiple blocks and the position coordinates are marked. According to the relative position relationship between the blocks and the event impact area, a weight relationship between each block and the monitoring data is established and stored as a data weighted model.

6. The data fusion method according to claim 1, characterized in that: The data fusion method further includes: Group the monitoring data collected from the same location coordinates according to the set rules; Substituting each set of monitoring data into the fusion formula to obtain multiple fusion values ​​T, forming a fusion array of the current position coordinates; The fused data associated with each position coordinate is stored to form a fusion matrix and output.

7. A multi-sensor data collection method for unmanned inspection machines in industrial parks, characterized in that: include: Acquire data features corresponding to each event and scene environment, and associate each event and scene environment with its corresponding data features and store them as an event determination model and a scene determination model respectively; Analyze and obtain the changing rules of various monitoring data corresponding to each event in different scene environments as the spatial position of the sampling position coordinates changes, and store them as event dispersion models; Acquire and store park map data; Obtain current monitoring data and determine the current event and scene environment based on the event determination model and the scene determination model, obtain the sampling location coordinates of the monitoring data from the current monitoring data analysis, and determine the event impact area in combination with the map data and the event dispersion model; Determine a monitoring data sampling plan based on the event impact area and park map data and collect corresponding data according to the monitoring data sampling plan; Wherein, the data features include multiple monitoring data values ​​and / or their combinations; Determining the monitoring data sampling plan includes determining one or more combinations of sampling data type, sampling route and location, sampling time, sampling frequency, and sampling timing relationship and position relationship between various types of sampling data.

8. The multi-sensor data collection method for unmanned inspection machines according to claim 7, characterized in that: The monitoring sites corresponding to the monitoring data are configured as fixed monitoring sites and mobile monitoring sites; The monitoring data is configured as basic monitoring data and supplementary monitoring data; The data collection method further comprises: Storing the location coordinates of the fixed monitoring site; Establish and store the weight configuration model of basic data and reinforcement data in different scene environments, different events and different location coordinates; Acquire first monitoring data and second monitoring data from the fixed monitoring site and the mobile monitoring site respectively; Extracting data features from the first monitoring data and determining the event type according to the event determination model, and combining the second monitoring data and determining the current scene environment according to the scene determination model; Based on the acquired event type and scene environment, a corresponding event dispersion model is acquired, and the event impact area is generated in combination with the position coordinates of the fixed monitoring site; Get the current location coordinates of the mobile monitoring site; Based on the location coordinates of the fixed monitoring sites, the location coordinates of the mobile monitoring sites, and the event impact area, a mobile sampling route and sampling method of the mobile monitoring sites are generated in combination with map data; The basic monitoring data is obtained from the fixed monitoring site, and the reinforced monitoring data is obtained from the mobile monitoring site according to the mobile sampling route and sampling method and then output, or the basic monitoring data and the reinforced monitoring data are weighted according to the weight configuration model and then output.

9. The multi-sensor data collection method for unmanned inspection machines according to claim 8, characterized in that: The fixed monitoring sites include charging piles, smart street lights or monitoring stations equipped with various sensors; The mobile monitoring sites include unmanned inspection vehicles and drones.

10. A multi-sensor data fusion system for unmanned inspection machines in industrial parks, characterized in that: include: A data acquisition unit (1) is configured to receive various monitoring data; An event type determination unit (2) is connected to the data acquisition unit (1) to acquire various monitoring data and extract data features, and determine the event type corresponding to the current monitoring data through a built-in or called event determination model; A scene type determination unit (3) is connected to the data acquisition unit (1) to acquire various monitoring data and extract data features, and determine the scene environment corresponding to the current monitoring data through a built-in or called scene determination model; An impact region generating unit (4) is configured to generate the impact region of the current event based on the above event type and scene environment through a built-in or called event dispersion model; A data weight configuration unit (5) is configured to be data-connected to the data acquisition unit (1), acquire and analyze the sampling position coordinates of each monitoring data, and perform weighted processing on each monitoring data in combination with a built-in or called data weighting model; A data fusion processing unit (6) is configured to be data-connected to the data weight configuration unit (5), obtain each weighted monitoring data, and fuse each data based on the fusion formula described in claim 1 to obtain at least one fusion value; The event determination model is configured to associate and store data features of each event type and its corresponding monitoring data, so as to identify the event type according to the data features; The scene determination model is configured to associate and store data features of each scene environment and its corresponding monitoring data, so as to identify the scene environment according to the data features; The above data features include multiple monitoring data values ​​and / or their combinations; The event dispersion model is configured to analyze and obtain the distribution law of each monitoring data corresponding to each event in different scene environments as the spatial location of the sampling site changes, so as to determine the area affected by the relevant monitoring data according to the current event and its scene environment; The data weighting model is configured to divide the event impact area into multiple blocks according to the above distribution law and mark the location coordinates. According to the relative position relationship between the block and the event impact area, a weight relationship between each block and the monitoring data is established to assign corresponding weight values ​​to each monitoring data according to the location coordinates when the monitoring data is collected.

Images