Complex city background abnormal target self-learning detection method and device

Abstract

The invention provides a complex city background abnormal target self-learning detection method, and relates to the technical field of information, and the method comprises the steps: collecting city scene data through a multi-modal visual sensor, and carrying out the preliminary anomaly detection of the collected multi-modal data, and obtaining a preliminary anomaly detection result; adding uncertainty knowledge to the preliminary anomaly detection result by using priori knowledge and data tuning, and constructing an uncertainty model; on the basis of an uncertainty model, calculating and comparing relative entropies among different modal data, determining a resampling magnitude according to the difference of the relative entropies, and adjusting weights of the different modal data; and based on the adjusted weight, performing multi-factor coupling reasoning on the acquired multi-modal data to obtain an anomaly detection result. By adopting the scheme, the abnormal target in a complex urban background environment can be reliably monitored in an unsupervised environment.

Description

Technical Field

[0001] The present application relates to the field of information technology, and in particular to a self-learning detection method and device for abnormal targets in complex urban backgrounds. Background Art

[0002] As the intensity distribution map of different bands in the optical signal, the spectrum provides a powerful tool for extracting scene features. The spectrum exceeds the perception limitations and resolution of the human eye, and effectively solves the problem caused by "heterochromatic spectra" in traditional gray / color imaging technology. This spectral data not only integrates rich spectral details, but also incorporates spatial information, which can more comprehensively and accurately describe the characteristics of the target object and accurately distinguish different material compositions. Therefore, spectral technology has been widely used in many fields, such as space remote sensing, biomedical analysis, agricultural monitoring, and intelligent anti-counterfeiting. Among them, hyperspectral anomaly detection has become a research focus in recent years. Its goal is to identify and separate potential artificial targets from observed images. This technology is of great significance to the development of remote sensing fields such as national defense and surveillance.

[0003] However, how to achieve accurate target identification under complex and changing conditions: There are complex and changing environments in many major applications such as smart transportation, military reconnaissance, and aerospace remote sensing. During the detection process, due to the coupling of multiple factors such as atmospheric refraction and ground reflection, multi-dimensional spectral data has spectral variation aliasing, and high false alarm or false alarm rates in abnormal target identification. In addition, multimodal data such as hyperspectral and infrared have image noise and information errors, which cause small disturbances to have a greater impact on the judgment results. The existing methods have limited adaptability and strong uncertainty in detection results, making it difficult to ensure accurate target identification. Summary of the invention

[0004] The present application aims to solve one of the technical problems in the related art at least to some extent.

[0005] To this end, the first purpose of this application is to propose a self-learning detection method for abnormal targets in complex urban backgrounds, which solves the technical problem that existing methods are difficult to ensure accurate identification of targets, and realizes reliable monitoring of abnormal targets in complex urban background environments in an unsupervised environment.

[0006] The second purpose of this application is to propose a self-learning detection device for abnormal targets in complex urban backgrounds.

[0007] To achieve the above-mentioned purpose, the first aspect of the embodiment of the present application proposes a self-learning detection method for abnormal targets in complex urban backgrounds, including: collecting urban scene data through a multimodal visual sensor, and performing preliminary anomaly detection on the collected multimodal data to obtain preliminary anomaly detection results; using prior knowledge and data tuning to add uncertainty knowledge to the preliminary anomaly detection results, and constructing an uncertainty model; based on the uncertainty model, calculating and comparing the relative entropy between different modal data, and determining the resampling order according to the relative entropy difference, and adjusting the weights of different modal data; based on the adjusted weights, performing multi-factor coupling reasoning on the collected multimodal data to obtain anomaly detection results.

[0008] Optionally, in one embodiment of the present application, the multimodal visual sensor includes a visible light camera, an infrared camera, a multispectral camera, a hyperspectral camera, an event camera, and a radar sensor.

[0009] Optionally, in one embodiment of the present application, the prior knowledge includes environmental knowledge, historical data and expert knowledge of the urban context;

[0010] Data tuning includes data enhancement, data cleaning, and parameter adjustment;

[0011] The expressions of uncertain knowledge include Bayesian networks, fuzzy logic and confidence intervals.

[0012] Optionally, in one embodiment of the present application, the calculation method of relative entropy includes Kullback-Leibler divergence, cross entropy, and Jensen-Shannon divergence.

[0013] Optionally, in one embodiment of the present application, the multi-factor coupled reasoning includes Bayesian reasoning, evidence theory and integrated learning;

[0014] The above method further includes:

[0015] When conducting multi-factor coupling reasoning, abnormal target conflict decisions are made, and when there are decision conflicts between multimodal data, the set mechanism is used to resolve the conflicts and obtain the final judgment results. Among them, the abnormal target conflict decision judgment includes: formulating decision rules, conflict resolution mechanisms and feedback mechanisms.

[0016] To achieve the above-mentioned purpose, the second embodiment of the present invention proposes a self-learning detection device for abnormal targets in complex urban backgrounds, comprising:

[0017] A data acquisition module is used to collect urban scene data through a multimodal visual sensor, and perform preliminary anomaly detection on the collected multimodal data to obtain preliminary anomaly detection results;

[0018] The uncertainty factor building module is used to add uncertainty knowledge to the preliminary anomaly detection results by using prior knowledge and data tuning to build an uncertainty model;

[0019] The relative entropy joint solution module is used to calculate and compare the relative entropy between different modal data based on the uncertainty model, and determine the resampling order according to the relative entropy difference and adjust the weight of different modal data;

[0020] The abnormal target determination module is used to perform multi-factor coupling reasoning on the collected multimodal data based on the adjusted weights to obtain abnormal detection results.

[0021] Optionally, in one embodiment of the present application, the multimodal visual sensor includes a visible light camera, an infrared camera, a multispectral camera, a hyperspectral camera, an event camera, and a radar sensor.

[0022] Optionally, in one embodiment of the present application, the prior knowledge includes environmental knowledge, historical data and expert knowledge of the urban context;

[0023] Data tuning includes data enhancement, data cleaning, and parameter adjustment;

[0024] The expressions of uncertain knowledge include Bayesian networks, fuzzy logic and confidence intervals.

[0025] Optionally, in one embodiment of the present application, the calculation method of relative entropy includes Kullback-Leibler divergence, cross entropy, and Jensen-Shannon divergence.

[0026] Optionally, in one embodiment of the present application, the multi-factor coupled reasoning includes Bayesian reasoning, evidence theory and integrated learning;

[0027] The abnormal target determination module is also used to:

[0028] When conducting multi-factor coupling reasoning, abnormal target conflict decisions are made, and when there are decision conflicts between multimodal data, the set mechanism is used to resolve the conflicts and obtain the final judgment results. Among them, the abnormal target conflict decision judgment includes: formulating decision rules, conflict resolution mechanisms and feedback mechanisms.

[0029] The self-learning detection method for abnormal targets in complex urban backgrounds in the embodiment of the present application performs multi-factor coupling reasoning of decision-level fusion for the problem of small disturbances affecting decisions in multi-mode fusion data, realizes accurate reasoning and judgment when multi-source sub-decision conflicts occur, realizes self-learning of abnormal targets in complex urban backgrounds, and improves the reliability of decision-making. This embodiment effectively solves the problem of intelligent state monitoring of abnormal targets in complex urban backgrounds.

[0030] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0032] Figure 1 A flowchart of a self-learning detection method for abnormal targets in a complex urban background provided in Example 1 of the present application;

[0033] Figure 2 A schematic diagram of the self-update of dynamic Bayesian network parameters according to an embodiment of the present application;

[0034] Figure 3 A schematic diagram of a design of a complex inference calculation of a relaxed OR gate according to an embodiment of the present application;

[0035] Figure 4 A schematic diagram of the structure of a self-learning detection device for abnormal targets in complex urban backgrounds provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0037] The following describes the self-learning detection method and device for abnormal targets in complex urban backgrounds according to an embodiment of the present application with reference to the accompanying drawings.

[0038] Figure 1 A flowchart of a self-learning detection method for abnormal targets in a complex urban background provided in Example 1 of the present application.

[0039] like Figure 1 As shown, the complex urban background abnormal target self-learning detection method includes the following steps:

[0040] Step 101, collecting urban scene data through a multimodal visual sensor, and performing preliminary anomaly detection on the collected multimodal data to obtain preliminary anomaly detection results;

[0041] In this embodiment, the multimodal visual sensors used for acquisition include but are not limited to visible light cameras, infrared cameras, multispectral cameras, hyperspectral cameras, event cameras, depth cameras and radar sensors. ;

[0042] In this embodiment, the method for performing preliminary anomaly detection on the collected data includes but is not limited to background modeling and subtraction, inter-frame motion difference detection, infrared thermal image analysis, depth map analysis, point cloud processing, spectral feature analysis, obstacle detection, machine learning model and deep learning model.

[0043] Step 102, using prior knowledge and data tuning to add uncertainty knowledge to the preliminary anomaly detection results, and construct an uncertainty model;

[0044] In this embodiment, prior knowledge includes but is not limited to environmental knowledge, historical data, and expert knowledge of urban scenes. Environmental knowledge includes but is not limited to building structures, road layouts, and traffic patterns in urban backgrounds; historical data includes but is not limited to previous detection results and records of abnormal events; expert knowledge includes but is not limited to the understanding and experience of professionals on abnormal urban situations;

[0045] In this embodiment, data tuning methods include but are not limited to data enhancement, data cleaning, and parameter adjustment. Among them, data enhancement includes but is not limited to rotation, translation, and scaling; data cleaning includes but is not limited to removing outliers and noise data; parameter adjustment is to dynamically adjust the parameters of the detection model according to actual conditions;

[0046] In this embodiment, the expression of uncertainty knowledge includes but is not limited to Bayesian network, fuzzy logic, and confidence interval. Among them, Bayesian network is used for uncertainty modeling, and the uncertainty of each detection result is obtained by calculating the prior probability and conditional probability; fuzzy logic is used to express uncertainty, such as "may be" instead of "definitely" a certain abnormality, and the robustness of the system is improved through fuzzy logic operations; a confidence interval is calculated for each detection result to indicate the credibility range of the result. For example, a confidence interval is given for the detected vehicle position to indicate the possible range of abnormal targets.

[0047] Step 103, based on the uncertainty model, calculate and compare the relative entropies between different modal data, determine the resampling order according to the relative entropy difference, and adjust the weights of different modal data;

[0048] In this embodiment, the relative entropy is calculated by methods including, but not limited to, the classic Kullback-Leibler divergence, cross entropy, and Jensen-Shannon divergence;

[0049] In this embodiment, the resampling level is determined according to the relative entropy difference, which means that the weight of each sensor is dynamically adjusted according to the calculated relative entropy. For example, if the relative entropy of a sensor is small, it can be given a higher weight, otherwise its weight can be reduced.

[0050] Step 104: Based on the adjusted weights, multi-factor coupling reasoning is performed on the collected multimodal data to obtain anomaly detection results.

[0051] In this embodiment, the multi-factor coupling reasoning method includes but is not limited to Bayesian reasoning, evidence theory, and integrated learning. Among them, Bayesian reasoning is based on Bayesian probability theory, and it calculates prior probability and likelihood function and integrates multi-source data for reasoning; evidence theory integrates multiple sources of evidence, calculates trust and distrust, and makes decisions on the target; integrated learning: combines multiple machine learning models, and integrates the decision results of each model through weighted voting or other methods.

[0052] In this embodiment, the abnormal target conflict decision determination includes but is not limited to the formulation of decision rules, conflict resolution mechanism, and feedback mechanism. According to the multi-factor coupling reasoning method, specific decision rules are formulated to clarify how to make comprehensive judgments based on the inputs of different data sources; when there are decision conflicts between multiple data sources, a specific mechanism is used to resolve the conflict and obtain the final judgment result; the detection results are fed back to the system for self-learning and parameter adjustment, and the detection model is continuously optimized to improve the accuracy of the judgment.

[0053] The self-learning detection method for abnormal targets in complex urban backgrounds in the embodiment of the present application performs multi-factor coupling reasoning of decision-level fusion for the problem of small disturbances affecting decisions in multi-mode fusion data, realizes accurate reasoning and judgment when multi-source sub-decision conflicts occur, realizes self-learning of abnormal targets in complex urban backgrounds, and improves the reliability of decision-making. This embodiment effectively solves the problem of intelligent state monitoring of abnormal targets in complex urban backgrounds.

[0054] Embodiment 2 of the present application provides a self-learning detection method for abnormal targets in a complex urban background.

[0055] This embodiment collects urban scene data through multimodal visual sensors, and performs preliminary abnormal target detection on the collected information. When there are conflicts in the target recognition results obtained from multi-source information, uncertainty modeling and abnormality judgment are performed through Bayesian coupled reasoning, thereby achieving reliable monitoring of abnormal targets in complex urban background environments in an unsupervised environment.

[0056] S1: Data collection: Collect urban scene data from multimodal visual sensors and perform preliminary anomaly detection on the collected data;

[0057] S2: Uncertainty factor construction and reasoning network update: Based on the preliminary detection results of step S1, prior knowledge and data-driven are used, combined with a multi-factor coupled Bayesian causal reasoning network, and uncertainty knowledge is quickly constructed through data tuning and knowledge-driven parameters. The network incorporates potential uncertainty factors in the preliminary detection results into the reasoning process, dynamically updates node relationships and reasoning links, and thus enhances the ability to detect abnormal targets. Through this process, the uncertainty expression of multi-source data detection results is improved, and conflicting information is fused early.

[0058] S3: Adaptive resampling and simplified reasoning solution: Based on the uncertainty model constructed by the Bayesian causal reasoning network in step S2, by calculating and comparing the relative entropy between different data sources, combined with the correlation between nodes in the reasoning network, the magnitude of resampling is determined according to the size of the relative entropy difference, and the weight of each data source is dynamically adjusted. In this process, the relaxation or gate structure is introduced to further simplify the complex reasoning calculation, and the calculation complexity is reduced through the relaxation mechanism, while ensuring the accuracy and robustness of the reasoning results.

[0059] S4: Abnormal target determination: Based on the relative entropy joint solution obtained in step S3, a multi-factor coupling reasoning method is used to comprehensively consider the uncertainty and conflict of multi-source sub-decisions to obtain the abnormal target detection result.

[0060] This embodiment first proposes an uncertain target reasoning framework, and improves the target reasoning accuracy and calculation time through adaptive resampling of image areas; proposes an adaptive resampling method based on relative entropy joint solution to operate on the parameters of data distribution; introduces a relaxed OR gate to further simplify the solution of complex reasoning calculations and the parallel representation of uncertainty factors.

[0061] like Figure 2 As shown in the figure, there is a strong uncertainty in the conflicting knowledge in the multi-source fusion data with redundant information, which makes it difficult to effectively capture the changes in the confidence level of the data source. At present, most fusion perception systems use a fixed confidence level or a minority-subordinate majority approach when applied. However, due to the increased uncertainty in sensor performance caused by complex environments, the data fusion efficiency is limited. How to realize the modeling of the conflicting data generation mechanism of the fusion system under the airborne edge computing capability is the key to the airborne light and small multi-source fusion perception system.

[0062] Based on the Bayesian causal network, a multi-factor coupling model for multi-source information fusion is constructed, each information source is regarded as a child node (X), the fusion result is regarded as a parent node (Y), and the uncertainty factor (U) is set synchronously. A method combining the asymptotic approximation characteristics of the sampling average approximation and the multi-step prediction and correction of time series data is proposed to realize the parameter learning of the Bayesian causal network across time scales.

[0063] At time t, the latent variable in Conditional independent distribution is used to represent the relationship between latent variables, cause variables, and random variables.

[0064]

[0065] In the formula, a tk is the cause variable, z t is the latent variable, x t is a random variable.

[0066] Based on the asymptotic convergence characteristics of the sampling average approximation method in the random optimization method, the long short-term memory time series model of the decision node is trained through historical data to realize the multi-step prediction of its time series data. The problem of no data in the prediction link is solved based on independent and identically distributed data generation, and continuous update learning is performed based on the sampling average approximation (SAA). The SAA method obtains the solution of the random discrete optimization problem through Monte Carlo calculation. When the number of samples N is large enough, the probability of the event occurrence converges to 1. The SAA formula is expressed as follows,

[0067]

[0068] In the formula, f(x,w i ) is Lipschitz continuous in x.

[0069] For any β>0, based on the central limit theorem,

[0070]

[0071] in, The probability of being true is 1-α. Therefore, f N The convergence rate of (x) to E[f(ω) is O(N -1 / 2 ). The constructed Bayesian causal inference decision model has the ability to perform incremental learning, characterizes the uncertainty of perception data based on multi-factor coupling modeling and parameter adaptation, and realizes quantitative analysis of the dynamic changes in signal source confidence caused by dynamic changes in the environment.

[0072] (2) Adaptive resampling for joint solution of relative entropy

[0073] Traditional methods for dealing with perceptual uncertainty usually operate directly on the noise distribution. This embodiment models the parameters of the data distribution so that the adaptive resampling generation process is continuous and differentiable when the data is discrete. Assuming that the processor of different data sources is an intelligent agent, the output of its results is a sub-decision, which solves the problem of the final reliable output when there is information conflict between different sub-decisions.

[0074] Assign initial category distributions to different agents, such as standard normal distribution for continuous data and uniform distribution for discrete data. In each solution step, add uncertainty noise distribution to the output of the previous step, so that both continuous and discrete data can output continuous distributions. Calculate the KL divergence between two continuous distributions.

[0075]

[0076] The number of resampling times is determined according to the difference in KL divergence to achieve adaptive resampling of target information, and Bayesian update is performed on this basis.

[0077] In each solution step, the parameters of the input distribution (such as the mean of the normal distribution, the probability of the categorical distribution) are passed to the multi-factor coupled Bayesian causal inference network. Taking two data sources I and S as an example, given d-dimensional data x, θ is the p-dimensionality of the decomposed data source I. I The parameters of (·|θ) are set as p I (x|θ), p S (y|x; α) are the distributions of data sources I and S, respectively, as shown in the following formula:

[0078]

[0079] When updating at different solution steps, the input parameters θ and processing time t are passed to the multi-factor coupled Bayesian causal inference network Ψ. The output distribution of this step is obtained by factorization as follows:

[0080]

[0081] The key difference between the input and output distributions is that, although each p I (x (d) ∣θ (d) ) depends only on the information collected through p S (y|x; α) depends (through the network) on all θ, so each p o (x|θ,t) depends on all θ and all x.

[0082] When the distribution p of a given data source S S (y|x; α) and the output distribution p o(x|θ,t), then the receiver distribution is defined as:

[0083]

[0084] Assuming that the input distribution models all variables in the data independently, the Bayesian update can be obtained in closed form. The Bayesian update distribution is defined by marginalizing y:

[0085]

[0086] where δ(·-a) is a multivariate Dirac distribution centered at vector a.

[0087] In traditional Bayesian updates, the input distribution receives information about each variable in the data independently, and contextual information is difficult to directly utilize, such as neighboring pixels in an image or related words in a text. In addition, the output distribution is generated by a multi-factor coupled Bayesian causal network, and the neural network jointly processes all parameters in the input distribution, enabling it to access all available context. This method combines the advantages of Bayesian reasoning and deep learning. The former provides a mathematically optimal, finely controllable way to collect and summarize information about a single variable, while the latter excels at integrating information from many interrelated variables.

[0088] The above transfer process defines an n-th order loss function, which can be generalized to continuous time by assigning n to ∞. In continuous time, Bayesian updates become Bayesian information flows from the data to the network. There is no need to predefine the number of steps during training, and continuous-time loss functions are mathematically simpler and easier to compute than discrete-time loss functions. During inference and sampling, the multi-factor coupled Bayesian causal inference trained with continuous-time loss can run any number of discrete steps, and performance improves with the number of steps.

[0089] (3) Relax the complex reasoning calculation of the OR gate to simplify the solution

[0090] Figure 3 Design a schematic diagram for relaxing the complex inference calculation of the OR gate, Figure 3 (a) is the Noisy-OR gate model, and (b) is the Leaky Noisy-OR gate model. Figure 3 As shown in Figure 1, the Noisy-OR gate model was proposed by Pearl and is one of the important tools for compact heuristics in Bayesian causal networks. A brief description is as follows: Assume that leaf node X and its parent node Y are both binary nodes, with "T" (True) or "F" (False) representing the true or false of a binary variable. Assume that each leaf node cannot be ignored and that leaf nodes X are independent of each other. In order to establish this Bayesian network, it is necessary to solve P(Y|X1,X2,…,XN ), the number of its parameters is 2 n The Noisy-OR gate model can reduce the number of parameters to be solved in this process. The basic idea is to first calculate i =T and {X k =T}(j≠i), the probability of Y=T is as follows:

[0091]

[0092] In the formula, the probability p i Also called marginal probability. Let X T For node X i The set when the state is "T", X F For node X i The set when the state is "F". Then,

[0093]

[0094] At this time, it is estimated that p(Y=T|X1,X2,…,X N ) has a number of parameters from 2 n becomes 2n.

[0095] In addition, in the process of modeling practical problems, it is difficult to obtain all leaf nodes, so a Leaky term is added, that is, node X i To represent all other unmodeled variables. The Leaky factor λ (0≤λ≤1) represents the probability that Y is true when all modeled factors are false, as shown in the following formula:

[0096]

[0097] Therefore, the standard form of the Leaky noisy OR model can be expressed as,

[0098]

[0099] Therefore, in the Leaky noisy OR model, only 2n weighted parameters and a Leaky factor λ are needed to solve the complete probability distribution table, thereby achieving the purpose of reducing the parameters for solving the full joint distribution.

[0100] (4) Abnormal target determination

[0101] Multi-factor coupling reasoning methods include but are not limited to Bayesian reasoning, evidence theory, and integrated learning. Bayesian reasoning is based on Bayesian probability theory, which calculates prior probabilities and likelihood functions and integrates multi-source data for reasoning; evidence theory integrates multiple sources of evidence, calculates trust and distrust, and makes decisions on targets; integrated learning: combines multiple machine learning models and integrates the decision results of each model through weighted voting or other methods.

[0102] Abnormal target conflict decision-making includes the formulation of decision rules, conflict resolution mechanism, and feedback mechanism. According to the multi-factor coupling reasoning method, specific decision rules are formulated to clarify how to make comprehensive judgments based on the inputs of different data sources; when there are decision conflicts between multiple data sources, specific mechanisms are used to resolve the conflicts and obtain the final judgment results; the detection results are fed back to the system for self-learning and parameter adjustment, and the detection model is continuously optimized to improve the accuracy of judgment.

[0103] In order to implement the above-mentioned embodiment, the present application also proposes a self-learning detection device for abnormal targets in complex urban backgrounds.

[0104] Figure 4 A schematic diagram of the structure of a self-learning detection device for abnormal targets in complex urban backgrounds provided in an embodiment of the present application.

[0105] like Figure 4 As shown, the complex urban background abnormal target self-learning detection device includes:

[0106] A data acquisition module is used to collect urban scene data through a multimodal visual sensor, and perform preliminary anomaly detection on the collected multimodal data to obtain preliminary anomaly detection results;

[0107] The uncertainty factor building module is used to add uncertainty knowledge to the preliminary anomaly detection results by using prior knowledge and data tuning to build an uncertainty model;

[0108] The relative entropy joint solution module is used to calculate and compare the relative entropy between different modal data based on the uncertainty model, and determine the resampling order according to the relative entropy difference and adjust the weight of different modal data;

[0109] The abnormal target determination module is used to perform multi-factor coupling reasoning on the collected multimodal data based on the adjusted weights to obtain abnormal detection results.

[0110] Optionally, in one embodiment of the present application, the multimodal visual sensor includes a visible light camera, an infrared camera, a multispectral camera, a hyperspectral camera, an event camera, and a radar sensor.

[0111] Optionally, in one embodiment of the present application, the prior knowledge includes environmental knowledge, historical data and expert knowledge of the urban context;

[0112] Data tuning includes data enhancement, data cleaning, and parameter adjustment;

[0113] The expressions of uncertain knowledge include Bayesian networks, fuzzy logic and confidence intervals.

[0114] Optionally, in one embodiment of the present application, the calculation method of relative entropy includes Kullback-Leibler divergence, cross entropy, and Jensen-Shannon divergence.

[0115] Optionally, in one embodiment of the present application, the multi-factor coupled reasoning includes Bayesian reasoning, evidence theory and integrated learning;

[0116] The abnormal target determination module is also used to:

[0117] When conducting multi-factor coupling reasoning, abnormal target conflict decisions are made, and when there are decision conflicts between multimodal data, the set mechanism is used to resolve the conflicts and obtain the final judgment results. Among them, the abnormal target conflict decision judgment includes: formulating decision rules, conflict resolution mechanisms and feedback mechanisms.

[0118] It should be noted that the aforementioned explanation of the embodiment of the self-learning detection method for abnormal targets in complex urban backgrounds is also applicable to the self-learning detection device for abnormal targets in complex urban backgrounds of this embodiment, and will not be repeated here.

[0119] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0120] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0121] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0122] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0123] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0124] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0125] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0126] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A self-learning detection method for abnormal targets in complex urban backgrounds, characterized in that: include: Collect urban scene data through multimodal visual sensors, perform preliminary anomaly detection on the collected multimodal data, and obtain preliminary anomaly detection results; Use prior knowledge and data optimization to add uncertainty knowledge to the preliminary anomaly detection results and build an uncertainty model; Based on the uncertainty model, the relative entropy between different modal data is calculated and compared, and the resampling order is determined according to the relative entropy difference, and the weight of different modal data is adjusted; Based on the adjusted weights, multi-factor coupling reasoning is performed on the collected multimodal data to obtain the anomaly detection results.

2. The method according to claim 1, characterized in that The multimodal visual sensor includes a visible light camera, an infrared camera, a multispectral camera, a hyperspectral camera, an event camera and a radar sensor.

3. The method according to claim 1, characterized in that The prior knowledge includes environmental knowledge of the urban context, historical data and expert knowledge; The data tuning includes data enhancement, data cleaning and parameter adjustment; The uncertainty knowledge is expressed in the following ways: Bayesian network, fuzzy logic and confidence interval.

4. The method according to claim 1, characterized in that The relative entropy is calculated by Kullback-Leibler divergence, cross entropy, and Jensen-Shannon divergence.

5. The method according to claim 1, characterized in that The multi-factor coupled reasoning includes Bayesian reasoning, evidence theory and integrated learning; The method further comprises: When conducting multi-factor coupling reasoning, abnormal target conflict decisions are made, and when there are decision conflicts between multimodal data, the set mechanism is used to resolve the conflicts and obtain the final judgment results. Among them, the abnormal target conflict decision judgment includes: formulating decision rules, conflict resolution mechanisms and feedback mechanisms.

6. A self-learning detection device for abnormal targets in complex urban backgrounds, characterized in that: include: A data acquisition module is used to collect urban scene data through a multimodal visual sensor, and perform preliminary anomaly detection on the collected multimodal data to obtain preliminary anomaly detection results; The uncertainty factor building module is used to add uncertainty knowledge to the preliminary anomaly detection results by using prior knowledge and data tuning to build an uncertainty model; A relative entropy joint solution module is used to calculate and compare the relative entropies between different modal data based on the uncertainty model, and determine the resampling order according to the relative entropy difference, and adjust the weights of different modal data; The abnormal target determination module is used to perform multi-factor coupling reasoning on the collected multimodal data based on the adjusted weights to obtain abnormal detection results.

7. The device according to claim 6, characterized in that The multimodal visual sensor includes a visible light camera, an infrared camera, a multispectral camera, a hyperspectral camera, an event camera and a radar sensor.

8. The device according to claim 6, characterized in that The prior knowledge includes environmental knowledge of the urban context, historical data and expert knowledge; The data tuning includes data enhancement, data cleaning and parameter adjustment; The uncertainty knowledge is expressed in the following ways: Bayesian network, fuzzy logic and confidence interval.

9. The device according to claim 6, characterized in that The relative entropy is calculated by Kullback-Leibler divergence, cross entropy, and Jensen-Shannon divergence.

10. The device according to claim 6, characterized in that The multi-factor coupled reasoning includes Bayesian reasoning, evidence theory and integrated learning; The abnormal target determination module is further used for: When conducting multi-factor coupling reasoning, abnormal target conflict decisions are made, and when decision conflicts exist between multimodal data, the set mechanism is used to resolve the conflicts and obtain the final judgment result. Among them, the abnormal target conflict decision judgment includes: formulating decision rules, conflict resolution mechanism and feedback mechanism.

Images