A detection and recognition method and device based on overlapping window constraints

Abstract

The present invention discloses a detection and identification method and device based on overlapping window constraints, the method comprising: obtaining multi-source traffic data; extracting feature data of the multi-source traffic data and constructing a traffic feature data set; setting constraints on features of interest in the traffic feature data set, filtering out redundant features, and establishing windows corresponding to the constraints; wherein only corresponding feature types that meet the constraints are included after the window; combining windows corresponding to different constraints to construct overlapping windows, so that the output of the overlapping windows can reflect comprehensive features with interpretable physical meanings; integrating the comprehensive features with a preset network model to construct an interpretable deep neural network, and then completing the detection and identification of different traffic events according to actual traffic data. The present invention can greatly compress the amount of data when processing multi-source traffic data with a huge amount of information, and provide convenient prerequisites for subsequent processing and statistical analysis of different types of traffic features.

Description

Technical Field

[0001] The present invention relates to the technical fields of multi-source data information processing and intelligent information processing, and in particular to a detection and recognition method and device based on overlapping window constraints. Background Art

[0002] In the specific application of autonomous driving, it is often necessary to extract and analyze different types of traffic features such as vehicle rear-end collisions, road congestion, speeding, obstacle recognition, etc. However, the traffic data source has a huge amount of data and too much redundant information and features. If the original data cannot be effectively classified and compressed, the calculation of detection, recognition, classification, statistical analysis, etc. will be huge and inefficient. Summary of the invention

[0003] The present invention provides a detection and identification method and device based on overlapping window constraints to solve the problem that the current detection, identification, classification, statistical analysis and the like based on traffic data sources have huge computational complexity and low efficiency.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0005] In one aspect, the present invention provides a detection and recognition method based on overlapping window constraints, comprising:

[0006] Obtain traffic data from multiple sources;

[0007] Extracting characteristic data of the multi-source traffic data and constructing a traffic characteristic data set;

[0008] Setting constraints on the features of interest in the traffic feature data set, filtering out redundant features, and establishing a window corresponding to the constraints; wherein only the corresponding feature types that meet the constraints are included behind the window;

[0009] Combining windows corresponding to different constraint conditions to construct overlapping windows, so that the output of the overlapping windows can reflect comprehensive features with interpretable physical meanings;

[0010] The comprehensive features are integrated with the preset network model to construct an interpretable deep neural network. The deep neural network is used to detect and identify different traffic events based on actual traffic data.

[0011] Furthermore, the multi-source traffic data is multi-source visual imaging traffic data obtained by describing the same or different traffic information using different detection systems; wherein the detection system includes any one or more combinations of hyperspectral, multispectral, visible light, infrared and lidar.

[0012] Furthermore, feature data of multi-source traffic data is extracted to construct a traffic feature data set, including:

[0013] Extract characteristic data of the multi-source traffic data, describe the traffic characteristics, and construct a traffic characteristic data set containing preset traffic characteristics; wherein the preset traffic characteristics include one or more combinations of WADT, location vehicle speed, road network density, and vehicle turnover.

[0014] Furthermore, the setting of the constraint conditions includes stipulating the setting of corresponding feature types, quantities and data set source attributes.

[0015] Furthermore, setting constraint conditions for the features of interest in the traffic feature data set, filtering out redundant features, and establishing windows corresponding to the constraint conditions include:

[0016] A variety of preset network models are adopted, and the layer structure of the network model itself and the feature set of unsupervised self-learning are used to increase the diversity of the acquired windows.

[0017] Furthermore, combining windows corresponding to different constraint conditions to construct overlapping windows includes:

[0018] Combine different windows, as well as windows and features, to build overlapping windows.

[0019] Furthermore, the comprehensive features are integrated with a preset network model to construct an interpretable deep neural network, and the deep neural network is used to complete the detection and identification of different traffic events according to actual traffic data, including:

[0020] Assume that the multi-source traffic data are S1, S2, ..., S n , the feature types corresponding to different data sources are recorded as X 1-1 ,…,X i-j ;X i-j Represents the i-th data source S i The jth feature type in the feature set is constructed to construct different constraints for the same or different feature types X from different or the same data sources. i-j Perform constraint screening, and obtain the corresponding comprehensive feature classification F through different overlap functions for different constraint conditions n , put the actual collected traffic data into the constructed deep neural network, and n Fusion learning is performed to obtain the final detection and recognition results.

[0021] On the other hand, the present invention also provides a detection and identification device based on overlapping window constraints, comprising:

[0022] A data acquisition module, used to acquire multi-source traffic data;

[0023] A traffic feature data set construction module, used to extract feature data of the multi-source traffic data acquired by the data acquisition module, and construct a traffic feature data set;

[0024] A window construction module is used to set constraints on the features of interest in the traffic feature data set constructed by the traffic feature data set construction module, filter out redundant features, and establish a window corresponding to the constraint conditions; wherein only the corresponding feature types that meet the constraint conditions are included after the window;

[0025] A window combination module, used for combining windows corresponding to different constraint conditions to construct overlapping windows, so that the output of the overlapping windows can reflect comprehensive features with interpretable physical meanings;

[0026] The detection and identification module is used to integrate the comprehensive features obtained by the window combination module with the preset network model to construct an interpretable deep neural network, and use the deep neural network to complete the detection and identification of different traffic events based on actual traffic data.

[0027] Furthermore, the multi-source traffic data is multi-source visual imaging traffic data obtained by using different detection systems to describe the same or different traffic information; wherein the detection system includes any one or more combinations of hyperspectral, multispectral, visible light, infrared and laser radar;

[0028] The traffic feature data set construction module is specifically used for:

[0029] Extracting characteristic data of the multi-source traffic data, and describing the traffic characteristics, and constructing a traffic characteristic data set including preset traffic characteristics; wherein the preset traffic characteristics include a combination of one or more of WADT, location vehicle speed, road network density, and vehicle turnover;

[0030] The setting of the constraint conditions includes setting the corresponding feature types, quantities and data set source attributes;

[0031] The window construction module is specifically used to: adopt a variety of preset network models, and use the layer structure of the network model itself and the feature set of unsupervised self-learning to increase the diversity of the acquired windows;

[0032] The window combination module is specifically used for:

[0033] Combine different windows, as well as windows and features, to build overlapping windows.

[0034] Furthermore, the detection and identification module is specifically used for:

[0035] Assume that the multi-source traffic data are S1, S2, ..., S n, the feature types corresponding to different data sources are recorded as X 1-1 ,…,X i-j ;X i-j Represents the i-th data source S i The jth feature type in the feature set is constructed to construct different constraints for the same or different feature types X from different or the same data sources. i-j Perform constraint screening, and obtain the corresponding comprehensive feature classification F through different overlap functions for different constraint conditions n , put the actual collected traffic data into the constructed deep neural network, and n Fusion learning is performed to obtain the final detection and recognition results.

[0036] On the other hand, the present invention further provides an electronic device, comprising a processor and a memory; wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the above method.

[0037] In yet another aspect, the present invention further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, and the instruction is loaded and executed by a processor to implement the above method.

[0038] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0039] The present invention obtains multi-source traffic data; extracts feature data of the multi-source traffic data to construct a traffic feature data set; sets constraints on features of interest in the traffic feature data set, screens out redundant features, and establishes windows corresponding to the constraints; combines windows corresponding to different constraints to construct overlapping windows, so that the output of the overlapping windows can reflect comprehensive features with interpretable physical meanings; integrates the comprehensive features with a preset network model to construct an interpretable deep neural network, and uses the deep neural network to complete the detection and identification of different traffic events based on actual traffic data. Thus, overlapping window constraints are applied to feature extraction of deep learning, which improves the interpretability of feature learning. Furthermore, when processing multi-source traffic data with a huge amount of information, the amount of data can be greatly compressed, and convenient prerequisites are provided for subsequent processing and statistical analysis of different types of traffic features. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0041] Figure 1is a flow chart of a detection and identification method based on overlapping window constraints provided by an embodiment of the present invention;

[0042] Figure 2 It is a schematic diagram of a detection and recognition method based on overlapping window constraints provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0044] First embodiment

[0045] This embodiment provides a detection and recognition method based on overlapping window constraints. The method can be implemented by an electronic device. The execution process of the method is as follows: Figure 1 As shown, the following steps are included:

[0046] S1, obtain multi-source traffic data;

[0047] Specifically, in this embodiment, Figure 2 As shown, multi-source traffic data is described and acquired by using different detection systems such as hyperspectral, multispectral, visible light, infrared, and lidar for the same or different traffic information. Moreover, it should be noted that this embodiment only takes the image data source as an example to explain the detection and recognition method based on overlapping window constraints of the present invention, but this embodiment does not limit the specific type of data. When this method is used for other types of data sources, it is still within the scope of protection of the claims of this specification.

[0048] S2, extracting feature data from multi-source traffic data and constructing a traffic feature dataset;

[0049] Specifically, in this embodiment, the implementation process of the above S2 is: extract the feature data of the acquired multi-source visual imaging traffic data and describe the traffic characteristics, construct a feature data set of traffic characteristics such as WADT, location vehicle speed, road network density and vehicle turnover, and complete the multi-dimensional feature description of different types of targets.

[0050] S3, setting constraints on the features of interest in the traffic feature data set, filtering out redundant features, and establishing a window corresponding to the constraints; wherein only the corresponding feature types that meet the constraints are included behind the window;

[0051] Specifically, in this embodiment, the implementation process of the above S3 is as follows:

[0052] Conditional constraints are designed and constructed for the features of interest in the above traffic feature datasets, and the corresponding feature types, quantities, dataset sources and other attributes are set to filter out redundant features and retain specific feature combinations. Different constraints correspond to different windows, through which only the feature type sets that meet the corresponding constraints can be seen, and the remaining feature types in the feature set are filtered out.

[0053] Moreover, it should be noted that due to the high complexity of traffic data and the large amount of information, when designing and constructing constraints, simply constructing linear and nonlinear conditional constraints is difficult to comprehensively cover various road traffic information. In this regard, this embodiment intends to adopt a variety of network models such as CNN and Transformer, and use the layer structure of the model itself and the feature set of unsupervised self-learning to increase the diversity of acquired windows, such as the Self-Attention mechanism in the Transformer model and the convolutional layer in the CNN model, which greatly increases the diversity of windows.

[0054] S4, combining windows corresponding to different constraint conditions to construct a stacked window, so that the output of the stacked window can reflect comprehensive features with interpretable physical meanings;

[0055] Specifically, in this embodiment, the implementation process of the above S4 is: combine different windows, and windows and features to ensure that the output of the overlapping windows can reflect comprehensive features with interpretable physical meanings, and name them for subsequent practical applications; wherein the combination method is not limited.

[0056] This concept can be applied to the interpretability of deep learning. The above-mentioned "window" concept can correspond to the different features extracted during the convolution process. The impact on the final feature extraction result can be judged according to the different linear and nonlinear combinations of the window, so as to explain the features in the learning process, that is, which features have an impact on the final result and which features are useless to the final result. Parameters can be designed to quantify the impact of different features on the results, so as to achieve the purpose of explaining the model.

[0057] S5 integrates comprehensive features with preset network models to build an interpretable deep neural network, and then completes the rapid and accurate detection and identification of different traffic events based on actual traffic data.

[0058] Specifically, in this embodiment, the implementation process of the above S5 is as follows:

[0059] For the actual traffic data, the comprehensive features obtained by S4 are integrated with existing network models such as CNN, Transformer, RNN, etc. For example, the combined comprehensive features P, Q, and X are respectively put into the kth, lth, and mth convolutional layers of the CNN network for fusion learning to construct an interpretable deep neural network to quickly and accurately detect and identify different traffic events such as speeding, obstacles, and vehicle rear-end collisions;

[0060] Assume that the multi-source traffic data are S1, S2, ..., S n , the feature types corresponding to different data sources are recorded as X 1-1 ,...,X i-j , i, j represent the jth feature type in the feature set of the i-th data source, respectively, and construct different constraint conditions S.tκ for the same or different feature types X from different or the same data sources. i-j Constraint screening is performed, and different constraints are obtained through different overlap functions ξ(·) to obtain the corresponding comprehensive feature classification F n , put the actual traffic data into the constructed network, and n Fusion learning, get the final detection and recognition result Resultα:

[0061] S.tκ=f(…,X i-j )

[0062] F n =ξ(…,S.tκ)

[0063] Result α=Net(…,F n )

[0064] In summary, this embodiment provides a detection and identification method based on overlapping window constraints, which obtains multi-source traffic data; extracts feature data of the multi-source traffic data to construct a traffic feature data set; sets constraints on features of interest in the traffic feature data set, screens out redundant features, and establishes windows corresponding to the constraints; combines windows corresponding to different constraints to construct overlapping windows, so that the output of the overlapping windows can reflect comprehensive features with interpretable physical meanings; integrates the comprehensive features with a preset network model to construct an interpretable deep neural network, and uses the deep neural network to complete the detection and identification of different traffic events based on actual traffic data. Thus, overlapping window constraints are applied to feature extraction of deep learning, which improves the interpretability of feature learning. In addition, when processing multi-source traffic data with a huge amount of information, the amount of data can be greatly compressed, and convenient prerequisites are provided for subsequent processing and statistical analysis of different types of traffic features.

[0065] Second embodiment

[0066] This embodiment provides a detection recognition transposition based on overlapping window constraints, including the following modules:

[0067] A data acquisition module, used to acquire multi-source traffic data;

[0068] A traffic feature data set construction module, used to extract feature data of the multi-source traffic data acquired by the data acquisition module, and construct a traffic feature data set;

[0069] A window construction module is used to set constraints on the features of interest in the traffic feature data set constructed by the traffic feature data set construction module, filter out redundant features, and establish a window corresponding to the constraint conditions; wherein only the corresponding feature types that meet the constraint conditions are included after the window;

[0070] A window combination module, used for combining windows corresponding to different constraint conditions to construct overlapping windows, so that the output of the overlapping windows can reflect comprehensive features with interpretable physical meanings;

[0071] The detection and identification module is used to integrate the comprehensive features obtained by the window combination module with the preset network model to construct an interpretable deep neural network, and use the deep neural network to complete the detection and identification of different traffic events based on actual traffic data.

[0072] The detection and identification device based on overlapping window constraints of this embodiment corresponds to the detection and identification method based on overlapping window constraints of the first embodiment mentioned above; wherein, the functions implemented by each functional module in the detection and identification device based on overlapping window constraints of this embodiment correspond one by one to each process step in the detection and identification method based on overlapping window constraints of the first embodiment mentioned above; therefore, they will not be repeated here.

[0073] Third embodiment

[0074] This embodiment provides an electronic device, which includes a processor and a memory; wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the method of the first embodiment.

[0075] The electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) and one or more memories, wherein the memory stores at least one instruction, and the instruction is loaded by the processor to execute the above method.

[0076] Fourth embodiment

[0077] This embodiment provides a computer-readable storage medium, which stores at least one instruction, and the instruction is loaded and executed by a processor to implement the method of the first embodiment. The computer-readable storage medium may be a ROM, a random access memory, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc. The instructions stored therein may be loaded by a processor in a terminal to execute the method.

[0078] In addition, it should be noted that the present invention can be provided as a method, an apparatus or a computer program product. Therefore, the embodiments of the present invention can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0079] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0080] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0081] It should also be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0082] Finally, it should be noted that the above is a preferred embodiment of the present invention. It should be pointed out that although the preferred embodiment of the present invention has been described, for those skilled in the art, once the basic creative concept of the present invention is known, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention. Therefore, the attached claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A detection and recognition method based on overlapping window constraints, characterized in that: include: Obtain traffic data from multiple sources; Extracting characteristic data of the multi-source traffic data and constructing a traffic characteristic data set; Setting constraints on the features of interest in the traffic feature data set, filtering out redundant features, and establishing a window corresponding to the constraints; wherein only the corresponding feature types that meet the constraints are included behind the window; Combining windows corresponding to different constraint conditions to construct overlapping windows, so that the output of the overlapping windows can reflect comprehensive features with interpretable physical meanings; The comprehensive features are integrated with the preset network model to construct an interpretable deep neural network, and the deep neural network is used to detect and identify different traffic events based on actual traffic data; The comprehensive features are integrated with the preset network model to construct an interpretable deep neural network, and the deep neural network is used to complete the detection and identification of different traffic events according to actual traffic data, including: Assume that the multi-source traffic data are S1, S2, ..., S n , the feature types corresponding to different data sources are recorded as X 1-1 ,…,X i-j ;X i-j Represents the i-th data source S i The jth feature type in the feature set is constructed to construct different constraints for the same or different feature types X from different or the same data sources. i-j Perform constraint screening, and obtain the corresponding comprehensive feature classification F through different overlap functions for different constraint conditions n , put the actual collected traffic data into the constructed deep neural network, and n Fusion learning is performed to obtain the final detection and recognition results.

2. The detection and recognition method based on overlapping window constraints as claimed in claim 1, characterized in that: The multi-source traffic data is multi-source visual imaging traffic data obtained by using different detection systems to describe the same or different traffic information; wherein the detection system includes any one or more combinations of hyperspectral, multispectral, visible light, infrared and lidar.

3. The detection and recognition method based on overlapping window constraints as claimed in claim 1, characterized in that: The step of extracting characteristic data of the multi-source traffic data and constructing a traffic characteristic data set includes: Extract characteristic data of the multi-source traffic data, describe the traffic characteristics, and construct a traffic characteristic data set containing preset traffic characteristics; wherein the preset traffic characteristics include one or more combinations of WADT, location vehicle speed, road network density, and vehicle turnover.

4. The detection and recognition method based on overlapping window constraints as claimed in claim 1, characterized in that: The setting of the constraint conditions includes setting corresponding feature types, quantities and data set source attributes.

5. The detection and recognition method based on overlapping window constraints as claimed in claim 1, characterized in that: The step of setting constraint conditions for the features of interest in the traffic feature data set, filtering out redundant features, and establishing a window corresponding to the constraint conditions includes: A variety of preset network models are adopted, and the layer structure of the network model itself and the feature set of unsupervised self-learning are used to increase the diversity of the acquired windows.

6. The detection and recognition method based on overlapping window constraints as claimed in claim 1, characterized in that: The combining of windows corresponding to different constraints to construct overlapping windows includes: Combine different windows, as well as windows and features, to build overlapping windows.

7. A detection and identification device based on overlapping window constraints, characterized in that: include: A data acquisition module, used to acquire multi-source traffic data; A traffic feature data set construction module, used to extract feature data of the multi-source traffic data acquired by the data acquisition module, and construct a traffic feature data set; A window construction module is used to set constraints on the features of interest in the traffic feature data set constructed by the traffic feature data set construction module, filter out redundant features, and establish a window corresponding to the constraint conditions; wherein only the corresponding feature types that meet the constraint conditions are included after the window; A window combination module, used for combining windows corresponding to different constraint conditions to construct overlapping windows, so that the output of the overlapping windows can reflect comprehensive features with interpretable physical meanings; A detection and identification module, which is used to integrate the comprehensive features obtained by the window combination module with a preset network model to construct an interpretable deep neural network, and use the deep neural network to complete the detection and identification of different traffic events based on actual traffic data; The detection and identification module is specifically used for: Assume that the multi-source traffic data are S1, S2, ..., S n , the feature types corresponding to different data sources are recorded as X 1-1 ,…,X i-j ;X i-j Represents the i-th data source S i The jth feature type in the feature set is constructed to construct different constraints for the same or different feature types X from different or the same data sources. i-j Perform constraint screening, and obtain the corresponding comprehensive feature classification F through different overlap functions for different constraint conditions n , put the actual collected traffic data into the constructed deep neural network, and n Fusion learning is performed to obtain the final detection and recognition results.

8. The detection and identification device based on overlapping window constraints as claimed in claim 7, characterized in that: The multi-source traffic data is multi-source visual imaging traffic data obtained by using different detection systems to describe the same or different traffic information; wherein the detection system includes any one or more combinations of hyperspectral, multispectral, visible light, infrared and laser radar; The traffic feature data set construction module is specifically used for: Extracting characteristic data of the multi-source traffic data, and describing the traffic characteristics, and constructing a traffic characteristic data set including preset traffic characteristics; wherein the preset traffic characteristics include a combination of one or more of WADT, location vehicle speed, road network density, and vehicle turnover; The setting of the constraint conditions includes setting the corresponding feature types, quantities and data set source attributes; The window construction module is specifically used to: adopt a variety of preset network models, and use the layer structure of the network model itself and the feature set of unsupervised self-learning to increase the diversity of the acquired windows; The window combination module is specifically used for: Combine different windows, as well as windows and features, to build overlapping windows.

Images