System for realizing multi-element analysis model customization data generation processing for video investigation

Abstract

The application relates to a system for realizing multi-element analysis model customized data generation processing for video investigation, wherein the system comprises: an image feature editing preprocessing module, which is used for pre-processing original images in a mosaic mode according to the demand characteristics of a video investigation analysis model, and removing redundant features with low correlation through feature editing; a customized data generation module, which is used for performing restoration operation on acquired image sample data through a task-oriented autoencoder model; and a multi-element analysis model sustainable optimization module, which is used for constructing an analysis model training and evaluation system based on customized generated data, and performing adaptive screening on newly generated data samples according to model training and evaluation results. The system for realizing multi-element analysis model customized data generation processing for video investigation can effectively solve the problem of mutual mismatch between models and data in the current system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, especially to the technical field of multi-element feature model, and specifically to a system for realizing multi-element analysis model customized data generation processing for video investigation. BACKGROUND

[0002] The multi-element analysis model for video investigation mainly analyzes and models the key attention elements in the monitoring scene, so as to meet the real-time analysis and query requirements of various targets. The existing analysis model usually adopts the current mainstream deep network structure, and the dependence on data quality is high. However, in the process of constructing the data set, the diversity of model attention elements and training tasks is not fully considered, which leads to the problem of mutual mismatch between the model and the data. Specifically, in actual application, the existing video investigation system mainly has the following defects:

[0003] 1) High data redundancy: there are a large amount of feature information in the original data that has low relevance to the model attention elements and training tasks, which reduces the model training efficiency and is not conducive to ensuring the privacy and security of the data;

[0004] 2) Low data diversity: due to the constraints of data collection environment and other conditions, it is difficult to obtain diversified feature information that meets the training needs of the multi-element analysis model through the original data;

[0005] 3) It is difficult to continuously optimize the data: it is impossible to realize sustainable learning through the cyclic iteration of the model and the data in the actual measurement environment, which leads to the deterioration of the generalization performance of the model. SUMMARY

[0006] The purpose of the present application is to overcome the above-mentioned shortcomings of the prior art, and to provide a system for realizing multi-element analysis model customized data generation processing for video investigation with model-oriented features.

[0007] In order to achieve the above-mentioned purpose, the system for realizing multi-element analysis model customized data generation processing for video investigation of the present application is as follows:

[0008] The system for realizing multi-element analysis model customized data generation processing for video investigation, its main feature is that the system comprises:

[0009] An image feature editing preprocessing module is used to preprocess the original image in a mosaic manner according to the demand characteristics of the video investigation analysis model, and to remove redundant features with low relevance through feature editing, thereby improving the safety and reliability of the data;

[0010] The customized data generation module is connected with the image feature editing preprocessing module, and the customized data generation module adopts a task-oriented autoencoder model to perform restoration operation on the obtained image sample data while retaining key features with high correlation.

[0011] The multi-element analysis model sustainable optimization module is connected with the customized data generation module, and is used for constructing an analysis model training and evaluation system based on the generated customized data, and performing adaptive screening on newly generated data samples according to model training and evaluation results, so as to improve the effectiveness of the multi-element analysis model for video investigation.

[0012] Preferably, the requirement characteristics of the video investigation analysis model are as follows:

[0013] The multi-element analysis model has different attention elements and training tasks in various video investigation related application scenarios, wherein the attention elements include pedestrians, motor vehicles and non-motor vehicles in the video investigation scenario, and the training tasks include target detection, attribute analysis and feature retrieval in the video investigation scenario.

[0014] Preferably, the image feature editing preprocessing module specifically performs the following processing:

[0015] The position range of the foreground region and the background region is determined on the original obtained image sample according to the attention elements of the multi-element analysis model.

[0016] The mosaic region position range and the proportion size of the image sample are further selected according to the training task of the multi-element analysis model.

[0017] Preferably, the image feature editing preprocessing module adopts the following way for target detection processing:

[0018] A larger mosaic region proportion is set on the picture sample, and features with low correlation to the detection task including image texture are removed; at the same time, the mosaic region near the foreground edge is reduced, so as to retain features with high correlation to the detection task including target contour.

[0019] Preferably, the image feature editing preprocessing module adopts the following way for attribute analysis and feature retrieval processing:

[0020] A larger mosaic region proportion is set outside the foreground region of the picture sample, and features with low correlation to the task including image background are removed; at the same time, a smaller mosaic region proportion is set inside the foreground region, so as to retain features with high correlation to the task including target details.

[0021] Preferably, the customized data generation module adopts an autoencoder framework for data generation, which includes an encoder and a decoder, wherein the encoder is a deep network model with high complexity, used for feature encoding of the mosaic image, and the decoder is a lightweight model with low complexity, used for feature restoration of the original image.

[0022] The customized data generation module adopts a discriminative model to perform task-related constraint filtering on the restoration results to eliminate generated data that do not meet the prediction consistency.

[0023] Preferably, the customized data generation module trains the original data into a target detection model, an attribute analysis model, and a feature retrieval model according to the task requirements of the multi-element analysis model, and for each original sample and corresponding generated sample, the following requirements need to be met according to the model task characteristics:

[0024] For the target detection model: the output target class and position information meet the consistency;

[0025] For the attribute analysis model: the output target attribute and confidence information meet the consistency;

[0026] For the feature retrieval model: the output target identity and feature similarity meet the consistency.

[0027] Preferably, after the customized data generation module completes the consistency filtering process according to each model, the generated samples will be scored by a quality evaluation model, and low-quality data that have obvious differences from the real data will be filtered out according to the authenticity of the output results.

[0028] Preferably, the analysis model training and evaluation system specifically comprises:

[0029] The newly generated sample data is introduced into the training and evaluation data of the multi-element analysis model, and a self-iteration framework is formed by the mutual circulation of the data generation submodule, the model training submodule, and the model evaluation submodule.

[0030] Preferably, the multi-element analysis model sustainable optimization module filters the newly generated data samples according to the model training and evaluation results in the mutual circulation process of the data generation submodule, the model training submodule, and the model evaluation submodule, thereby realizing adaptive updating of the model and the data.

[0031] The system for realizing multi-element analysis model customized data generation processing for video investigation in the application can effectively solve the problem of mutual mismatch between the model and the data in the current system. In particular, in some long-tail business scenarios, the model task-oriented data generation can further improve the explainability and scene generalization ability of the analysis model, and has relatively outstanding applicability. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 The structural diagram of the system for realizing multi-element analysis model customized data generation processing for video investigation in the application.

[0033] Figure 2 The structural diagram of the image feature editing preprocessing module in the application.

[0034] Figure 3 The structural diagram of the customized data generation module in the application.

[0035] Figure 4 The structural diagram of the multi-element analysis model sustainable optimization module in the application. DETAILED DESCRIPTION

[0036] In order to more clearly describe the technical content of the application, the following further describes in combination with specific embodiments.

[0037] Before the detailed description of the embodiments according to the application, it should be noted that in the following, the terms "comprise", "include" or any other variant is intended to cover the non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes these elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device.

[0038] In view of the main problems existing in the multi-element analysis model of the video investigation system, the overall goal of the technical solution is to design and realize a model-oriented customized data generation system combined with various concerned elements and related training tasks in actual scenarios. The technology adopts different mosaic methods for preprocessing operation on the original data, removes redundant features with low correlation through feature editing, and can effectively solve the problem of reduced model training efficiency caused by high data redundancy. On the other hand, the task-oriented autoencoder model is used for restoration operation, which can effectively retain key features with high correlation, and introduce more task-related features in the customized data generation process, further enhancing the diversity of the data. In addition, in the training and evaluation process of the multi-element analysis model, the self-iteration system of model and data cyclic optimization is used to continuously optimize the data in the actual measurement environment, thereby effectively improving the generalization ability of the analysis model in different scenarios.

[0039] Please refer toFigure 1 The system shown realizes multi-element analysis model customization data generation processing for video investigation, wherein the system comprises:

[0040] An image feature editing preprocessing module is configured to perform preprocessing operation on original images in a mosaic manner according to the requirement characteristics of the video investigation analysis model, and remove redundant features with low correlation through feature editing to improve the security and reliability of data.

[0041] A customized data generation module is connected with the image feature editing preprocessing module, and is configured to perform restoration operation on the obtained image sample data using a task-oriented autoencoder model while retaining key features with high correlation.

[0042] A multi-element analysis model sustainable optimization module is connected with the customized data generation module, configured to construct an analysis model training and evaluation system based on the generated customized data, and perform adaptive screening on newly generated data samples according to the model training and evaluation results, thereby improving the effectiveness of the multi-element analysis model for video investigation.

[0043] Please refer to Figure 2 As a preferred embodiment of the present application, the requirement characteristics of the video investigation analysis model are specifically:

[0044] The multi-element analysis model has different attention elements and training tasks in various video investigation related application scenarios; wherein the attention elements include pedestrians, motor vehicles and non-motor vehicles in the video investigation scene, and the training tasks include target detection, attribute analysis and feature retrieval in the video investigation scene.

[0045] As a preferred embodiment of the present application, the image feature editing preprocessing module specifically performs the following processing:

[0046] According to the attention elements of the multi-element analysis model, the position range of the foreground region and the background region in the original obtained image sample is determined.

[0047] According to the training tasks of the multi-element analysis model, the mosaic region position range and the proportion size of the image sample are further selected.

[0048] As a preferred embodiment of the present application, the image feature editing preprocessing module performs target detection processing in the following manner:

[0049] A large mosaic region proportion is set on the picture sample, and features with low correlation to the detection task including image texture are removed; at the same time, the mosaic region near the foreground edge is reduced, thereby retaining features with high correlation to the detection task including target contour.

[0050] As a preferred embodiment of the present application, the image feature editing preprocessing module performs attribute analysis and feature retrieval processing in the following manner:

[0051] A larger mosaic area ratio is set outside the foreground area of the picture sample, and features related to the task with low relevance including the image background are removed; meanwhile, a smaller mosaic area ratio is set inside the foreground area, thereby retaining features related to the task with high relevance including target details.

[0052] Please refer to Figure 3 As a preferred embodiment of the present application, the customization data generation module uses a self-encoder framework for data generation, as shown in the figure. The self-encoder framework includes an encoder and a decoder, wherein the encoder is a deep network model with high complexity, used for feature encoding of the mosaic image, and the decoder is a lightweight model with low complexity, used for feature restoration of the original image.

[0053] The customization data generation module uses a discriminative model to perform task-related constraint filtering on the restoration result, and removes generated data that does not meet the prediction consistency.

[0054] As a preferred embodiment of the present application, the customization data generation module trains the original data into a target detection model, an attribute analysis model, and a feature retrieval model according to the task requirements of the multi-element analysis model, and for each original sample and the corresponding generated sample, the following requirements need to be met according to the model task characteristics:

[0055] For the target detection model: the output target class and position information meet consistency;

[0056] For the attribute analysis model: the output target attribute and confidence information meet consistency;

[0057] For the feature retrieval model: the output target identity and feature similarity meet consistency.

[0058] As a preferred embodiment of the present application, after the customization data generation module completes consistency filtering processing according to each model, the generated sample will be scored by a quality evaluation model, and low-quality data that has obvious differences from real data will be filtered out according to the authenticity of the output result.

[0059] Please refer to Figure 4 As a preferred embodiment of the present application, the analysis model training and evaluation system is as follows:

[0060] The newly generated sample data is introduced into the training and evaluation data of the multi-element analysis model, and a mutual circulation self-iteration framework is formed by the data generation submodule, the model training submodule and the model evaluation submodule.

[0061] As a preferred embodiment of the application, the multi-element analysis model sustainable optimization module can filter the newly generated data samples according to the model training and evaluation results in the mutual circulation process of the data generation submodule, the model training submodule and the model evaluation submodule, so as to realize adaptive updating of the model and the data.

[0062] As shown in Figure 1 The system for realizing multi-element analysis model customized data generation processing designed by the application for video investigation mainly includes three functional modules of image feature editing preprocessing, customized data generation and multi-element analysis model sustainable optimization.

[0063] Firstly, the image feature editing preprocessing module (module 1, as shown in Figure 2 The module 1-1 determines the position range of the foreground region and the background region on the original sample according to the concerned elements of the multi-element analysis model, and specifically includes various concerned targets such as pedestrians, motor vehicles and non-motor vehicles in the video investigation scene.

[0064] 1) For the target detection task: a larger mosaic region proportion is set to remove the image texture and other features with low relevance to the detection task; and the mosaic region near the edge of the foreground (indicated by the dashed line in the figure) is reduced to retain the target contour and other features with high relevance to the detection task.

[0065] 2) For the attribute analysis and feature retrieval task: a larger mosaic region proportion is set outside the foreground region to remove the image background and other features with low relevance to the task; and a smaller mosaic region proportion is set inside the foreground region to retain the target details and other features with high relevance to the task.

[0066] Secondly, the task-oriented customized data generation module (module 2, as shown in Figure 3The new data samples are randomly generated based on the migration learning framework to introduce task-related common sense features, thereby effectively enhancing the diversity of the original data set. In module 2-1, the application generates data using a self-encoder framework, which includes an encoder and a decoder. The encoder model is a deep network model with high complexity, which is used to encode the features of the mosaic image. The decoder is a lightweight model with low complexity, which is used to restore the features of the original image. In module 2-2, the application uses a discriminative model to filter the restoration results based on task-related constraints. The generated data that does not meet the prediction consistency is removed, i.e., each original sample and the corresponding generated sample needs to meet the following requirements according to the characteristics of the model task:

[0067] 1) For the target detection model: the output target class and position information are consistent.

[0068] 2) For the attribute analysis model: the output target attribute and confidence information are consistent.

[0069] 3) For the feature retrieval model: the output target identity and feature similarity are consistent.

[0070] In module 2-3, the application uses a quality evaluation model to score the data and filter low-quality data that is significantly different from the real data.

[0071] Finally, the application can continuously optimize the multi-element analysis model (module 3, as shown in Figure 4 The data generation (module 3-1), model training (module 3-2), and model evaluation (module 3-3) three sub-modules are cyclically optimized in the self-iterative framework. According to the model training and evaluation results, the newly generated data samples are adaptively selected, which realizes continuous optimization without additional annotation cost, and further improves the generalization performance of the multi-element analysis model in the real environment.

[0072] Any process or method descriptions or any other information outlined in the flow charts or described elsewhere herein can be understood as representing embodiments of embodiments of the application, wherein functions performed by the illustrated operational blocks (or other elements appearing in the flow charts) can be implemented as code means employing available technologies by those skilled in the relevant art to implement the specific logical functions or steps in the operation supported by the elements and combined actions. The scope of embodiments of the application includes further implementations in which the order of operations can be changed, including according to the functionality involved, performed at substantially the same time or carried out in reverse order, understood by those skilled in the art to which embodiments of the application belong.

[0073] It should be understood that parts of the present application can be realized in hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be realized by software or firmware stored in a memory and executed by a suitable instruction execution apparatus.

[0074] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiments can be completed by programs instructing related hardware, and the programs can be stored in a computer readable storage medium, and when the programs are executed, one or a combination of the steps of the method embodiments is included.

[0075] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0076] In the description of the present specification, the description of the terms "an embodiment", "some embodiments", "an example", "a specific example" or "embodiments" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0077] Although the embodiments of the present application have been shown and described above, it is understood that the above-mentioned embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-mentioned embodiments within the scope of the present application.

[0078] The system for generating multi-element analysis model customized data using the video investigation-oriented implementation of the present application can effectively solve the problem of mutual mismatch between models and data in the current system. In particular, in some long-tail business scenarios, model task-oriented data generation can further improve the explainability and scene generalization ability of the analysis model, and has more prominent applicability.

[0079] In this specification, the present application has been described with reference to its specific embodiments. However, it is obvious that various modifications and changes can be made without departing from the spirit and scope of the present application. Therefore, the specification and drawings should be considered as illustrative rather than limiting.

Claims

1. A system for customized data generation and processing of multivariate analytical models for video investigation, characterized in that, The system comprises: An image feature editing preprocessing module, which is used for preprocessing the original image in a mosaic manner according to the demand characteristics of the video investigation analysis model, and removing redundant features with low relevance through feature editing to improve the safety and reliability of the data; A customized data generation module connected with the image feature editing preprocessing module, which is used for restoring the obtained image sample data through a task-oriented autoencoder model while retaining key features with high relevance; A multi-analysis model sustainable optimization module connected with the customized data generation module, which is used for constructing an analysis model training and evaluation system based on the generated customized data, and adaptively screening the newly generated data samples according to the model training and evaluation results, so as to improve the effectiveness of the multi-analysis model for video investigation; The demand characteristics of the video investigation analysis model are as follows: The multi-analysis model has different attention elements and training tasks in various video investigation related application scenarios; the attention elements include pedestrians, motor vehicles and non-motor vehicles in the video investigation scenario, and the training tasks include target detection, attribute analysis and feature retrieval in the video investigation scenario; The image feature editing preprocessing module specifically performs the following processing: According to the attention elements of the multi-analysis model, the position range of the foreground region and the background region in the originally obtained image sample is determined; According to the training tasks of the multi-analysis model, the mosaic region position range and the proportion size of the image sample are further selected; The customized data generation module uses an autoencoder framework for data generation, which includes an encoder and a decoder; the encoder is a deep network model with high complexity, which is used for feature encoding of the mosaic image, and the decoder is a lightweight model with low complexity, which is used for feature restoration of the original image; The customized data generation module uses a discriminative model to perform task-related constraint filtering on the restoration results to remove generated data that does not meet the prediction consistency. 2.The system of video-oriented investigation-oriented implementation of multi-element resolution model customization data generation processing according to claim 1, wherein, The image feature editing preprocessing module uses the following method for target detection processing: A large mosaic region proportion is set on the picture sample, and features with low relevance to the detection task including image texture are removed; at the same time, the mosaic region near the foreground edge is reduced, so as to retain features with high relevance to the detection task including target contour. 3.The system of video-oriented investigation-oriented implementation of multi-element resolution model customization data generation processing according to claim 1, wherein, The image feature editing preprocessing module uses the following method for attribute analysis and feature retrieval processing: A large mosaic region proportion is set outside the foreground region of the picture sample, and features with low relevance to the task including image background are removed; at the same time, a small mosaic region proportion is set inside the foreground region, so as to retain features with high relevance to the task including target details.

4. The system for video-oriented investigative multi-resolution model customization data generation process according to claim 1, wherein, The customized data generation module trains original data into a target detection model, an attribute analysis model and a feature retrieval model according to task requirements of the multi-element analysis model, and each original sample and corresponding generated sample needs to meet the following requirements according to the model task characteristics: For the target detection model: the output target category and position information meet consistency; For the attribute analysis model: the output target attribute and confidence information meet consistency; For the feature retrieval model: the output target identity and feature similarity meet consistency.

5. The system for video-oriented investigative multi-resolution model customization data generation process according to claim 4, wherein, After the customized data generation module completes consistency filtering processing according to each model, generates samples, and then scores the data through a quality evaluation model, low-quality data that has obvious differences with real data is filtered out according to the authenticity of the output results.

6. The system for video-oriented investigative multi-resolution model customization data generation process according to claim 4, wherein, The analysis model training and evaluation system specifically comprises: Newly generated sample data is introduced into the training and evaluation data of the multi-element analysis model, and a mutual circulation self-iteration framework composed of a data generation submodule, a model training submodule and a model evaluation submodule.

7. The system for video-oriented investigative implementation of multi-resolution model customization data generation processing according to claim 6, characterized in that, The multi-element analysis model sustainable optimization module filters the newly generated data samples according to the model training and evaluation results in the mutual circulation process of the data generation submodule, the model training submodule and the model evaluation submodule, thereby realizing adaptive updating of the model and the data.

Images