A method for target functional group identification and model training

Abstract

The present invention discloses a target functional group identification and a model training method thereof, and belongs to the technical field of situational awareness. The target functional group identification model training method first inputs the normalized data, text data and image data related to the query target into a multimodal large model, and extracts the multimodal feature coding vector of the query target; secondly, the feature coding is passed through a multimodal coding alignment module based on a cross-attention mechanism to achieve vector alignment; then, the aligned feature coding vector is spliced ​​with the query coding vector and input into a multilayer perceptron, and after a SoftMax operation, the probability that the query target belongs to each given functional group is obtained; the model uses a cross entropy loss function to update the parameters of the functional group identification model. Under strong confrontation conditions, the target functional group identification model trained by the present invention can more effectively identify the target functional group.

Description

Technical Field

[0001] The present invention belongs to the technical field of situation recognition, and more specifically, relates to a target function group identification and a model training method thereof. Background Art

[0002] In a strong confrontation scenario involving group targets, enemy targets are characterized by large numbers, various types, and chaotic movement trajectories. Focusing on the analysis of individual enemies will often lead to confusion, which is not conducive to grasping the rules of enemy activities and forming an easy-to-understand situation. However, the forces of both the enemy and us are deployed and gathered according to certain rules, so by grouping enemy targets, we can reduce information entropy and achieve more effective support for situation cognition.

[0003] Based on the abstract degree of clustering, the clustering of battlefield entity targets includes four levels from bottom to top: spatial group, functional group, interaction group, and enemy-neutral group. Functional group refers to a set of targets with the same function, in which all target entities have the same or similar functional characteristics. The specific definition of functional group can be divided into narrow and broad. In a narrow sense, a functional group is obtained by combining targets of the same type in a spatial group. In a broad sense, it is not limited to the same spatial group or the same type. A functional group refers to a set of targets that perform the same combat mission or achieve the same function. From a theoretical perspective, the division of generalized functional groups is more reasonable. However, due to the following problems, the division method of generalized target functional groups is difficult to apply in practice. (1) The identification of generalized functional groups requires the learning of knowledge such as tactics, combat regulations, and open source intelligence, and often requires the manual participation of experts. Existing automated methods are difficult to extract and match effective functional grouping rules; (2) The identification of generalized functional groups performs well on historical track training data, but its model has weak generalization and poor migration ability. In a strong confrontation environment, targets change frequently, and the existing generalized functional group division method is difficult to ensure real-time functional grouping. (3) Under strong confrontation conditions, target information is incomplete, and a series of information such as the distance and direction of the group target is difficult to obtain effectively. The traditional model only relies on current information and is difficult to effectively determine the functional group. Summary of the invention

[0004] In view of the above defects or improvement needs of the prior art, the present invention provides a target function group identification and model training method thereof, which aims to solve the technical problems that the target function group identification is difficult and requires human participation in a strong confrontation environment.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a target function group recognition model training method, the method comprising:

[0006] Inputting normalized data, text data and image data related to the query object into the multimodal large model to obtain a multimodal feature encoding vector related to the query object;

[0007] The feature encoding vectors are aligned after passing through a multimodal encoding alignment module based on a cross-attention mechanism;

[0008] The aligned feature encoding vector is concatenated with the query encoding vector and input into a multi-layer perceptron, and the function group of the query target is output after a SoftMax operation;

[0009] A cross entropy loss function is used to update the parameters of the functional group identification model.

[0010] Preferably, the cross entropy loss function is:

[0011]

[0012] Where M is the number of functional groups; N is the number of samples in this batch; y ic is the indicator function. If the true functional group of sample i is equal to c, then y ic Take 1, otherwise y ic Take 0; p ic is the probability that sample i belongs to functional group c; ||θ|| 2 Represents the 2-norm of the model parameters, which is used to prevent the model from overfitting. is a hyperparameter.

[0013] Preferably, the recognition model is optimized online according to the recognition result of the target functional group recognition model, specifically:

[0014] If the recognition result of the recognition model is accurate, the count of the feedback is increased by one; otherwise, the actual function group of the target is given, and the counts of negative feedback and positive feedback are both increased by one;

[0015] The samples, predicted values ​​and true values ​​of the recognition model are stored in a user feedback buffer pool in a first-in-first-out mode;

[0016] At every preset time period, the data in the user feedback buffer pool is used to fine-tune the parameters of the present value model; the initial parameters of the present value model are the same as the parameters of the identification model;

[0017] If the negative feedback count ratio increases to a negative feedback threshold, the present value model is used to replace the identification model.

[0018] Preferably, the data in the user feedback buffer pool is used to fine-tune the parameters of the present value model, specifically: based on the existing present value model parameters, the data in the user feedback buffer pool continues to train the present value model so that the recognition result of the updated present value model is the true value in the feedback buffer pool.

[0019] Preferably, the normalized data includes space group division results, track data and combat rules; the text data includes tactics and text intelligence; the image data includes tactical rule maps, satellite images, close reconnaissance images and image intelligence.

[0020] Preferably, a rule data set used for target functional group recognition model recognition is constructed in the form of Prompt.

[0021] Preferably, the compression of the target functional group recognition model is achieved while ensuring recognition function and performance by adopting model distillation and model pruning.

[0022] In a second aspect, the present invention provides a method for identifying a target functional group, the method comprising:

[0023] Obtain normalized data, text data, and image data related to the query target;

[0024] Using a pre-trained target function group recognition model to recognize the normalized data, text data and image data, and outputting the recognized target function group;

[0025] Wherein, the target functional group recognition model is trained according to any method in the first aspect.

[0026] In a third aspect, the present application provides an electronic device, comprising: a memory for storing programs; a processor for executing the programs stored in the memory, when the program stored in the memory is executed, the processor is used to execute the method described in the first aspect; or, execute the method described in the second aspect.

[0027] In a fourth aspect, the present application provides a storage medium storing a computer program, which, when executed on a processor, enables the processor to execute the method described in the first aspect; or, execute the method described in the second aspect.

[0028] In general, the above technical solution conceived by the present invention has the following beneficial effects compared with the prior art:

[0029] (1) Under strong confrontation conditions, information is incomplete. The method of the present invention introduces a multimodal large model to more effectively identify the identity and other attribute information of the group target, which is conducive to the determination of functional group attributes;

[0030] (2) The method of the present invention effectively utilizes multimodal data including normalized data, text data and image data, improves the automation level of target functional group recognition, and can improve the recognition accuracy;

[0031] (3) The method of the present invention includes an online optimization mechanism for recognition results, and uses a reward function to drive the self-evaluation evolution of the model, thereby enhancing the generalization ability of the model and improving the real-time response capability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a target function group identification model diagram using multimodal data provided in an embodiment of the present invention;

[0033] Figure 2 Schematic diagram of a model distillation provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0035] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0036] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0037] Next, the technical solution provided in the embodiment of the present application is introduced. The embodiment of the present application is a method for training and online optimization of a target function group recognition model under strong confrontation, including: an automated rule extraction stage and a prompt learning online optimization stage, wherein:

[0038] The automated rule extraction phase includes:

[0039] (1) The recognition model is constructed using end-to-end functional group rules.

[0040] (1.1) The model input is defined as the normalized data, text data and image data corresponding to the target group, and the output is the specific functional group division result (such as defense group, attack group, reconnaissance group, etc.).

[0041] (1.2) Identification model network structure design Figure 1 shown.

[0042] Specifically, the network structure of the recognition model is divided into three parts: multimodal feature extraction, multimodal decoding alignment, and multi-layer perceptron (MLP) functional group matching. The model input is the query object (including time, space and other attribute information) and multimodal information related to the query object, and the output is the functional group recognition result of the query object.

[0043] Taking into account the powerful capabilities of current multimodal pre-trained models, the multimodal feature extraction part of the present invention directly uses the pre-trained multimodal large model as the training basis (such as VisCPM and VisualGLM-6B, etc.), and uses its open source model parameters to initialize the parameters of the feature extraction part, and extracts standardized data such as space group division results, track data, combat rules; text data such as tactics and strategies, public text intelligence, and features such as tactical rule maps, satellite images, close reconnaissance images, and public image intelligence.

[0044] However, since the general multimodal large model does not consider time series data such as tracks, it is difficult to align time series data with modal data such as text, pictures, and inference rules. At the same time, considering that the original structure of some large models is difficult to adjust, the present invention also designs a multimodal encoding alignment module based on the cross-attention mechanism to address the problem of limited alignment ability of the general multimodal large model in this application. This module takes the encoding vector output by the large model as input, outputs the vector encoding after the alignment operation, and realizes the alignment operation of the vectorized data of the same object.

[0045] After obtaining the aligned high-quality encoding, the splicing operation ⊕ is used to first splice the feature-aligned multimodal data encoding to obtain e m , and then encode the query vector e q Also splice it with, get e mq Finally, the obtained concatenated vector is input into the multi-layer perceptron (MLP), and after the SoftMax operation, the predicted value of each functional group is output.

[0046] For the task of functional group identification, the present invention proposes that the learning of model parameters adopts the cross entropy loss function.

[0047]

[0048] Where M is the number of functional groups; N is the number of samples in this batch; y ic is the indicator function. If the true functional group of sample i is equal to c, then y ic Take 1, otherwise y ic Take 0; p ic is the probability that sample i belongs to functional group c; ||θ|| 2 Represents the 2-norm of the model parameters, which is used to prevent the model from overfitting. is a hyperparameter.

[0049] (2) Using prompts, a high-quality rule dataset is constructed for target functional group recognition. The data contains information in the form of formatted text, tracks, images, and other modalities.

[0050] The Prompt data construction method of track, image and other information is consistent with the general model construction method. In particular, the formatted text is a known functional group identification rule, and its construction adopts the "Step by Step" reasoning Prompt construction method.

[0051] (3) Use the constructed multimodal functional group recognition rule dataset to fine-tune the open source multimodal Chinese model;

[0052] Specifically, the structure of the general multimodal large model constructed in (1) is used to train the model using the Prompt dataset constructed in (2). The large model in (1) is initialized using its open source parameters, and the rest of the model is randomly initialized.

[0053] (4) Model distillation is used to trim the model and achieve model compression while ensuring the function recognition function and performance.

[0054] Direct application of pre-trained large model parameters helps the model have stronger generalization and reasoning capabilities. However, for the target functional group recognition task, such a large-scale model will limit its application scenarios and hinder its application. For this specific application, the large model contains a lot of structural and parameter redundancy, and the model needs to be effectively pruned.

[0055] Specifically, the "student-teacher" model is used for model distillation. The original model is defined as a teacher model, and the one to be simplified is the student model. First, copy the teacher model parameters to the student model; then try to gradually simplify the model structure and model parameters. In recent years, adversarial training has been successful in generative networks. Inspired by this, many adversarial knowledge distillation methods have been proposed to enable the teacher and student networks to better understand the real data distribution. Specifically, this study uses a discriminator (D) to distinguish the discriminant vectors of the teacher model (T) and the student model (S). The teacher model is used as the real data and remains unchanged during training. The student model is used as the generator (G), such as Figure 2 shown.

[0056] (5) Adopt a “human in the loop” model to store feedback data from user application models.

[0057] Specifically, a user feedback buffer pool is set up to cyclically record the samples, predicted values ​​and true values ​​of the recognition model in a first-in-first-out (FIFO) pipeline structure. For the function group judgment given by the model, if it is consistent with the actual situation, the feedback data count is increased by one; otherwise, the negative feedback and feedback number are both increased by one.

[0058] (6) Drawing on the model parameter adjustment strategy of mainstream reinforcement learning methods, the two models of "application" and "present value" are maintained at the same time (where the output of the application model is the model output, and the parameters of the application model are used to initialize the present value model).

[0059] (6.1) At certain time intervals, the data from the user feedback buffer pool in (5) is used to fine-tune the parameters of the present value model;

[0060] Specifically, to fine-tune the present value model, it is necessary to train the present value model with data in the user feedback buffer pool based on the existing present value model parameters, so that the trained present value model can better fit the true value in the feedback buffer pool.

[0061] (6.2) Record the proportion of negative feedback from users. When the proportion of negative feedback increases to a preset threshold, use the present value model parameters to replace the application model parameters.

[0062] This completes a training and online optimization method for the target functional group recognition model under strong adversarial conditions.

[0063] The embodiment of the present application also implements a method for identifying a target function group under strong confrontation, comprising the following steps:

[0064] Obtain normalized data, text data, and image data related to the query target;

[0065] Using a pre-trained target function group recognition model to recognize the normalized data, text data and image data, and outputting the recognized target function group;

[0066] It should be understood that the target function group identification method is based on the target function group identification model in the above embodiment, so the training method of the target function group identification model is not described in detail here.

[0067] Based on the method in the above embodiment, the embodiment of the present application further provides an electronic device. The device may include: a memory for storing a program and a processor for executing the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the above embodiment.

[0068] Based on the method in the above embodiment, an embodiment of the present application further provides a storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0069] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0070] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0071] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a storage medium or transmitted through the storage medium. The computer instructions may be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0072] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0073] Those skilled in the art will readily understand that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A target function group recognition model training method, characterized in that: The method comprises: Inputting normalized data, text data and image data related to the query object into the multimodal large model to obtain a multimodal feature encoding vector related to the query object; The feature encoding vectors are aligned after passing through a multimodal encoding alignment module based on a cross-attention mechanism; The aligned feature encoding vector is concatenated with the query encoding vector and input into a multi-layer perceptron, and the function group of the query target is output after a SoftMax operation; Using a cross entropy loss function to update the parameters of the functional group identification model; The standardized data includes space group division results, track data and combat rules; the text data includes tactics and text intelligence; the image data includes tactical rule maps, satellite images, close reconnaissance images and image intelligence; The recognition model is optimized online according to the recognition result of the target functional group recognition model, specifically: If the recognition result of the recognition model is accurate, the count of the feedback is increased by one; otherwise, the actual function group of the target is given, and the counts of negative feedback and positive feedback are both increased by one; The samples, predicted values ​​and true values ​​of the recognition model are stored in a user feedback buffer pool in a first-in-first-out mode; At every preset time period, the data in the user feedback buffer pool is used to fine-tune the parameters of the present value model; the initial parameters of the present value model are the same as the parameters of the identification model; If the negative feedback count ratio increases to a negative feedback threshold, the present value model is used to replace the identification model.

2. The method according to claim 1, characterized in that The cross entropy loss function is: Where M is the number of functional groups; N is the number of batch samples; y ic is the indicator function. If the true functional group of sample i is equal to c, then y ic Take 1, otherwise y ic Take 0; p ic is the probability that sample i belongs to functional group c; ||θ||2 represents the 2-norm of the model parameters, which is used to prevent the model from overfitting. is a hyperparameter.

3. The method according to claim 1, characterized in that The parameters of the present value model are fine-tuned using the data in the user feedback buffer pool. Specifically, based on the existing present value model parameters, the present value model is continuously trained using the data in the user feedback buffer pool so that the recognition result of the updated present value model is the true value in the feedback buffer pool.

4. The method according to claim 1, characterized in that: The rule data set used for target functional group recognition model recognition is constructed in the form of Prompt.

5. The method according to claim 1, characterized in that: The compression of the target functional group recognition model is achieved while ensuring the recognition function and performance by adopting the method of model distillation and model pruning.

6. A method for identifying a target functional group, characterized in that: The method comprises: Obtain normalized data, text data, and image data related to the query target; Using a pre-trained target function group recognition model to recognize the normalized data, text data and image data, and outputting the recognized target function group; Wherein, the target functional group recognition model is trained according to the method according to any one of claims 1-5.

7. An electronic device, characterized in that: include: Memory, used to store programs; A processor, configured to execute the program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the method according to any one of claims 1 to 5; Alternatively, perform the method of claim 6.

8. A storage medium storing a computer program, characterized in that: When the computer program runs on a processor, the processor is enabled to execute the method according to any one of claims 1 to 5; or, to execute the method according to claim 6.

Images