Radar terminal training operation acquisition logging devices, apparatuses, and media

Abstract

The present application relates to the technical field of radar terminal analysis, and particularly relates to a radar terminal training operation collection and recording device, equipment and medium, which can efficiently and timely extract the operation actions of training personnel on equipment in the process of drill training, generate formatted drill training equipment operation log records, and provide data collection means for data used for radar training evaluation and analysis. The formatted operation record data is generated by relying on screen recording video image data and designing software algorithm to extract the operation behavior of training personnel, and the operation record of equipment is quickly analyzed and recognized by combining the equipment operation capture of training personnel with terminal screen capture images, so as to provide the functions of user operation marking, radar terminal image real-time interception, operation behavior image analysis and recognition, and operation log generation for radar equipment data collection.

Description

Technical Field

[0001] This invention relates to the field of radar terminal analysis technology, specifically to a radar terminal training operation acquisition and recording device, equipment, and medium. Background Technology

[0002] In routine training and major exercises for radar troops, it is usually necessary to record the radar operators' operation of the equipment to facilitate post-training effectiveness evaluation. One approach is to pre-design a data collection form, requiring radar operators to fill in detailed records of key operational times and functional objects during training exercises to support post-training evaluation. Another approach is to use screen recording software to record the entire training process on the equipment terminal screen, allowing training analysis and evaluation personnel to review the video playback for assessment. The first method, relying on manual recording, is inefficient, increases the workload for trainees, and can affect their concentration. The second method, using overall screen recording, often only focuses on specific times and events, requiring manual re-analysis and extraction of trainee operational information, comparison with electronic countermeasures equipment interference times to assess impact, and time-consuming and inefficient re-playback, making it unsuitable for analysis and use by exercise evaluation systems. Summary of the Invention

[0003] In view of this, the present invention proposes a radar terminal training operation acquisition and recording device, equipment and medium, which can efficiently extract the operation actions of trainees on equipment during the exercise and training process in real time, generate formatted exercise and training equipment operation log records, and provide data acquisition means for radar training evaluation and analysis.

[0004] To achieve the above objectives, the technical solution of the present invention is as follows: A radar terminal training operation acquisition and recording device includes a user operation marking module, a radar terminal image real-time positioning and capture module, an operation behavior image analysis and extraction module, and an operation log generation module. The user operation marking module is used to locate the time window of user operation behavior and the coordinate information of mouse click operations. The radar terminal image real-time capture module captures the corresponding radar terminal image in real time based on the user operation marking module. The operation behavior image analysis and extraction module identifies and extracts the operation object or text entered by the user on the terminal screen based on the coordinate information corresponding to the user's mouse click operation and the captured terminal image, and identifies the user's specific operation behavior. The operation log generation module records and stores the user operation behavior in a database according to the time sequence.

[0005] The user operation marking module marks mouse operation events performed by trainees through the radar operation terminal during training, records the time of mouse clicks, and collects the coordinate position of the mouse clicks. Specifically, the user operation marking module aims to quickly locate the time window of user operation behavior and the coordinate information of mouse click operations, record the specific time when trainees perform system clicks or text input operations through the radar operation terminal keyboard and mouse during training, use relative time for time recording, synchronize the start time with the radar terminal VGA video recording time, and use the time recording format HH:MM:SS, requiring accuracy to the second, while also recording the position coordinate information of the user's mouse clicks.

[0006] The radar terminal image real-time capture module listens for user operation events based on the user operation marking module. When an event occurs, it captures the corresponding radar terminal image in real time. Specifically, it records the corresponding operation events through the user operation marking module, captures the radar terminal image corresponding to the operation in real time, and establishes the association information between user terminal operation information and radar terminal image.

[0007] The operation behavior image analysis and extraction module includes an image element recognition and analysis module and an operation behavior element determination and recognition module. Specifically, the image element recognition and analysis module uses Faster R-CNN's OCR recognition to realize radar image element recognition, detection, and boundary extraction. The operation behavior element determination and recognition module first combines the coordinate information when the user clicks the mouse with the bounding box information corresponding to the element to determine the specific element corresponding to the mouse operation. Then, it uses the OCR recognition algorithm to identify and record the content details of the element based on the element boundary, and finally completes the extraction of the boundary and content of the element in the image.

[0008] The image element recognition and analysis module uses the terminal images output by the real-time image capture module of the radar terminal to construct training and testing datasets through data annotation, trains the Faster R-CNN model, and after training, uses Faster R-CNN to recognize and detect elements in the radar terminal image and extract the boundaries of buttons and text boxes.

[0009] The specific process for training the Faster R-CNN model is as follows: 001: Add a label to each element in the radar image dataset; 002: Divide the labeled dataset into a training set and a test set; 003: Define the loss function, backpropagate to update parameters, and iterate through training; 004: Evaluate the model using the test set and calculate the average of the average precision for all categories; The process of identifying and detecting elements in radar terminal images using Faster R-CNN includes the following steps: 001: Input a screenshot of the original radar terminal output; 002: Extract feature maps from the input image using a convolutional neural network; 003: The feature map is used as input by the region proposal network to generate a set of candidate bounding boxes. The candidate bounding boxes are filtered and non-maximum suppression is applied to obtain a set of candidate target regions. 004: Based on the feature map and the generated candidate regions, use ROI pooling to convert candidate regions of different sizes into feature maps of fixed size; 005: Use a classification algorithm to further classify the pooled features to determine the target category; use a regression algorithm to accurately adjust the target bounding box; finally, output the detected target category and the precise bounding box position.

[0010] The operation behavior element determination and recognition module specifically includes the following steps: 001: Determine the specific element corresponding to the mouse operation; 002: Element content recognition; 003: Associate the mouse click time, the defined element bounding box, and the content recognized by OCR to generate complete information about the element.

[0011] The specific process by which the radar terminal image real-time capture module captures the corresponding radar terminal image is as follows: 001: Mouse or keyboard event listener, which listens for mouse clicks and keyboard inputs on the terminal. 002: Terminal image capture: Capture the terminal image in real time after a mouse click event or keyboard input event occurs; 003: Store the terminal image and update the terminal image storage path in the user operation log table.

[0012] The present invention also provides an electronic device, the electronic device including a processor and a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the radar terminal training operation acquisition and recording process of the device of the present invention.

[0013] The present invention also provides a computer-readable storage medium storing a computer program for executing the radar terminal training operation acquisition and recording process of the device described in the present invention. Beneficial Effects 1. This invention utilizes a method that combines radar terminal operation capture recording, operation image capture, and image recognition analysis to efficiently and in real-time extract the operational actions of trainees on equipment during exercises and training, generate formatted exercise and training equipment operation log records, provide data collection means for radar training evaluation and analysis, and solve the problem of radar operators having no means to collect equipment operation data during exercises and training.

[0014] 2. This invention does not require the formulation and distribution of equipment operation standard forms before exercises and training, nor does it require modification of equipment operation software to pre-embed code points for operation log recording. It relies on screen-recorded video image data, designs software algorithms to extract the operation behavior of trainees, and generates formatted operation record data.

[0015] 3. This invention adopts a method that combines training personnel's equipment operation capture with terminal screen image capture to quickly analyze and identify equipment operation records, providing radar equipment data acquisition with functions such as user operation marking, real-time radar terminal image capture, operation behavior image analysis and recognition, and operation log generation.

[0016] 4. This invention uses the collected radar terminal operation capture records and images captured by the operation terminal as input, and uses image recognition analysis and processing methods to extract the operation actions of trainees on radar equipment during the exercise and training process in real time, and generates a formatted exercise and training equipment operation log. It does not require immersion modification of the original terminal operation software and has good universality.

[0017] 5. The device of the present invention can also record operations in real time. Compared with the traditional method of recording videos and then playing them back for evaluation, the efficiency is greatly improved, which better meets the needs of radar operator training and evaluation.

[0018] 6. The medium of this invention is based on the method of this invention. It takes the collected radar terminal operation capture records and the images captured by the operation terminal as input, and uses the image recognition analysis and processing method to extract the operation actions of the trainees on the radar equipment in real time during the exercise and training, and generates a formatted exercise and training equipment operation log. It does not require immersion modification of the original terminal operation software and has good universality. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall structure of the device of the present invention.

[0020] Figure 2 This is a schematic diagram of the Faster R-CNN model training process in an embodiment of the present invention.

[0021] Figure 3This is a schematic diagram illustrating the principle of using Faster R-CNN to identify and detect elements in radar terminal images in an embodiment of the present invention.

[0022] Figure 4 This is a schematic diagram of the operation behavior element determination and recognition process in an embodiment of the present invention.

[0023] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0024] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0025] This invention proposes a radar terminal training operation acquisition and recording device, including a user operation marking module, a radar terminal image real-time positioning and capture module, an operation behavior image analysis and extraction module, and an operation log generation module. The overall structure is as follows: Figure 1 As shown. The user operation marking module is used to locate the time window of user operation behavior and the coordinate information of mouse click operation; the radar terminal image real-time capture module captures the corresponding radar terminal image in real time based on the user operation marking module; the operation behavior image analysis and extraction module uses image recognition detection algorithm and OCR technology to identify and extract the operation object or text entered by the user on the terminal screen based on the coordinate information corresponding to the user's mouse click operation and the captured terminal image, and identifies the user's specific operation behavior; the operation log generation module records and stores the user operation behavior in the database according to the time sequence.

[0026] The user operation marking module marks mouse operation events performed by trainees through the radar operating terminal during training, recording the time of mouse clicks and collecting the coordinates of the mouse clicks. Specifically, the goal of the user operation marking module is to quickly locate the time window of user operation behavior and the coordinate information of mouse click operations. It mainly records the specific time when trainees perform system clicks or text input operations through the keyboard and mouse on the radar operating terminal during training. The time recording uses relative time, with the start time (T) synchronized with the VGA video recording time of the radar terminal. The time recording format is HH:MM:SS, accurate to the second. It also records the position coordinates of the user's mouse clicks. Specific recorded information is shown in Table 1.

[0027] Table 1 User Terminal Operation Record Table

[0028] The real-time radar terminal image capture module listens for user operation events based on the user operation marking module. When an event occurs, it captures the corresponding radar terminal image in real time. Specifically, the user operation marking module records the corresponding operation events, captures the corresponding radar terminal image in real time, and establishes a correlation between user terminal operation information and radar terminal image, improving the efficiency of analyzing and extracting trainees' operation behavior. In this embodiment, the specific process is as follows: 001: Mouse or keyboard event listener, which listens for mouse clicks and keyboard inputs on the terminal.

[0029] 002: Terminal image capture. When a mouse click event or keyboard input event occurs, a screenshot of the terminal image is captured in real time.

[0030] 003: Store the terminal image and update the terminal image storage path img_path in the user operation record table in Table 1.

[0031] The operation behavior image analysis and extraction module uses the coordinate information corresponding to the user's mouse click operation and the captured terminal image to identify buttons and text boxes in the terminal image using image recognition and detection algorithms. At the same time, it generates the boundaries of the corresponding elements. Combined with the coordinate information when the user clicks the mouse, it determines the specific button or text box corresponding to the operation. Furthermore, it uses OCR technology to identify and extract the operation object or text entered by the user on the terminal screen, and identifies the user's specific operation behavior.

[0032] In this embodiment, the operation behavior image analysis and extraction module includes an image element recognition and analysis module and an operation behavior element determination and recognition module. Specifically, the image element recognition and analysis module mainly adopts OCR recognition technology based on Faster R-CNN to realize radar image element recognition, detection, and boundary extraction. Specifically, for the terminal image output by the radar terminal image real-time capture module, training and testing datasets are constructed through data annotation, and the Faster R-CNN model is trained. After training, Faster R-CNN is used to recognize and detect radar terminal image elements and extract the boundaries of buttons and text boxes. The operation behavior element determination and recognition module first combines the coordinate information when the user clicks the mouse and the bounding box information corresponding to the element to determine the specific element corresponding to the mouse operation. Then, it uses the OCR recognition algorithm to identify and record the content details of the element based on the element boundary, and finally completes the extraction of the boundary and content of the elements in the image.

[0033] In the image element recognition and analysis module, the training process of the Faster R-CNN model is as follows: Figure 2 As shown, it includes the following steps: 001: Data Labeling Each element in the radar image dataset (such as buttons and text boxes) needs to be labeled to indicate its category and location. Typically, the label includes a category identifier and bounding box information. The bounding box is usually represented by four coordinate values: These coordinates define the position of the element in the image.

[0034] Each labeled data item is represented as a triplet. , Radar terminal image, : Category identifier of the element : The bounding box of an element .

[0035] The results of data annotation will form a training set, which is usually stored in a standard format, such as PASCAL VOC XML or COCO JSON.

[0036] 002: Dataset Classification The labeled dataset is divided into training and test sets, using either cross-validation or random partitioning. The dataset is partitioned proportionally: 80% for training, 10% for validation, and 10% for testing, forming the training, validation, and test sets. This approach allows for performance evaluation of the model and avoids overfitting.

[0037] 003: Model training enables the model to learn how to recognize buttons and text boxes. In Faster R-CNN, model training includes the following process: ① Define the loss function: In object detection tasks, a multi-task loss function is typically used, consisting of a classification loss and a bounding box regression loss. The classification loss determines the category of each candidate bounding box, i.e., a button or a text box, and is calculated using cross-entropy loss. The bounding box regression loss adjusts the position of the candidate bounding boxes to better match the true location of the target, and is typically calculated using a smoothed L1 loss.

[0038] Total loss function It is a linear combination of classification loss and bounding box regression loss. Typically, each candidate bounding box has an associated loss, with the loss function as follows:

[0039] in, It is classification loss. It is the bounding box regression loss. It is a hyperparameter that balances the two.

[0040] ② Backpropagation to update parameters: The backpropagation algorithm is used to calculate the gradient of the loss function with respect to the model parameters in order to update the model's weights and biases. The model parameters include the weights of the convolutional neural network, the weights of the region proposal network, and the weights and biases of the neural network layers used for classification and bounding box regression.

[0041] ③ Iterative training: Training a model typically requires multiple iterations, each called a training epoch. In each training epoch, the model performs forward and backward propagation using a batch of samples from the training dataset, and then updates its parameters. The model's performance can be monitored by evaluating it on a validation set.

[0042] In summary, the model training process includes defining a loss function, using the backpropagation algorithm to update model parameters, and gradually improving model performance through multiple iterations of training.

[0043] 004: Model Evaluation After the training phase, the model is evaluated using the test set, and the mean average precision (mAP) for all classes is calculated.

[0044] The principle of Faster R-CNN for identifying and detecting elements in radar images is as follows: Figure 3 As shown, it includes the following steps: 001: Input image The input image is represented as This represents a screenshot of the original radar terminal output.

[0045] 002: Feature Extraction Input image Feature maps of images are extracted using convolutional neural networks (such as VGG, ResNet, etc.).

[0046]

[0047] 003: Candidate Target Region Generation Use the Region Proposal Network (RPN) to transform the feature map As input, a set of candidate bounding boxes is generated, which include regions that may contain target elements. The candidate bounding boxes are then filtered and subjected to techniques such as nonmaximum suppression (NMS) to reduce redundant bounding boxes, ultimately yielding a set of candidate target regions.

[0048]

[0049] 004: Region of Interest (ROI) Pooling Based on feature map and candidate regions generated by RPN By using ROI pooling, candidate regions of different sizes are transformed into feature maps of a fixed size.

[0050]

[0051] 005: Classification and Regression The pooled features are further classified using a classification algorithm to determine the target category. .

[0052]

[0053] Precisely adjust the target bounding box using regression algorithms .

[0054]

[0055] Final output: Detected target category and precise bounding box location .

[0056] The specific recognition process in the operation behavior element determination and recognition module is as follows: Figure 4 As shown, it includes the following steps: 001: Determine the specific element corresponding to the mouse operation. ① User clicks to obtain coordinates By capturing mouse click events, the coordinates of the click and the mouse click time can be further extracted when processing the mouse click event. .

[0057]

[0058] ② Determine elements: Traverse the bounding boxes of all elements If the user clicks on the coordinates Within the current element's bounding box The polygonal geometry formed If the text inside indicates that the current element box is the element that the user clicked.

[0059]

[0060] in, This indicates the bounding box of the element that was clicked.

[0061] 002: Element Content Recognition (OCR) ① Extracting element images: , It contains sub-images corresponding to specific elements.

[0062] ②OCR recognition: , It contains the text content recognized by OCR.

[0063] 003: Information Integration Mouse click time , defined element bounding box and content recognized by OCR They are linked together to generate complete information about the elements.

[0064] The operation log module records and stores user operations in the database according to a pre-defined format and in chronological order, supporting post-event evaluation and analysis. Each operation includes the following details: unique operation ID, operation time, operation content, and the user performing the operation. The final operation record table is shown in Table 2.

[0065] Table 2 Final Operation Behavior Record Sheet

[0066] This application also provides an electronic device. Figure 5 The structure of an electronic device provided in an embodiment of the present invention is illustrated. For example, the electronic device 50 may include a processor 51, a memory 72, and a transmission device 53. The processor is used to implement the function of a radar terminal training operation acquisition and recording device. The processor and the memory can be connected via a bus or other means, taking a bus connection as an example. The transmission device can be connected to the processor and the memory via wired or wireless means. The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the device in the embodiments of this application. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby realizing the radar terminal training operation acquisition and recording in the above method embodiments. The memory may include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created by the processor, etc. In addition, the memory may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. One or more modules are stored in the memory and, when executed by the processor, perform radar terminal training operations and data collection records in the embodiment.

[0067] In another aspect, this application also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the apparatus described in the above embodiments; or it may be a standalone computer-readable storage medium not assembled into the device. This computer-readable storage medium may be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art. The computer-readable storage medium stores one or more programs, which are used by one or more processors to implement the functions of the radar terminal training operation acquisition and recording device.

[0068] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A radar terminal training operation acquisition and recording device, characterized in that, It includes a user operation marking module, a radar terminal image real-time positioning and capture module, an operation behavior image analysis and extraction module, and an operation log generation module; The user action marker module is used to locate the time window of user actions and the coordinate information of mouse click operations; The radar terminal image real-time capture module captures the corresponding radar terminal image in real time based on the user operation marking module; The operation behavior image analysis and extraction module is based on the coordinate information corresponding to the user's mouse click operation and the captured terminal image to identify and extract the operation object or text entered by the user on the terminal screen and identify the user's specific operation behavior. The operation log generation module records and stores user operation behaviors in the database according to time sequence; The operation behavior image analysis and extraction module includes an image element recognition and analysis module and an operation behavior element determination and recognition module. Specifically, the image element recognition and analysis module uses the Faster R-CNN model to realize radar image element recognition, detection, and boundary extraction. Training and testing datasets are constructed in advance based on the terminal images output by the real-time radar terminal image capture module through data annotation. The data annotation includes labeling the category of each element in each image of the radar image dataset and its bounding box information. The Faster R-CNN model is trained, and the trained Faster R-CNN model is used to recognize and detect image elements in the radar terminal images captured in real-time according to the user operation labeling module, outputting the category of the element and its corresponding bounding box information. The elements include buttons and text boxes. The operation behavior element determination and recognition module first combines the coordinate information when the user clicks the mouse with the bounding box information of the element, traverses all elements, determines the specific element corresponding to the mouse operation, further uses the OCR recognition algorithm, identifies and records the content details of the element based on the element boundary, and finally integrates the elements, associating the mouse click time, the determined element bounding box and the element text content to generate complete information of the element; The operation log generation module records and stores user operation behaviors in the database according to a pre-agreed format and in chronological order, supporting post-event evaluation and analysis. The specific content corresponding to each operation includes: operation unique ID, operation time, operation content, and operation user; the operation content includes button name.

2. The apparatus as claimed in claim 1, characterized in that, The user operation marking module marks mouse operation events performed by trainees through the radar operating terminal during training, records the time of mouse clicks, and simultaneously collects the coordinates of the mouse clicks. Specifically, the user operation marking module aims to quickly locate the time window of user operation behavior and the coordinate information of mouse click operations. It records and marks the specific time when trainees perform system clicks or text input operations through the keyboard and mouse of the radar operating terminal during training. The time recording uses relative time, and the start time is synchronized with the VGA video recording time of the radar terminal. The time recording format is HH:MM:SS, and it is required to be accurate to the second. It also records the position coordinate information of the user's mouse click.

3. The apparatus as described in claim 2, characterized in that, The real-time radar terminal image capture module listens for user operation events based on the user operation marking module. When an event occurs, it captures the corresponding radar terminal image in real time. Specifically, it records the corresponding operation events through the user operation marking module, captures the radar terminal image corresponding to the operation in real time, and establishes a correlation between user terminal operation information and radar terminal image.

4. The apparatus as claimed in claim 1, characterized in that, The specific process for training the Faster R-CNN model is as follows: 001: Add a label to each element in the radar image dataset; 002: Divide the labeled dataset into a training set and a test set; 003: Define the loss function, backpropagate to update parameters, and iterate through training; 004: Evaluate the model using the test set and calculate the average of the average precision for all categories; The process of identifying and detecting elements in radar terminal images using Faster R-CNN includes the following steps: 001: Input a screenshot of the original radar terminal output; 002: Extract feature maps from the input image using a convolutional neural network; 003: The feature map is used as input by the region proposal network to generate a set of candidate bounding boxes. The candidate bounding boxes are filtered and non-maximum suppression is applied to obtain a set of candidate target regions. 004: Based on the feature map and the generated candidate regions, use ROI pooling to convert candidate regions of different sizes into feature maps of fixed size; 005: Use a classification algorithm to further classify the pooled features to determine the target category; use a regression algorithm to accurately adjust the target bounding box; finally, output the detected target category and the precise bounding box position.

5. The apparatus as described in claim 3 or 4, characterized in that, The specific process by which the radar terminal image real-time capture module captures the corresponding radar terminal image is as follows: 001: Mouse or keyboard event listener, which listens for mouse clicks and keyboard inputs on the terminal. 002: Terminal image capture. When a mouse click event or keyboard input event occurs, a screenshot of the terminal image is captured in real time. 003: Store the terminal image and update the terminal image storage path in the user operation log table.

6. An electronic device, characterized in that, The electronic device includes a processor and a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the radar terminal training operation acquisition and recording process of any of the devices described in claims 1 to 5.

7. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which is used to execute the radar terminal training operation acquisition and recording process of any of the devices described in claims 1 to 5.

Images