A video generation method for marine insurance operation situation awareness
By constructing an original dataset and using simulation tools to generate virtual videos, the problems of difficult video data collection and identification errors in aviation safety operations were solved, and the generation and identification accuracy of diverse video samples were improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHENGZHOU UNIV
- Filing Date
- 2026-03-23
- Publication Date
- 2026-06-23
AI Technical Summary
In modern aircraft carrier-based aircraft support operations, video data collection is difficult, and there are multiple overlapping operational elements, which may lead to the risk of mission non-identification and identification errors. In addition, the special nature of the scene makes identification difficult.
The original dataset is constructed, virtual videos are generated using simulation tools, core targets are detected using Faster R-CNN, video frames are described using bounding boxes and image captioning models, video frame attributes are decomposed and diverse video samples are rendered, and the final video samples are selected.
It addresses the risks of unidentified tasks and incorrect identification, generates diverse video samples suitable for situational awareness in aviation safety operations, and improves identification accuracy.
Smart Images
Figure CN122269100A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of video sample generation technology, and in particular to a video generation method for situational awareness in aviation safety operations. Background Technology
[0002] With the rapid development of the information age, video data is experiencing explosive growth, leading to an increasing demand for downstream applications based on video, such as video object detection and video prediction. Thanks to the development of artificial intelligence, AI technologies, represented by deep learning, have achieved tremendous success in various applications, a success inseparable from the support of a large number of training samples. However, despite the abundance of video data on the internet, collecting diverse video samples remains extremely difficult, especially in modern aircraft carrier-based aircraft support operations where multiple operational elements overlap, posing risks of unidentified or incorrectly identified tasks, as well as unique scenario-specific challenges. Summary of the Invention
[0003] The purpose of this invention is to provide a video generation method for situational awareness in aviation safety operations. By constructing an original dataset and target bounding boxes, and using simulation tools to generate virtual videos in special scenarios, this method solves the risks of unidentified tasks and identification errors in existing methods, as well as the problem of special scenarios.
[0004] To achieve the above objectives, this invention provides a video generation method for situational awareness in aviation safety operations, comprising the following steps: S1. Construct the original video dataset; S2. Process the original video dataset and construct a fused video set; S3. Extract keywords from the fused video set and describe the objects in the image. S4. Perform attribute decomposition and processing on the results in S3 to obtain a diverse video sample library. S5. Evaluate the video samples in the diverse video sample library, and filter the video samples according to the evaluation structure to obtain the final video samples.
[0005] Preferably, the process of S1 is as follows: S11. Review relevant information in the target field and collect relevant actual operation videos; S12. Mark the core objectives and scene attributes of the actual operation video; S13. Generate virtual videos using simulation tools based on data from the target domain; S14. Remove abnormal videos from the actual operation videos and virtual videos to obtain the original video dataset.
[0006] Preferably, the process in S2 is as follows: S21. Use Faster R-CNN to detect the core targets in the videos in the original dataset; S22. Set the target bounding box according to the detection results, use the target bounding box to surround the core target, make the core target located in the center of the target bounding box, and cut the target bounding box; S23. Scale the cut target box regions to a uniform size of 224×224 to obtain a standardized video frame sequence and a fused video set.
[0007] Preferably, the process of S3 is as follows: S31. Use an image captioning model to describe the video frame sequence in the fused video set to obtain the text description of the video frame; S32. Use a pre-trained large language model to fuse the textual descriptions of video frames into a comprehensive video description.
[0008] Preferably, the process of S4 is as follows: S41. Use a pre-trained residual network as a video encoder to encode the video frame sequence into a frame-by-frame n-dimensional feature vector. ; S42. Decompose the feature vector into frame-by-frame multi-attributes, calculate the variance of each attribute, construct an attribute decomposition network containing 5 sub-networks, and perform corresponding processing. S43. Calculate the loss function and fuse each target rendering frame and background frame to obtain a fused image. All fused images form a diverse video sample library.
[0009] Preferably, the process of constructing the attribute decomposition network and performing corresponding processing in S42 is as follows: S421. Decompose the feature vector in S41 into frame-by-frame multi-dimensional attributes, which include geometric parameters. Texture parameters Attitude parameters Illumination parameters and deformation parameters ; S422. Use the least squares objective function to perform consistency processing on geometric parameters and texture parameters, as follows: Perform consistency processing on geometric parameters: ; Perform consistency processing on texture parameters: ; in and These represent the consistency vectors of the geometric parameters and the consistency vectors of the texture parameters, respectively. S423. Smoothing is applied to the attitude parameters, lighting parameters, and deformation parameters. The process is as follows: ; ; ; in , and Let these represent the smoothness vectors for attitude, lighting, and deformation, respectively. Represents a one-dimensional convolution kernel. This represents the discrete convolution operation.
[0010] Preferably, the process of S43 is as follows: S431. The processed attributes from S42 are combined into a multi-attribute model, as follows: ; in This indicates a vector concatenation operation; S432. Use an encoder to map the multi-attribute model to a t-dimensional embedding space and reconstruct the multi-attribute model. S433. Establish the loss function, the process is as follows: ; in Represents the total loss function. Indicates the losses incurred during reconstruction. Indicates smoothing loss. Indicates consistency loss. , and All are balance factors; S434. Randomly sample the multi-attribute model in the t-dimensional embedding space, and use the vector after processing the texture and pose parameters to generate the target video frame of the t-th frame through the renderer. Then, randomly crop the scene background and merge the video frames and scene background to obtain a fused image. The process is as follows: ; in Let be the target mask for frame t. Represents the background image of the scene; S435, All fused images form a diverse video sample library.
[0011] Preferably, the process of S5 is as follows; S51. Use the video-dataset-scripts model to evaluate the overall aesthetic score of samples in a diverse video sample library. ; S52. Calculate the task relevance score using the UMT video language model. ; S53. Filter sample videos from the video sample library based on aesthetic scores and task relevance scores. and The samples were discarded from the pool to form preliminary video samples; S54. Manually composite the preliminary video samples to obtain the final video samples.
[0012] Therefore, the present invention provides a video generation method for situational awareness of aviation safety operations using the above-mentioned structure. By constructing an original dataset and target bounding boxes, and using simulation tools to generate virtual videos in special scenarios, the method solves the risks of unidentified tasks and identification errors, as well as the problems of special scenarios, that exist in existing methods.
[0013] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description
[0014] Figure 1 This is an overall flowchart of a video generation method for situational awareness in aviation support operations according to the present invention; Figure 2 This is a sample image from the final video sample of the video generation method for situational awareness in aviation support operations according to the present invention. Detailed Implementation
[0015] Example To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0016] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.
[0017] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.
[0018] In the description of this invention, it should be noted that the terms "upper," "lower," "inner," "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the product of this invention is usually placed when in use. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.
[0019] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," and "connect" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.
[0020] The following detailed description of some embodiments of the present invention is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0021] like Figures 1-2 As shown, this invention provides a video generation method for situational awareness in aircraft carrier support operations, addressing the relevant requirements of modern aircraft carrier-based aircraft support operations. The method includes the following steps: S1. Construct the original video dataset; S11. Review data on carrier-based aircraft deck operations and collect relevant actual operation videos (such as carrier-based aircraft takeoff, landing, and scheduling scenarios). S12. Mark the core targets (such as carrier-based aircraft, deck personnel, and command equipment) and scene attributes (such as light intensity and sea state level) in the actual operation video. S13. Based on data from the target field and with the guidance and assistance of experts in the relevant field, generate virtual videos of carrier-based aircraft deck operations using simulation technology; S14. Remove abnormal videos from the actual operation videos and virtual videos. Abnormal videos include videos that are too short (less than 3 seconds) or too long (more than 10 minutes), as well as videos with blurry images, to obtain the original video dataset.
[0022] S2. Process the original video dataset and construct a fused video set; S21. Use Faster R-CNN to detect the core target, carrier-based aircraft, in the videos of the original dataset; S22. Set the target bounding box according to the detection results, use the target bounding box to surround the core target, make the core target located in the center of the target bounding box, and cut the target bounding box; S23. Scale the cut target box regions to a uniform size of 224×224 to obtain a standardized video frame sequence and a fused video set.
[0023] S3. Extract keywords from the fused video set and describe the objects in the image. S31. Use an image captioning model to describe the video frame sequence in the fused video set to obtain the text description of the video frame; S32. Use a pre-trained large language model to fuse the textual descriptions of video frames into a comprehensive video description.
[0024] S4. Perform attribute decomposition and processing on the results in S3 to obtain a diverse video sample library. S41. Use a pre-trained residual network as a video encoder to encode the video frame sequence into a frame-by-frame n-dimensional feature vector. ; S42. Decompose the feature vector into frame-by-frame multi-attributes, calculate the variance of each attribute, construct an attribute decomposition network containing 5 sub-networks, and perform corresponding processing. S421. Decompose the feature vector in S41 into frame-by-frame multi-dimensional attributes, which include geometric parameters. Texture parameters Attitude parameters Illumination parameters and deformation parameters ; S422. Use the least squares objective function to perform consistency processing on geometric parameters and texture parameters, as follows: Perform consistency processing on geometric parameters: ; Perform consistency processing on texture parameters: ; in and These represent the consistency vectors of the geometric parameters and the consistency vectors of the texture parameters, respectively. S423. Smoothing is applied to the attitude parameters, lighting parameters, and deformation parameters. The process is as follows: ; ; ; in , and Let these represent the smoothness vectors for attitude, lighting, and deformation, respectively. Represents a one-dimensional convolution kernel. This represents the discrete convolution operation.
[0025] S43. Calculate the loss function and fuse each target rendering frame and background frame to obtain a fused image. All fused images form a diverse video sample library.
[0026] S431. The processed attributes from S42 are combined into a multi-attribute model, as follows: ; in This indicates a vector concatenation operation; S432. Use an encoder to map the multi-attribute model to a t-dimensional embedding space and reconstruct the multi-attribute model. S433. Establish the loss function, the process is as follows: ; ; ; ; in Represents the total loss function. Indicates the losses incurred during reconstruction. Indicates smoothing loss. Indicates consistency loss. , and All are balance factors. The multi-attribute model represents the first A vector, Reconstructed multi-attribute model A vector, and These represent the attribute dimensions of geometry and texture, respectively; S434. Randomly sample the multi-attribute model in the t-dimensional embedding space, and use the vector after processing the texture and pose parameters to generate the target video frame of the t-th frame through the renderer. Then, randomly crop the scene background and merge the video frames and scene background to obtain a fused image. The process is as follows: ; in Let be the target mask for frame t. Represents the background image of the scene; S435, All fused images form a diverse video sample library.
[0027] S5. Evaluate the video samples in the diverse video sample library, and filter the video samples according to the evaluation structure to obtain the final video samples.
[0028] S51. Use the video-dataset-scripts model to evaluate the overall aesthetic score of samples in a diverse video sample library. ; S52. Calculate the task relevance score using the UMT video language model. ; S53. Filter sample videos from the video sample library based on aesthetic scores and task relevance scores. and The initial video samples were obtained by removing other samples. S54. Manually composite the preliminary video samples to obtain the final video samples.
[0029] Therefore, the present invention provides a video generation method for situational awareness of aviation safety operations using the above-mentioned structure. By constructing an original dataset and target bounding boxes, and using simulation tools to generate virtual videos in special scenarios, the method solves the risks of unidentified tasks and identification errors, as well as the problems of special scenarios, that exist in existing methods.
[0030] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.
Claims
1. A video generation method for situational awareness in aviation safety operations, characterized in that, Includes the following steps: S1. Construct the original video dataset; S2. Process the original video dataset and construct a fused video set; S3. Extract keywords from the fused video set and describe the objects in the image. S4. Perform attribute decomposition and processing on the results in S3 to obtain a diverse video sample library. S5. Evaluate the video samples in the diverse video sample library, and filter the video samples according to the evaluation structure to obtain the final video samples.
2. The video generation method for situational awareness in air traffic control operations according to claim 1, characterized in that, The process of S1 is as follows: S11. Review relevant information in the target field and collect relevant actual operation videos; S12. Mark the core objectives and scene attributes of the actual operation video; S13. Generate virtual videos using simulation tools based on data from the target domain; S14. Remove abnormal videos from the actual operation videos and virtual videos to obtain the original video dataset.
3. The video generation method for situational awareness in air traffic control operations according to claim 2, characterized in that, The process in S2 is as follows: S21. Use Faster R-CNN to detect the core targets in the videos in the original dataset; S22. Set the target bounding box according to the detection results, use the target bounding box to surround the core target, make the core target located in the center of the target bounding box, and cut the target bounding box; S23. Scale the cut target box regions to a uniform size of 224×224 to obtain a standardized video frame sequence and a fused video set.
4. The video generation method for situational awareness in air traffic control operations according to claim 3, characterized in that, The process of S3 is as follows: S31. Use an image captioning model to describe the video frame sequence in the fused video set to obtain the text description of the video frame; S32. Use a pre-trained large language model to fuse the textual descriptions of video frames into a comprehensive video description.
5. A video generation method for situational awareness in air traffic control operations according to claim 4, characterized in that, The process of S4 is as follows: S41. Use a pre-trained residual network as a video encoder to encode the video frame sequence into a frame-by-frame n-dimensional feature vector. ; S42. Decompose the feature vector into frame-by-frame multi-attributes, calculate the variance of each attribute, construct an attribute decomposition network containing 5 sub-networks, and perform corresponding processing. S43. Calculate the loss function and fuse each target rendering frame and background frame to obtain a fused image. All fused images form a diverse video sample library.
6. The video generation method for situational awareness in air traffic control operations according to claim 5, characterized in that, The process of constructing the attribute decomposition network and performing corresponding processing in S42 is as follows: S421. Decompose the feature vector in S41 into frame-by-frame multi-dimensional attributes, which include geometric parameters. Texture parameters Attitude parameters Illumination parameters and deformation parameters ; S422. Use the least squares objective function to perform consistency processing on geometric parameters and texture parameters, as follows: Perform consistency processing on geometric parameters: ； Perform consistency processing on texture parameters: ； in and These represent the consistency vectors of the geometric parameters and the consistency vectors of the texture parameters, respectively. S423. Smoothing is applied to the attitude parameters, lighting parameters, and deformation parameters, as follows: ； ； ； in , and Let these represent the smoothness vectors for attitude, lighting, and deformation, respectively. Represents a one-dimensional convolution kernel. This represents the discrete convolution operation.
7. A video generation method for situational awareness in air traffic control operations according to claim 6, characterized in that, The process of S43 is as follows: S431. The processed attributes from S42 are combined into a multi-attribute model, as follows: ； in This indicates a vector concatenation operation; S432. Use an encoder to map the multi-attribute model to a t-dimensional embedding space and reconstruct the multi-attribute model. S433. Establish the loss function, the process is as follows: ； in Represents the total loss function. Indicates the losses incurred during reconstruction. Indicates smoothing loss. Indicates consistency loss. , and All are balance factors; S434. Randomly sample the multi-attribute model in the t-dimensional embedding space, and generate the target video frame t-th frame from the vector after processing the texture and pose parameters through the renderer. Then, randomly crop the scene background and merge the video frames and scene background to obtain a fused image. The process is as follows: ； in Let be the target mask for frame t. Represents the background image of the scene; S435, All fused images form a diverse video sample library.
8. A video generation method for situational awareness in aviation support operations according to claim 7, characterized in that, The process of S5 is as follows; S51. Use the video-dataset-scripts model to evaluate the overall aesthetic score of samples in a diverse video sample library. ; S52. Calculate the task relevance score using the UMT video language model. ; S53. Filter sample videos from the video sample library based on aesthetic scores and task relevance scores. and The samples were discarded from the pool to form preliminary video samples; S54. Manually composite the preliminary video samples to obtain the final video samples.