A semantic retrieval method and retrieval system for ramp videos based on deep learning

Through the deep learning-based apron video semantic retrieval method, the YOLOv5s model and time convolution network are used to solve the problem of inefficient apron video retrieval, and efficient and accurate video retrieval is achieved, saving manpower and time costs.

CN114064971BActive Publication Date: 2024-11-15CIVIL AVIATION UNIV OF CHINA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111383673.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-22
Publication Date
2024-11-15
Estimated Expiration
2041-11-22

AI Technical Summary

Technical Problem

In the field of civil aviation, apron video retrieval has not been effectively applied, resulting in apron management requiring a lot of manpower and time to screen video clips, which is inefficient and increases costs.

Method used

Using deep learning-based apron video semantic search method, the apron object detection data set is constructed, and the target detection is performed using the YOLOv5s model. The time convolution network and attention mechanism are combined to generate feature matrix for video semantic search to achieve the Automatic retrieval of apron video.

Benefits of technology

It significantly improves the efficiency of apron event retrieval, shortens time, saves costs, and ensures high accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114064971B_ABST
    Figure CN114064971B_ABST
Patent Text Reader

Abstract

The present invention relates to a deep learning-based apron video semantic retrieval method and retrieval system, which belongs to the technical field of video information processing, and comprises S1, constructing an apron target detection data set; S2, training the apron target detection data set to generate a final target detection model; S3, preprocessing the apron video; S4, detecting visual targets in the preprocessed apron video, and generating the position and label information of the visual targets; S5, analyzing the target detection results, and screening out feature sequences of visual targets that meet the apron operation rules; S6, extracting features from the feature sequences, and using an attention mechanism to fuse features to generate a feature matrix; S7, inputting the feature matrix into a neural network to train a video semantic retrieval model; S8, obtaining the apron video to be detected, inputting a query event, and generating a video candidate segment; S9, inputting the video candidate segment to obtain a video segment that meets the semantics of the query event.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of video information processing, and in particular relates to a deep learning-based apron video semantic retrieval method and retrieval system. Background Art

[0002] With the development of the Internet, video has gradually become the main information carrier after text and graphics. In the field of public security, video surveillance systems have become an important part of maintaining public order and strengthening social management. However, video recordings have the characteristics of large storage data volume and long storage time. Finding the required video clips through recordings means searching a large number of manual indexes and performing lengthy linear screening, which consumes a lot of manpower, material resources and time, is extremely inefficient, and increases unnecessary costs.

[0003] With the advancement of technology, video retrieval technology has emerged. It is now booming and widely used in digital television, distance education, telemedicine, security and other fields, becoming a new star in the big data era. However, the video retrieval system in the civil aviation field has not yet started, and the retrieval research of apron videos is still blank. Apron management is complex, requiring 7×24 hours of all-weather monitoring of operators and vehicles on the apron to record their illegal operations, identify vehicle driving paths, the number of trailers, the placement of cones, flatbed trucks, etc., violations such as personnel crossing the aircraft positions, statistics on historical violations, and correcting and handling violations. Many managements consume a lot of manpower. Therefore, the task of developing a search system with video search as the core is imminent. Summary of the invention

[0004] In order to solve the technical problems existing in the known technology, the present invention provides a deep learning-based apron video semantic retrieval method and retrieval system; by using computer data processing technology, the original surveillance video is imported into the retrieval system, and the query event is selected to retrieve the corresponding video clip.

[0005] The first object of the present invention is to provide a method for semantic retrieval of apron videos based on deep learning, which at least comprises:

[0006] Step 1, construct a ramp target detection dataset;

[0007] Step 2: Use the YOLOv5s model to train the apron target detection dataset to generate the final target detection model;

[0008] Step 3, pre-processing the apron video according to the apron operation rules;

[0009] Step 4, using the target detection model obtained in S2 to detect the visual target in the preprocessed apron video, and generate the position and label information of the visual target;

[0010] Step 5: Analyze the target detection results and select the feature sequence of the visual targets that meet the apron operation rules;

[0011] Step 6: Extract features from the feature sequence through the temporal convolutional network, and use the attention mechanism to fuse features to generate a feature matrix;

[0012] Step 7, input the feature matrix into the neural network to train the video semantic retrieval model;

[0013] Step 8, obtain the apron video to be detected, input the query event, and generate the video candidate segments through preprocessing;

[0014] Step 9: Input the candidate video segments obtained in S8 into the apron video semantic retrieval system based on deep learning to obtain video segments that meet the semantics of the query event.

[0015] Further: In step 1, the data source of the apron target detection dataset is the surveillance video of the 209th parking space of Guiyang Longdongbao International Airport. OpenCV is used to split the video into images frame by frame, and the target detection annotation tool labelImg is used to annotate all images to construct a target detection dataset in YOLO format. The image annotation types cover 10 categories, namely people, jetties, garbage trucks, airplanes, refueling trucks, platform trucks, water trucks, luggage trucks, catering trucks, and tractors.

[0016] Further: In step 2, the YOLOv5s model is used as a pre-trained model to retrain the apron target detection dataset constructed in S1. The training parameters are set as follows: epochs = 80, batch_size = 16, and the final target detection model is generated.

[0017] Further: In step 3, the apron video data source is the surveillance video of aircraft stand 209 of Guiyang Longdongbao Airport. According to the prior knowledge of the apron operation process, the video clips of each apron operation process are cropped, and the average duration and average number of video frames of each apron operation process are counted.

[0018] Further: In step 4, OpenCV is used to split the apron operation clips cropped in S3 into pictures frame by frame, and input them into the target detection model obtained in S2 to generate labels and coordinate information of visual objects.

[0019] Further: In step 5, the labels and coordinate information of the visual objects that meet the apron operation rules are filtered out from the labels and coordinate information of all visual objects obtained in step 4. Three types of features are set, namely, the relative distance of the visual object, the relative position of the visual object, and the speed of the visual object. The relative pixel distance of the visual objects that meet the apron operation rules in each frame of the picture is counted, and a relative pixel distance sequence is generated. The relative position of the visual objects that meet the apron operation rules in each frame of the picture is counted. The relative position refers to the angle formed by the pixel coordinates of the remaining visual objects and the positive direction of the x-axis when the coordinate system is constructed with the pixel coordinates of the aircraft as the origin. The speed of the visual object is calculated using the inter-frame difference method to generate a visual object speed feature sequence.

[0020] Further: In step 6, a temporal convolutional network is used to extract features from the feature sequence obtained in step 5, and the extracted feature vectors are concatenated into a feature matrix. The attention mechanism is applied to the feature matrix, and different weights are assigned to the feature matrix to achieve feature fusion.

[0021] Further: In step 7, the feature matrix obtained in step 6 is input into a semantic retrieval network composed of a fully connected layer to train a semantic retrieval model.

[0022] Further: In step 8, any video to be detected in the apron scene is obtained, a query event is input, and a sliding window method is used to split the long video into video candidate segments with the average number of frames of the work flow corresponding to the input event as the sliding window size.

[0023] Further: in step 9, the video candidate segment is input into the target detection model obtained in step 2, the position and label information of the visual target that meets the apron operation rules in the video candidate segment is obtained according to step 5, and a feature matrix is ​​generated according to step 6. The feature matrix is ​​input into the video semantic retrieval model generated in step 7 to obtain the video segment that meets the semantics of the query event.

[0024] The second object of the present invention is to provide a deep learning-based apron video semantic retrieval system, comprising at least:

[0025] Component module, builds the apron target detection dataset;

[0026] The training module uses the YOLOv5s model to train the apron target detection dataset and generate the final target detection model;

[0027] The pre-processing module pre-processes the apron video according to the apron operation rules;

[0028] A detection module, which uses the target detection model to detect visual targets in the preprocessed apron video and generates the position and label information of the visual targets;

[0029] The analysis and screening module analyzes the target detection results and screens out the feature sequences of visual targets that meet the apron operation rules;

[0030] The feature generation module extracts features from feature sequences through a temporal convolutional network and uses an attention mechanism to fuse features and generate a feature matrix.

[0031] The retrieval module inputs the feature matrix into the neural network to train the video semantic retrieval model;

[0032] The video candidate segment generation module obtains the apron video to be detected, inputs the query event, and generates the video candidate segments through preprocessing;

[0033] The result output module inputs the candidate video segments into the apron video semantic retrieval system based on deep learning to obtain video segments that meet the semantics of the query event.

[0034] The third objective of the present invention is to provide an information data processing terminal for implementing the above-mentioned apron video semantic retrieval method based on deep learning.

[0035] The fourth object of the present invention is to provide a computer-readable storage medium, comprising instructions, which, when executed on a computer, enables the computer to execute the above-mentioned deep learning-based apron video semantic retrieval method.

[0036] The advantages and positive effects of the present invention are:

[0037] The present invention can effectively improve the retrieval efficiency of specific events in the apron according to the characteristics of the apron operation process.

[0038] The present invention designs a target detection network to identify apron operation vehicles and count their feature sequences, and uses a temporal convolutional network to highly abstract the feature sequences and attention mechanism into a feature matrix for the apron operation process, which facilitates the retrieval of specific events in the apron. Compared with manual retrieval, it greatly shortens the time, improves efficiency, saves costs, and ensures a high accuracy rate under the premise of having a large number of data sets to support training. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flowchart of a preferred embodiment of the present invention;

[0040] Figure 2 This is the first effect diagram of the apron target detection model in the preferred embodiment of the present invention;

[0041] Figure 3 This is a second effect diagram of the apron target detection model in the preferred embodiment of the present invention;

[0042] Figure 4This is the third effect diagram of the apron target detection model in the preferred embodiment of the present invention;

[0043] Figure 5 This is the fourth effect diagram of the apron target detection model in the preferred embodiment of the present invention;

[0044] Figure 6 A bar chart showing the detection accuracy of the apron target detection model in a preferred embodiment of the present invention;

[0045] Figure 7 It is the relative position feature sequence in the preferred embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to further understand the content, features and effects of the present invention, the following embodiments are given as examples and described in detail with reference to the accompanying drawings:

[0047] The semantic retrieval method and retrieval system of apron video based on deep learning is essentially a feature sequence matching problem based on target detection. The visual targets of the apron are identified through the target detection model, the label position information of the visual targets is obtained, and the feature sequence is generated. The feature sequence is extracted through the temporal convolutional network to generate a feature matrix, and the feature matrix is ​​weighted redistributed using the attention mechanism to achieve feature fusion. The above steps highly abstract the workflow of the apron into a feature matrix, and then the feature matrix is ​​input into the fully connected network to train the classifier. For the video to be detected, the corresponding operation process is judged according to the input query event, the sliding window size is determined according to the prior knowledge, and the video to be detected is decomposed into video candidate segments using the sliding window method. The video candidate segments are input into the semantic retrieval system of apron video based on deep learning. After target detection, feature sequence acquisition, feature extraction, feature fusion, and classifier classification, the video segments that meet the semantics of the query event can be selected to complete the semantic retrieval of the apron video.

[0048] See also Figure 1-Figure 7 , a semantic retrieval method for apron videos based on deep learning, including two implementation stages, namely the establishment of a semantic model of apron videos based on deep learning and the preprocessing of the videos to be detected. Specifically, it includes the following steps:

[0049] Step 1: Construct an apron target detection dataset. The data source of the apron target detection dataset is the surveillance video of the 209th parking space of Guiyang Longdongbao International Airport. Use OpenCV to split the video into images frame by frame, use the target detection annotation tool labelImg to annotate all images, and construct a target detection dataset in YOLO format. The image annotation types cover 10 categories, namely people, jetties, garbage trucks, airplanes, refueling trucks, platform trucks, water trucks, luggage trucks, catering trucks, and tractors.

[0050] Step 2: Use the YOLOv5s model to train the apron target detection dataset to generate the final target detection model; specifically: use the YOLOv5s model as a pre-training model to re-train the apron target detection dataset constructed in S1, and set the training parameters as follows: epochs = 80, batch_size = 16, to generate the final target detection model;

[0051] Step 3: pre-process the apron video according to the apron operation rules; specifically: the apron video data source is the surveillance video of the 209 aircraft stand of Guiyang Longdongbao Airport. According to the prior knowledge of the apron operation process, the video clips of each apron operation process are cut out, and the average duration and average number of video frames of each apron operation process are counted;

[0052] Step 4: Use the target detection model obtained in S2 to detect the visual targets in the preprocessed apron video and generate the position and label information of the visual targets; specifically, use OpenCV to split the apron operation clips cropped in S3 into pictures frame by frame, input them into the target detection model obtained in S2, and generate the labels and coordinate information of the visual objects;

[0053] Step 5, analyze the target detection results, and filter out the feature sequence of visual targets that meet the apron operation rules; specifically: filter out the labels and coordinate information of visual objects that meet the apron operation rules from the labels and coordinate information of all visual objects obtained in step 4. Set three types of features, namely the relative distance of visual objects, the relative position of visual objects, and the speed of visual objects. Count the relative pixel distances of visual objects that meet the apron operation rules in each frame of the image, and generate a relative pixel distance sequence. Count the relative positions of visual objects that meet the apron operation rules in each frame of the image. The relative position refers to the angle formed by the pixel coordinates of the remaining visual objects and the positive direction of the x-axis when the coordinate system is constructed with the pixel coordinates of the aircraft as the origin. Use the inter-frame difference method to calculate the speed of the visual object and generate a visual object speed feature sequence;

[0054] Step 6: extract features from the feature sequence through the temporal convolutional network, and use the attention mechanism to fuse features to generate a feature matrix; specifically: use the temporal convolutional network to extract features from the feature sequence obtained in step 5, and splice the extracted feature vectors into a feature matrix, apply the attention mechanism to the feature matrix, and assign different weights to the feature matrix to achieve feature fusion;

[0055] Step 7, inputting the feature matrix into a neural network to train a video semantic retrieval model; specifically, inputting the feature matrix obtained in step 6 into a semantic retrieval network composed of a fully connected layer to train the semantic retrieval model;

[0056] Step 8, obtain the apron video to be detected, input the query event, and generate the video candidate segments through preprocessing; specifically: obtain any video to be detected in the apron scene, input the query event, use the sliding window method, and use the average number of frames of the operation process corresponding to the input event as the sliding window size to split the long video into video candidate segments;

[0057] Step 9, input the video candidate segments obtained in S8 into the apron video semantic retrieval system based on deep learning to obtain video segments that meet the semantics of the query event; specifically: input the video candidate segments into the target detection model obtained in step 2, obtain the position and label information of the visual targets that meet the apron operation rules in the video candidate segments according to step 5, generate a feature matrix according to step 6, input the feature matrix into the video semantic retrieval model generated in step 7, and obtain the video segments that meet the semantics of the query event.

[0058] A deep learning-based apron video semantic retrieval system, comprising at least:

[0059] The component module is used to construct an apron target detection dataset. The data source of the apron target detection dataset is the surveillance video of the 209th parking space of Guiyang Longdongbao International Airport. OpenCV is used to split the video into images frame by frame, and the target detection annotation tool labelImg is used to annotate all images to construct a target detection dataset in YOLO format. The image annotation types cover 10 categories, namely people, bridges, garbage trucks, airplanes, refueling trucks, platform trucks, water trucks, luggage trucks, catering trucks, and tractors.

[0060] The training module uses the YOLOv5s model to train the apron target detection dataset and generate the final target detection model. The YOLOv5s model is used as a pre-trained model to train the apron target detection dataset constructed in the component module again. The training parameters are set as follows: epochs = 80, batch_size = 16, and the final target detection model is generated.

[0061] The preprocessing module preprocesses the apron video according to the apron operation rules; the apron video data source is the surveillance video of the 209 aircraft stand of Guiyang Longdongbao Airport. Based on the prior knowledge of the apron operation process, the video clips of each apron operation process are cut out, and the average duration and average number of video frames of each apron operation process are counted;

[0062] The detection module detects the visual targets in the preprocessed apron video using the target detection model to generate the position and label information of the visual targets; uses OpenCV to split the apron operation clips cut from S3 into pictures frame by frame, and inputs them into the target detection model obtained by the training module to generate the labels and coordinate information of the visual objects;

[0063] The analysis and screening module analyzes the target detection results and screens out the feature sequences of visual targets that meet the apron operation rules; the labels and coordinate information of visual objects that meet the apron operation rules are screened out from the labels and coordinate information of all visual objects obtained by the detection module. Three types of features are set, namely the relative distance of visual objects, the relative position of visual objects, and the speed of visual objects. The relative pixel distances of visual objects that meet the apron operation rules in each frame of the image are counted, and a relative pixel distance sequence is generated. The relative positions of visual objects that meet the apron operation rules in each frame of the image are counted. The relative position refers to the angle formed by the pixel coordinates of the remaining visual objects and the positive direction of the x-axis when the coordinate system is constructed with the pixel coordinates of the aircraft as the origin. The speed of the visual object is calculated using the inter-frame difference method to generate a visual object speed feature sequence;

[0064] The feature generation module extracts features from the feature sequence through the temporal convolutional network, and uses the attention mechanism to fuse features to generate a feature matrix. The feature sequence obtained by the analysis and screening module is extracted using the temporal convolutional network, and the extracted feature vectors are spliced ​​into a feature matrix. The attention mechanism is applied to the feature matrix, and different weights are assigned to the feature matrix to achieve feature fusion.

[0065] The retrieval module inputs the feature matrix into the neural network to train the video semantic retrieval model; the feature matrix obtained by the feature generation module is input into the semantic retrieval network composed of a fully connected layer to train the semantic retrieval model;

[0066] The video candidate segment generation module obtains the apron video to be detected, inputs the query event, and generates the video candidate segments through preprocessing; obtains any video to be detected in the apron scene, inputs the query event, and uses the sliding window method to split the long video into video candidate segments with the average number of frames of the operation process corresponding to the input event as the sliding window size;

[0067] The result output module inputs the candidate video segments into the apron video semantic retrieval system based on deep learning to obtain the video segments that meet the semantics of the query event; the candidate video segments are input into the target detection model obtained by the training module, the position and label information of the visual targets that meet the apron operation rules in the candidate video segments are obtained according to the analysis and screening module, and a feature matrix is ​​generated according to the feature generation module, and the feature matrix is ​​input into the video semantic retrieval model generated by the retrieval module to obtain the video segments that meet the semantics of the query event.

[0068] An information data processing terminal for implementing the above-mentioned apron video semantic retrieval method based on deep learning.

[0069] A computer-readable storage medium includes instructions, which, when executed on a computer, enable the computer to execute the above-mentioned apron video semantic retrieval method based on deep learning.

[0070] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When the use is implemented in whole or in part in the form of a computer program product, the computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL) or wireless (e.g., infrared, wireless, microwave, etc.) mode) to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk SolidState Disk (SSD)), etc.

[0071] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are within the scope of the technical solution of the present invention.

Claims

1. A semantic retrieval method for ramp videos based on deep learning, characterized in that: At least: S1, build apron target detection dataset; S2. Use the YOLOv5s model to train the apron target detection dataset and generate the final target detection model; S3. Pre-process the apron video according to the apron operation rules; S4, using the target detection model to detect the visual target in the preprocessed apron video, and generate the position and label information of the visual target; S5. Analyze the target detection results and select the feature sequence of the visual target that meets the apron operation rules; S6. Extract features from feature sequences through a temporal convolutional network, and use an attention mechanism to fuse features to generate a feature matrix. S7, inputting the feature matrix into the neural network to train the video semantic retrieval model; S8, obtaining the apron video to be detected, inputting the query event, and generating the video candidate segments through preprocessing; S9. Input the candidate video segments into a deep learning-based apron video semantic retrieval system to obtain video segments that meet the semantics of the query event.

2. The deep learning-based apron video semantic retrieval method according to claim 1 is characterized in that: Specifically, S1 includes: the data source of the apron target detection dataset is the aircraft monitoring video, OpenCV is used to split the video into images, the target detection annotation tool labelImg is used to annotate all images, and a target detection dataset in YOLO format is constructed; The types of image annotations include people, air bridges, garbage trucks, airplanes, refueling trucks, platform trucks, water trucks, luggage trucks, catering trucks, and tractors.

3. The deep learning-based apron video semantic retrieval method according to claim 2 is characterized in that: Specifically, S2 uses the YOLOv5s model as a pre-training model to train the apron target detection dataset constructed in S1, and the training parameters are set as follows: epochs=80, batch_size=16, to generate the final target detection model.

4. The deep learning-based apron video semantic retrieval method according to claim 3 is characterized in that: Specifically, S3 includes: the apron video data source is the aircraft stand monitoring video, and based on the prior knowledge of the apron operation process, the video clips of each apron operation process are cut out, and the average duration and average number of video frames of each apron operation process are counted.

5. The deep learning-based apron video semantic retrieval method according to claim 4 is characterized in that: Specifically, S4 includes: using OpenCV to split the apron operation clips obtained by cutting in S3 into pictures frame by frame, and inputting them into the target detection model obtained in S2 to generate labels and coordinate information of visual objects.

6. The method for semantic retrieval of ramp videos based on deep learning according to claim 5 is characterized in that: The S5 is specifically as follows: filter out the labels and coordinate information of visual objects that meet the apron operation rules from the labels and coordinate information of all visual objects obtained in S4; set three types of features, namely, relative distance of visual objects, relative position of visual objects, and speed of visual objects; count the relative pixel distance of visual objects that meet the apron operation rules in each frame of the picture, and generate a relative pixel distance sequence; count the relative position of visual objects that meet the apron operation rules in each frame of the picture, and the relative position refers to the angle formed by the pixel coordinates of the remaining visual objects and the positive direction of the x-axis in the coordinate system constructed with the pixel coordinates of the aircraft as the origin; calculate the speed of the visual object using the inter-frame difference method to generate a visual object speed feature sequence.

7. The method for semantic retrieval of ramp videos based on deep learning according to claim 6 is characterized in that: S6 specifically comprises: using a temporal convolutional network to extract features from the feature sequence obtained in S5, and splicing the extracted feature vectors into a feature matrix, applying an attention mechanism to the feature matrix, and performing different weight allocations on the feature matrix to achieve feature fusion; S7 specifically includes: inputting the feature matrix obtained in S6 into a semantic retrieval network composed of a fully connected layer to train a semantic retrieval model; S8 specifically includes: obtaining any video to be detected in the apron scene, inputting a query event, using a sliding window method, taking the average number of frames of the operation flow corresponding to the input event as the sliding window size, and splitting the long video into video candidate segments; The S9 specifically includes: inputting the candidate video segment into the target detection model obtained by S2, obtaining the position and label information of the visual target that meets the apron operation rules in the candidate video segment according to S5, generating a feature matrix according to S6, inputting the feature matrix into the video semantic retrieval model generated by S7, and obtaining the video segment that meets the semantics of the query event.

8. A deep learning-based apron video semantic retrieval system, characterized by: At least: Component module, builds the apron target detection dataset; The training module uses the YOLOv5s model to train the apron target detection dataset and generate the final target detection model; The pre-processing module pre-processes the apron video according to the apron operation rules; A detection module, which uses the target detection model to detect visual targets in the preprocessed apron video and generates the position and label information of the visual targets; The analysis and screening module analyzes the target detection results and screens out the feature sequences of visual targets that meet the apron operation rules; The feature generation module extracts features from feature sequences through a temporal convolutional network and uses an attention mechanism to fuse features and generate a feature matrix. The retrieval module inputs the feature matrix into the neural network to train the video semantic retrieval model; The video candidate segment generation module obtains the apron video to be detected, inputs the query event, and generates the video candidate segments through preprocessing; The result output module inputs the candidate video segments into the apron video semantic retrieval system based on deep learning to obtain video segments that meet the semantics of the query event.

9. An information data processing terminal for implementing the deep learning-based apron video semantic retrieval method described in any one of claims 1 to 7.

10. A computer-readable storage medium comprising instructions, which, when executed on a computer, enables the computer to execute the deep learning-based apron video semantic retrieval method as described in any one of claims 1 to 7.