Target detection method and device, equipment, storage medium and program product
By performing frame-level decomposition and multi-target recognition on aircraft towing videos, combined with target detection models and feature sample data amplification, the problem of inaccurate detection caused by environmental changes during aircraft towing was solved, and accurate detection of aircraft towing status and targets was achieved.
Patent Information
- Application Number
- CN202411049799.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-08
AI Technical Summary
Existing image recognition technologies cannot adapt to rapid environmental changes during aircraft towing, resulting in inaccurate target detection.
By acquiring aircraft towing videos, performing frame-level decomposition, and using a target detection model for multi-target recognition and state assessment, combined with feature sample data amplification and gradient descent model training, accurate detection of the first preset target is achieved.
This technology enables accurate assessment of aircraft towing status and precise detection of the first preset target amidst environmental changes, thereby improving the accuracy and efficiency of detection.
Smart Images

Figure CN118918512A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of automated detection technology, and in particular to a target detection method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art
[0002] With the development of image recognition technology, some image recognition technologies have been applied in the field of aircraft maintenance, but these technologies usually rely on static images for detection, and most of them are performed in fixed scenes, which cannot adapt to the ever-changing environment during aircraft towing. Therefore, there is an urgent need for a target detection method that can adapt to the ever-changing environment. Summary of the invention
[0003] Based on this, it is necessary to provide a target detection method, device, computer equipment, computer-readable storage medium and computer program product that can adapt to the ever-changing environment in order to solve the above-mentioned technical problems.
[0004] In a first aspect, the present application provides a target detection method, comprising:
[0005] Acquire an aircraft towing video from a camera, and perform frame-level decomposition on the aircraft towing video to obtain towing video frames;
[0006] Performing multi-target recognition on the towed video frame according to the target detection model to obtain a multi-target recognition result;
[0007] Perform aircraft drag status evaluation according to the multi-target recognition result to obtain an evaluation result;
[0008] If the evaluation result indicates that the aircraft is in a towing state, target detection is performed on a first preset target according to the target detection model to obtain a first target detection result; wherein the first preset target is used to determine whether the aircraft is in a normal towing state.
[0009] In one embodiment, performing multi-target recognition on the dragging video frame according to the target detection model to obtain a multi-target recognition result includes:
[0010] When the target detection model recognizes that there is a second preset target in the towing video frame, the number of times each second preset target is recognized is recorded within a preset time; wherein the second preset target is used to determine whether the aircraft is in a towing state;
[0011] Statistics are performed based on the number of recognitions to obtain the multi-target recognition result.
[0012] In one embodiment, the multi-target recognition result includes: a first recognition number of a first candidate target and a second recognition number of a second candidate target; the target detection model includes: a first weight and a second weight; the aircraft drag state evaluation is performed according to the multi-target recognition result to obtain an evaluation result, including:
[0013] Multiplying the first recognition times according to the first weight to obtain a first evaluation value;
[0014] The second identification times are multiplied according to the second weight to obtain a second evaluation value; wherein the first weight and the second weight are respectively used to reflect the influence of the first candidate target and the second candidate target on judging the drag state of the aircraft;
[0015] Accumulating the first evaluation value and the second evaluation value to obtain a total evaluation value;
[0016] A dragging judgment is performed according to the total evaluation value and a preset dragging threshold to obtain the evaluation result.
[0017] In one of the embodiments, after performing target detection on the first preset target according to the target detection model to obtain a first target detection result, the method further includes:
[0018] If the first target detection result is that the first preset target is detected, performing target detection on the dragging video frame at the current moment according to the target detection model to obtain a target detection result;
[0019] If within the preset safety time, the target detection results are both that the first preset target and the second preset target are not detected, then an exit drag state result is generated.
[0020] In one embodiment, the method further comprises:
[0021] Obtaining a shooting distance between the towing observation area of the aircraft and a preset shooting point;
[0022] Performing focal length calculation according to the shooting distance and a preset focal length model to obtain a target focal length;
[0023] A focal length lens is selected according to the target focal length to obtain the camera.
[0024] In one of the embodiments, before performing multi-target recognition on the drag video frame according to the target detection model to obtain a multi-target recognition result, the method further includes:
[0025] Acquire feature sample data, and perform data amplification on the feature sample data to obtain training sample data;
[0026] The original detection model is trained according to the training sample data, and during the training process, parameters of the original detection model are adjusted according to the gradient descent model to minimize the loss function corresponding to the original detection model, thereby obtaining a target detection model to be evaluated;
[0027] The target detection model to be evaluated is evaluated according to the validation set data of the training sample data. If the evaluation result meets the preset evaluation condition, the target detection model to be evaluated is used as the target detection model.
[0028] In a second aspect, the present application also provides a target detection device, comprising:
[0029] An acquisition module is used to acquire the aircraft towing video of the camera, and perform video frame decoding on the aircraft towing video to obtain a towing video frame;
[0030] A multi-target recognition module, used to perform multi-target recognition on the towed video frame according to a target detection model to obtain a multi-target recognition result;
[0031] An evaluation module, used to evaluate the aircraft drag state according to the multi-target recognition result to obtain an evaluation result;
[0032] The target detection module is used to perform target detection on a first preset target according to the target detection model to obtain a first target detection result if the evaluation result indicates that the aircraft is in a towing state; wherein the first preset target is used to determine whether the aircraft is in a normal towing state.
[0033] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0034] Acquire an aircraft towing video from a camera, and perform frame-level decomposition on the aircraft towing video to obtain towing video frames;
[0035] Performing multi-target recognition on the towed video frame according to the target detection model to obtain a multi-target recognition result;
[0036] Perform aircraft drag status evaluation according to the multi-target recognition result to obtain an evaluation result;
[0037] If the evaluation result indicates that the aircraft is in a towing state, target detection is performed on a first preset target according to the target detection model to obtain a first target detection result; wherein the first preset target is used to determine whether the aircraft is in a normal towing state.
[0038] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:
[0039] Acquire an aircraft towing video from a camera, and perform frame-level decomposition on the aircraft towing video to obtain towing video frames;
[0040] Performing multi-target recognition on the towed video frame according to the target detection model to obtain a multi-target recognition result;
[0041] Perform aircraft drag status evaluation according to the multi-target recognition result to obtain an evaluation result;
[0042] If the evaluation result indicates that the aircraft is in a towing state, target detection is performed on a first preset target according to the target detection model to obtain a first target detection result; wherein the first preset target is used to determine whether the aircraft is in a normal towing state.
[0043] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:
[0044] Acquire an aircraft towing video from a camera, and perform frame-level decomposition on the aircraft towing video to obtain towing video frames;
[0045] Performing multi-target recognition on the towed video frame according to the target detection model to obtain a multi-target recognition result;
[0046] Perform aircraft drag status evaluation according to the multi-target recognition result to obtain an evaluation result;
[0047] If the evaluation result indicates that the aircraft is in a towing state, target detection is performed on a first preset target according to the target detection model to obtain a first target detection result; wherein the first preset target is used to determine whether the aircraft is in a normal towing state.
[0048] The above-mentioned target detection method, device, computer equipment, computer-readable storage medium and computer program product obtain the aircraft towing video of the camera, and perform frame-level decomposition on the aircraft towing video to obtain the towing video frame; perform multi-target recognition on the towing video frame according to the target detection model to obtain the multi-target recognition result; perform aircraft towing state evaluation according to the multi-target recognition result to obtain the evaluation result; if the evaluation result indicates that the aircraft is in the towing state, perform target detection on the first preset target according to the target detection model to obtain the first target detection result; wherein the first preset target is used to determine whether the aircraft is in a normal towing state. Therefore, after performing multi-target recognition on the towing video frame obtained in the process of real-time environmental changes through the target detection model, the aircraft towing state evaluation is performed according to the multi-target recognition result to accurately determine the towing state of the current aircraft, and then further determine whether the aircraft is in a normal towing state according to the target detection result of the first preset target, so that the first preset target can be accurately detected while the environment is constantly changing. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0050] FIG1 is a diagram of an application environment of a target detection method according to an embodiment;
[0051] FIG2 is a schematic diagram of a process flow of a target detection method in one embodiment;
[0052] FIG3 is a schematic diagram of a process flow of a target detection method in one embodiment;
[0053] FIG4 is a schematic diagram of a labeling strategy of a target detection method in one embodiment;
[0054] FIG5 is an accuracy-recall curve of a target detection model to be evaluated in a target detection method in one embodiment on a validation data set;
[0055] FIG6 is a schematic diagram of a flow chart of a target detection method in another embodiment;
[0056] FIG7 is a block diagram of a target detection device in one embodiment;
[0057] FIG. 8 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0058] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0059] The target detection method provided in the embodiment of the present application can be applied in the application environment shown in FIG. 1 . The terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The server 104 obtains the aircraft towing video of the camera and decomposes the aircraft towing video at the frame level to obtain the towing video frame; performs multi-target recognition on the towing video frame according to the target detection model to obtain the multi-target recognition result; performs aircraft towing state evaluation according to the multi-target recognition result to obtain the evaluation result; if the evaluation result indicates that the aircraft is in the towing state, the first preset target is detected according to the target detection model to obtain the first target detection result; wherein the first preset target is used to determine whether the aircraft is in a normal towing state. The terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices, and the Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. The portable wearable device may be a smart watch, a smart bracelet, a head-mounted device, etc. The head-mounted device may be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.
[0060] In an exemplary embodiment, as shown in FIG2 , a target detection method is provided, which is described by taking the method applied to the server 104 in FIG1 as an example, and includes the following steps 202 to 208. Among them:
[0061] Step 202 : Acquire the aircraft towing video of the camera, and perform frame-level decomposition on the aircraft towing video to obtain towing video frames.
[0062] The aircraft towing video is a video image of the front wheel well area of the aircraft acquired in the aircraft towing scene.
[0063] In some embodiments, intermittent shooting may be used to continuously shoot and acquire the aircraft towing video from the camera, thereby saving computer resources required for uploading the video while ensuring the accuracy of target recognition.
[0064] It should be noted that in the aircraft towing scenario, due to the radio restrictions and network security requirements in the airport environment, especially the reality that edge devices do not have 4G network access capabilities, in some embodiments, the edge device is customized through perl scripts, APN (access point name), user name, password, dial-up number, baud rate, etc., and uses AT commands to drive the USB modem to connect to the internal network. When the trailer carrying the camera starts, the device is powered on, and then intermittent shooting is adopted to continuously shoot and upload to the background, thereby ensuring a stable connection between the edge device and the platform, avoiding the use of high-power radio equipment to interfere with aviation communications and navigation systems, while also ensuring the network security of the edge device and the control background.
[0065] In some embodiments, before obtaining the aircraft towing video of the camera, the target detection method also includes: obtaining the shooting distance of the aircraft's towing observation area from a preset shooting point; performing focal length calculation based on the shooting distance and a preset focal length model to obtain the target focal length; and selecting a focal length lens based on the target focal length to obtain a camera.
[0066] Among them, the aircraft's towing observation area is the aircraft's front wheel bay area, and the preset shooting point is the camera placement point set on the trailer.
[0067] In some embodiments, the shooting distance between the aircraft's towing observation area and the preset shooting point can be obtained through computer vision technology, and other technologies can also be used to obtain the precise shooting distance, but are not limited thereto.
[0068] In some embodiments, the focal length is calculated according to the shooting distance and the preset focal length model to obtain the target focal length, and the focal length lens is selected according to the target focal length to obtain the camera. For example, in the aircraft towing scene, the shooting distance between the front wheel turning pin and the camera is 6 meters, and the height of the front wheel turning pin is 1 meter. After the focal length calculation, the equivalent focal length of 25mm lens is finally selected, and the shooting picture at this time is properly exposed and the target features to be detected are clear.
[0069] The camera includes a target focal length lens, and the preset focal length model is as follows:
[0070]
[0071] Where f is the focal length, h is the imaging width of the object on the CMOS target surface (the imaging height of a 1 / 1.9-inch CMOS is 4.05 mm), D is the distance from the object to the camera lens, and H is the actual height of the object.
[0072] Step 204 , performing multi-target recognition on the dragging video frame according to the target detection model to obtain a multi-target recognition result.
[0073] Among them, the target detection model is a pre-trained model for detecting preset targets in the aircraft towing scene, and the multi-target recognition is to detect and recognize multiple preset targets in the same video frame and record them.
[0074] In some embodiments, when the target detection model recognizes that there is a second preset target in the towing video frame, the number of times each second preset target is recognized is recorded within a preset time; wherein the second preset target is used to determine whether the aircraft is in a towing state; statistics are performed based on the number of recognitions to obtain a multi-target recognition result.
[0075] Among them, the second preset target is the front wheel turning total addition mechanism and the towing rod connector in the aircraft. The front wheel turning total addition mechanism and the towing rod connector are key components that need to be used in the aircraft towing process. Therefore, when the key components are detected in the towing video frame, it means that there is a possibility of towing the aircraft.
[0076] In some embodiments, when the target detection model recognizes the presence of a second preset target in the drag video frame, a timer is started. During the preset time period, each time the second preset target is detected, it is recorded in the multi-target recognition result, and the number of recognitions of each second preset target is recorded separately.
[0077] It should be further explained that, based on the actual situation of aircraft towing, before the aircraft towing operation, the trailer carrying the camera usually waits for 5 to 10 minutes. When the trailer arrives at the designated location, the target detection model is started to perform target detection. When the second preset target is detected in the towing video frame, the preset time of the timer is set to 2 minutes. In this way, the complete multi-target recognition processing lasting 2 minutes can be performed within 5 to 10 minutes as much as possible, so that the recognition accuracy of the multi-target recognition processing is optimal.
[0078] In some embodiments, the recognition times of each second preset target are distinguished and counted to obtain a multi-target recognition result, so as to facilitate the subsequent dragging state evaluation step and improve the data processing efficiency.
[0079] Step 206, evaluating the aircraft dragging state according to the multi-target recognition result to obtain an evaluation result.
[0080] Among them, the aircraft drag status assessment is to evaluate the aircraft's drag status based on the number of times different preset targets appear within a certain period of time, so that the staff can monitor the aircraft's operating status based on the assessment results.
[0081] In some embodiments, the multi-target recognition result includes: the first recognition number of the first candidate target and the second recognition number of the second candidate target; the target detection model includes: a first weight and a second weight; the aircraft dragging status is evaluated according to the multi-target recognition result to obtain an evaluation result, including: multiplying the first recognition number according to the first weight to obtain a first evaluation value; multiplying the second recognition number according to the second weight to obtain a second evaluation value; wherein the first weight and the second weight are respectively used to reflect the influence of the first candidate target and the second candidate target on judging the aircraft dragging status; accumulating the first evaluation value and the second evaluation value to obtain a total evaluation value; performing dragging judgment according to the total evaluation value and a preset drag threshold to obtain an evaluation result.
[0082] Among them, the first candidate target is the front wheel turning mechanism, the second candidate target is the towing rod connector, and the preset towing threshold is a towing threshold that conforms to the actual situation after training the target detection model and combining the verification set. It can accurately determine whether the aircraft is in a towing state.
[0083] It should be further explained that during the training phase of the target detection model, the first candidate target and the second candidate target will be evaluated for their influence on the towing state. After the training, a weight will be assigned to the first candidate target and the second candidate target respectively. The weight is used to reflect the influence of the corresponding candidate target on whether the aircraft is in a towing state when it appears.
[0084] In some embodiments, the first number of recognition times is multiplied according to the first weight to obtain a first evaluation value that can reflect the total influence of the first candidate target on the aircraft state, and the second number of recognition times is multiplied according to the second weight to obtain a second evaluation value that can reflect the total influence of the second candidate target on the aircraft state. Finally, the first evaluation value and the second evaluation value are added to obtain a total evaluation value for determining whether the aircraft is in a towing state, and a towing judgment is performed based on the total evaluation value and a preset towing threshold to obtain an evaluation result.
[0085] In this embodiment, by performing a simple weighted accumulation method on the multi-target recognition results and using a preset drag threshold that conforms to the actual situation for comparison, an accurate assessment of the aircraft status is completed, thereby greatly improving the efficiency of aircraft status detection.
[0086] Step 208, if the evaluation result indicates that the aircraft is in a towing state, target detection is performed on a first preset target according to the target detection model to obtain a first target detection result; wherein the first preset target is used to determine whether the aircraft is in a normal towing state.
[0087] Among them, the first preset target is the front wheel turning pin of the aircraft. This target component is a component that must be used in the aircraft towing mission. Therefore, it is a core component for judging whether the aircraft is in a normal towing state. When the aircraft is in a normal towing state, it means that the aircraft is performing an aircraft towing mission.
[0088] In some embodiments, if the evaluation result indicates that the aircraft is in a towing state, that is, it indicates that the current aircraft is in a towing motion with a large running trajectory angle, it is necessary to further distinguish whether the aircraft is currently in a normal towing state of the aircraft towing task, or whether the trajectory of the aircraft has changed due to other external factors, which corresponds to an abnormal towing state, so as to autonomously judge whether the current aircraft is safe or whether manual intervention is required. Therefore, according to the target detection model, target detection is performed on the first preset target to obtain a first target detection result, which can accurately determine whether the aircraft is in a normal towing state, thereby accurately completing the judgment of the aircraft state.
[0089] In some embodiments, after performing target detection on the first preset target according to the target detection model to obtain the first target detection result, the target detection method also includes: if the first target detection result is that the first preset target is detected, then performing target detection on the dragging video frame at the current moment according to the target detection model to obtain the target detection result; if within the preset safety time, the target detection results are that the first preset target and the second preset target are not detected, then generating an exit drag state result.
[0090] Among them, the preset safety time is determined based on actual needs. During the preset safety time, the aircraft's operating status generally will not change too much. Therefore, the aircraft's status is relatively safe and reliable during this period.
[0091] In some embodiments, if the first target detection result is that the first preset target is detected, it means that the current aircraft is in a normal towing state. At this time, it is necessary to monitor in real time whether the aircraft state has changed. Therefore, the target detection model is used to perform target detection on the towing video frame at the current moment to monitor the aircraft state. If within the preset safety time, the target detection results are that the first preset target and the second preset target are not detected, it means that the operating state of the current aircraft has changed, and it is necessary to promptly remind and inform the staff of the result of exiting the towing state of the change in the aircraft operating state.
[0092] In the above target detection method, the aircraft towing video of the camera is obtained, and the aircraft towing video is decomposed at the frame level to obtain the towing video frame; the towing video frame is subjected to multi-target recognition according to the target detection model to obtain the multi-target recognition result; the aircraft towing state is evaluated according to the multi-target recognition result to obtain the evaluation result; if the evaluation result indicates that the aircraft is in the towing state, the first preset target is detected according to the target detection model to obtain the first target detection result; wherein the first preset target is used to determine whether the aircraft is in the normal towing state. Therefore, after the towing video frame obtained in the process of real-time environmental changes is subjected to multi-target recognition by the target detection model, the aircraft towing state is evaluated according to the multi-target recognition result to accurately determine the towing state of the current aircraft, and then further according to the target detection result of the first preset target, it is determined whether the aircraft is in the normal towing state, so that the first preset target can be accurately detected while the environment is constantly changing.
[0093] In an exemplary embodiment, as shown in FIG3 , before performing multi-target recognition on the drag video frame according to the target detection model to obtain the multi-target recognition result, the target detection method further includes: Step 302 to Step 306. Among them:
[0094] Step 302: Acquire feature sample data, and perform data amplification on the feature sample data to obtain training sample data.
[0095] The characteristic sample data is sample pictures with obvious characteristics screened out from video images of the front wheel well area of the aircraft acquired in real time in the aircraft towing scene.
[0096] In some embodiments, feature sample data is amplified by rotation, scaling, translation, adding random noise, etc., thereby expanding the sample size and obtaining a large amount of training sample data with obvious features, so as to facilitate subsequent training to obtain a more accurate target detection model.
[0097] Step 304, training the original detection model according to the training sample data, and adjusting the parameters of the original detection model according to the gradient descent model during the training process to minimize the loss function corresponding to the original detection model, thereby obtaining a target detection model to be evaluated;
[0098] Among them, the original detection model can adopt the YOLOv8 model.
[0099] It should be further explained that, in the process of training the original detection model according to the training sample data, a labeling strategy was adopted to label the front wheel turning pin, the front wheel turning total mechanism, and the tow bar connector respectively, so as to ensure that the trained detection model to be evaluated can autonomously distinguish between the normal driving state of the trailer and the state of performing the aircraft towing task. The specific diagram of the labeling strategy can be seen in Figure 4, in which coblebox is the front wheel turning total mechanism, towbar is the tow bar connector, and latch is the front wheel turning pin.
[0100] In some embodiments, the original detection model is trained using a deep learning framework, and during the training process, the original detection model continuously adjusts parameters through back propagation and gradient descent models to minimize the loss function corresponding to the original detection model, thereby obtaining a target detection model to be evaluated with excellent recognition accuracy.
[0101] Step 306 , the target detection model to be evaluated is evaluated based on the validation set data of the training sample data. If the evaluation result meets the preset evaluation conditions, the target detection model to be evaluated is used as the target detection model.
[0102] Among them, the preset evaluation conditions can be determined according to actual needs.
[0103] In some embodiments, the target detection model to be evaluated is evaluated based on the validation set data of the training sample data. If the evaluation result meets the preset evaluation conditions, it means that the target detection model to be evaluated has good generalization ability on unseen data. Therefore, the target detection model to be evaluated with good generalization ability and excellent recognition accuracy can be used as the target detection model.
[0104] Among them, the specific accuracy-recall curve of the target detection model to be evaluated on the verification data set can be found in Figure 5, where the target object can be the first preset target or the second preset target.
[0105] In this embodiment, feature sample data is obtained and data amplification is performed on the feature sample data. The original detection model is trained based on the training sample data obtained by the data amplification. During the training process, the parameters of the original detection model are adjusted according to the gradient descent model to minimize the loss function corresponding to the original detection model. The target detection model to be evaluated is obtained. Then, the target detection model to be evaluated is evaluated based on the verification set data of the training sample data. If the evaluation result meets the preset evaluation conditions, the target detection model to be evaluated is used as the target detection model, so that a target detection model with good generalization ability and excellent recognition accuracy can be obtained.
[0106] In some embodiments, in order to more clearly understand the target detection process of the present application, please refer to FIG. 6 to more specifically understand the judgment process in the target detection process.
[0107] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0108] Based on the same inventive concept, the embodiment of the present application also provides a target detection device for implementing the target detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more target detection device embodiments provided below can refer to the limitations on the target detection method above, and will not be repeated here.
[0109] In an exemplary embodiment, as shown in FIG. 7 , a target detection device is provided, including: an acquisition module 702, a multi-target recognition module 704, an evaluation module 706 and a target detection module 708, wherein:
[0110] The acquisition module 702 is used to acquire the aircraft towing video of the camera and perform video frame decoding on the aircraft towing video to obtain the towing video frame;
[0111] A multi-target recognition module 704 is used to perform multi-target recognition on the dragging video frame according to the target detection model to obtain a multi-target recognition result;
[0112] An evaluation module 706 is used to evaluate the aircraft drag state according to the multi-target recognition result to obtain an evaluation result;
[0113] The target detection module 708 is used to perform target detection on a first preset target according to a target detection model to obtain a first target detection result if the evaluation result indicates that the aircraft is in a towing state; wherein the first preset target is used to determine whether the aircraft is in a normal towing state.
[0114] In some embodiments, the multi-target recognition module 704 is also used to record the number of recognitions of each second preset target within a preset time when the target detection model recognizes that there is a second preset target in the towing video frame; wherein the second preset target is used to determine whether the aircraft is in a towing state; and statistics are performed based on the number of recognitions to obtain a multi-target recognition result.
[0115] In some embodiments, the multi-target recognition result includes: the first recognition number of the first candidate target and the second recognition number of the second candidate target; the target detection model includes: a first weight and a second weight; the evaluation module 706 is also used to multiply the first recognition number according to the first weight to obtain a first evaluation value; multiply the second recognition number according to the second weight to obtain a second evaluation value; wherein the first weight and the second weight are respectively used to reflect the influence of the first candidate target and the second candidate target on judging the drag status of the aircraft; accumulate the first evaluation value and the second evaluation value to obtain a total evaluation value; perform drag judgment based on the total evaluation value and a preset drag threshold to obtain an evaluation result.
[0116] In some embodiments, the target detection device also includes: an exit drag state module, which is used to perform target detection on the dragging video frame at the current moment according to the target detection model to obtain a target detection result if the first target detection result is that the first preset target is detected; if within the preset safety time, the target detection results are that the first preset target and the second preset target are not detected, then an exit drag state result is generated.
[0117] In some embodiments, the target detection device also includes: a focal length lens selection module, which is used to obtain the shooting distance between the aircraft's towing observation area and a preset shooting point; perform focal length calculation based on the shooting distance and a preset focal length model to obtain the target focal length; and perform focal length lens selection based on the target focal length to obtain a camera.
[0118] In some embodiments, before performing multi-target recognition on the dragged video frame according to the target detection model to obtain the multi-target recognition result, the target detection device also includes: a model training module, which is used to obtain feature sample data and perform data amplification on the feature sample data to obtain training sample data; training the original detection model according to the training sample data, and adjusting the parameters of the original detection model according to the gradient descent model during the training process to minimize the loss function corresponding to the original detection model to obtain the target detection model to be evaluated; evaluating the target detection model to be evaluated according to the verification set data of the training sample data, and if the evaluation result meets the preset evaluation conditions, the target detection model to be evaluated is used as the target detection model.
[0119] In the above-mentioned target detection device, the aircraft towing video of the camera is obtained, and the aircraft towing video is decomposed at the frame level to obtain the towing video frame; the towing video frame is subjected to multi-target recognition according to the target detection model to obtain the multi-target recognition result; the aircraft towing state is evaluated according to the multi-target recognition result to obtain the evaluation result; if the evaluation result indicates that the aircraft is in the towing state, the first preset target is detected according to the target detection model to obtain the first target detection result; wherein the first preset target is used to determine whether the aircraft is in the normal towing state. Therefore, after the towing video frame obtained in the process of real-time environmental changes is subjected to multi-target recognition by the target detection model, the aircraft towing state is evaluated according to the multi-target recognition result to accurately determine the towing state of the current aircraft, and then further according to the target detection result of the first preset target, it is determined whether the aircraft is in the normal towing state, so that the first preset target can be accurately detected while the environment is constantly changing.
[0120] Each module in the above target detection device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.
[0121] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG8. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store aircraft towing video data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a target detection method is implemented.
[0122] Those skilled in the art will understand that the structure shown in FIG. 8 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.
[0123] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-mentioned target detection method when executing the computer program.
[0124] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned target detection method are implemented.
[0125] In one embodiment, a computer program product is provided, comprising a computer program, which implements the steps of the above-mentioned target detection method when executed by a processor.
[0126] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0127] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.
[0128] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0129] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A target detection method, characterized in that: The method comprises: Acquire an aircraft towing video from a camera, and perform frame-level decomposition on the aircraft towing video to obtain towing video frames; Performing multi-target recognition on the towed video frame according to the target detection model to obtain a multi-target recognition result; Perform aircraft drag status evaluation according to the multi-target recognition result to obtain an evaluation result; If the evaluation result indicates that the aircraft is in a towing state, target detection is performed on a first preset target according to the target detection model to obtain a first target detection result; wherein the first preset target is used to determine whether the aircraft is in a normal towing state.
2. The method according to claim 1, characterized in that The performing multi-target recognition on the dragging video frame according to the target detection model to obtain a multi-target recognition result includes: When the target detection model recognizes that there is a second preset target in the towing video frame, the number of times each second preset target is recognized is recorded within a preset time; wherein the second preset target is used to determine whether the aircraft is in a towing state; Statistics are performed based on the number of recognitions to obtain the multi-target recognition result.
3. The method according to claim 1, characterized in that: The multi-target recognition result includes: a first recognition number of the first candidate target and a second recognition number of the second candidate target; the target detection model includes: a first weight and a second weight; the aircraft drag state evaluation is performed according to the multi-target recognition result to obtain an evaluation result, including: Multiplying the first recognition times according to the first weight to obtain a first evaluation value; The second identification times are multiplied according to the second weight to obtain a second evaluation value; wherein the first weight and the second weight are respectively used to reflect the influence of the first candidate target and the second candidate target on judging the drag state of the aircraft; Accumulating the first evaluation value and the second evaluation value to obtain a total evaluation value; A dragging judgment is performed according to the total evaluation value and a preset dragging threshold to obtain the evaluation result.
4. The method according to claim 1, characterized in that: After performing target detection on the first preset target according to the target detection model to obtain a first target detection result, the method further includes: If the first target detection result is that the first preset target is detected, performing target detection on the dragging video frame at the current moment according to the target detection model to obtain a target detection result; If within the preset safety time, the target detection results are both that the first preset target and the second preset target are not detected, then an exit drag state result is generated.
5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Obtaining a shooting distance between the towing observation area of the aircraft and a preset shooting point; Performing focal length calculation according to the shooting distance and a preset focal length model to obtain a target focal length; A focal length lens is selected according to the target focal length to obtain the camera.
6. The method according to claim 1, characterized in that Before performing multi-target recognition on the dragged video frame according to the target detection model to obtain a multi-target recognition result, the method further includes: Acquire feature sample data, and perform data amplification on the feature sample data to obtain training sample data; The original detection model is trained according to the training sample data, and during the training process, parameters of the original detection model are adjusted according to the gradient descent model to minimize the loss function corresponding to the original detection model, thereby obtaining a target detection model to be evaluated; The target detection model to be evaluated is evaluated according to the validation set data of the training sample data. If the evaluation result meets the preset evaluation condition, the target detection model to be evaluated is used as the target detection model.
7. A target detection device, characterized in that: The device comprises: An acquisition module is used to acquire the aircraft towing video of the camera, and perform video frame decoding on the aircraft towing video to obtain a towing video frame; A multi-target recognition module, used to perform multi-target recognition on the towed video frame according to a target detection model to obtain a multi-target recognition result; An evaluation module, used to evaluate the aircraft drag state according to the multi-target recognition result to obtain an evaluation result; The target detection module is used to perform target detection on a first preset target according to the target detection model to obtain a first target detection result if the evaluation result indicates that the aircraft is in a towing state; wherein the first preset target is used to determine whether the aircraft is in a normal towing state.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
Citation Information
Patent Citations
Method of automatically detecting entry and exit of airplane based on video monitoring
CN109302580A
Airport apron airplane state detection method and device
CN112530205A
Double-stage article detection method and device
CN113505771A
Target detection tracking identification method and system based on multi-feature information fusion
CN114119676A
Intelligent monitoring method, device and system for flight guarantee nodes
CN115550611A
