Target object classification device based on image recognition and use method thereof

Abstract

The invention relates to a target object classification device based on image recognition and a use method thereof, when a camera is in an FOD detection mode, foreign matters on an airport pavement can be automatically recognized in real time, the image, the type, the position and the detection time of the foreign matters are output, and AI detection with a model as a basis and data as a center is carried out; detection result data are exported to a GIS visual management system for statistics and analysis of all tasks, including the number, characteristics, types, sources and the like of FOD, and statistical analysis results are incorporated into an FOD database, so that sensitive areas can be identified conveniently, and the FOD management level is improved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to an object classification device based on image recognition and a method for using the same. Background Art

[0002] The presence of FOD is a major safety hazard for aircraft, because a small piece of plastic cloth sucked into the engine may cause it to stall in mid-air. A small screw, a small piece of metal, or a sharp pebble may puncture the aircraft tire and cause a blowout. The resulting tire fragments may cause serious consequences due to high-speed impact on the aircraft body, hydraulic pipes, fuel tanks and other parts.

[0003] Currently, most airports still use traditional manual visual inspections or radar to regularly inspect and clean runways. However, due to the subjectivity of inspectors and weather conditions, it is difficult to ensure a sufficient detection rate. Millimeter-wave radar can monitor around the clock, but requires fixed installation and is only used for runway FOD detection. If the detection target is larger than 2 cm, the FOD type cannot be output. Summary of the Invention

[0004] The problem solved by the present invention is to provide a target object classification device based on image recognition and a method for using the same, which solves the current problem that most airports still use traditional manual visual patrol methods or radar to conduct regular inspections and cleaning of runways. Due to the subjectivity of the inspectors themselves and weather conditions, it is difficult to ensure a sufficient detection rate. Millimeter-wave radar can monitor all day long, but requires fixed installation and is only used for runway FOD detection; if the detection target is larger than 2 cm, the type of FOD cannot be output.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] An image recognition-based object classification device includes a mobile vehicle, a detection mechanism mounted on the top side of the mobile vehicle, a positioning module mounted inside the mobile vehicle, a mobile terminal module, a system configuration module, and a management and control module. The detection mechanism includes a frame, a first camera assembly, and a second camera assembly. The first camera assembly is symmetrically mounted near the rear end of the mobile vehicle, and the second camera assembly is symmetrically mounted in the middle of the frame, with the two second camera assemblies located between the two first camera assemblies. The first camera assembly and the second camera assembly each include an electric turntable, a support rod rotatably mounted on the electric turntable, and a camera is mounted on the outer end of the support rod.

[0007] The positioning module receives the data from the positioning module and parses the latitude, longitude and heading information. It uses the geodetic theme forward solution algorithm to calculate the latitude and longitude information of each camera image and the position information of the FOD based on the calibration external parameters.

[0008] The mobile terminal module monitors the connection status and communication status of the mobile terminal device, connects to read the database to obtain data, and writes data processing results;

[0009] The system configuration module includes camera initial value settings and image preview, calibration parameter settings, and airport basic data import. It can adjust the camera exposure value and gain according to the lighting conditions, and set calibration parameters including distance, angle, height, and focal length; basic data settings include plate data and area division;

[0010] Management and control module, performs operations and controls related to project engineering.

[0011] Preferably, a bottom groove is provided on the bottom side of the frame near the rear end of the moving vehicle, and a motor is symmetrically installed on the frame near the rear end of the moving vehicle.

[0012] Preferably, the output end of the motor is located in the bottom groove and is equipped with a rotating arm, and the outer end of the rotating arm is equipped with a lighting lamp.

[0013] Preferably, a reducer is installed on the frame, and the input end of the reducer is connected to the output end of the regulating motor.

[0014] Preferably, a rotating table is installed at the output end of the reducer, and a rotating table is installed on the rotating table.

[0015] A method for using an object classification device based on image recognition, wherein the specific steps of the method are as follows:

[0016] S1. When the camera is in FOD detection mode, it can automatically identify foreign objects on the airport pavement in real time and output the image, type, location, and detection time of the foreign object through model-based, data-centric AI detection;

[0017] S102, algorithm design, adopts a single-stage detection algorithm, inputs the original image, and after CNN feature extraction, outputs the target classification and location;

[0018] Changes in feature extraction methods: The FOCUS structure is used to divide the input feature map into four sub-maps, and then these four sub-maps are spliced ​​in the channel dimension to form the output feature map;

[0019] Optimizing detection accuracy: We use an FPN+PAN structure. FPN is a feature map pyramid network that fuses features from different feature maps and then performs predictions on the fused feature maps. This significantly improves the performance of small object detection without increasing the computational complexity of the original model. A PAN is added after FPN, forming a bottom-up pyramid that complements FPN and passes low-level positioning features upwards. The resulting pyramid combines both semantic and positioning information.

[0020] S102, sample data, build data set;

[0021] Data annotation quality: First, ensure the consistency of annotations. All instances of all classes in all images must be labeled. Second, ensure the accuracy of annotations. The bounding box must tightly surround each object, with no space between the object and the bounding box. The error must be within 1 pixel. Finally, no object should be missing a label, and the labels must be accurate.

[0022] Data annotation quantity: more than 10,000 images per category and more than 50,000 instances per category;

[0023] Data diversity: images from different time periods, seasons, weather conditions, lighting conditions, and angles;

[0024] Data balance;

[0025] S103, training strategy;

[0026] Cosine annealing learning rate: By cycling the learning rate between high and low bounds, it gradually decreases at the beginning, reaches a minimum at the maximum number of iterations, and then increases again

[0027] Balance between accuracy and speed: The input image size during training should be consistent with the image size during inference. The image size needs to be balanced between accuracy and speed based on the original image size and target recognition performance.

[0028] S2. The identified targets are divided into three categories based on their hazard level: high-risk, medium-risk, and low-risk. High-risk targets include stones, metal parts, maintenance tools, wires, etc., which are all high-risk regardless of size; medium-risk targets include wood, rubber, cloth, plastic, etc., which are all high-risk if they are too large; low-risk targets include leaves and paper.

[0029] The beneficial effects of the present invention are:

[0030] As the mobile vehicle moves, the first camera assembly and the second camera assembly are used to conduct a full range of inspections of foreign objects at the airport, ensuring inspection efficiency and effectively solving the problem of foreign objects at the airport.

[0031] The detection result data is exported to the GIS visualization management system for statistics and analysis of its various tasks, including the number, characteristics, type, source, etc. of FOD, and the results of the statistical analysis are included in the FOD database to facilitate the identification of sensitive areas and improve the level of FOD management. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a schematic diagram of the first structure of the detection mechanism of the present invention;

[0033] Figure 2 This is a second structural diagram of the detection mechanism of the present invention.

[0034] Legend:

[0035] 1. Frame; 2. First camera assembly; 3. Second camera assembly; 4. Electric turntable; 5. Support rod; 6. Camera; 7. Bottom trough; 8. Motor; 9. Rotating arm; 10. Lighting lamp; 11. Reducer; 12. Rotating table; 13. Antenna. DETAILED DESCRIPTION

[0036] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0037] Specific examples are given below.

[0038] See also Figure 1 and Figure 2, a target object classification device based on image recognition, including a mobile vehicle, a detection mechanism installed on the top side of the mobile vehicle, a positioning module installed inside the mobile vehicle, a mobile terminal module, a system configuration module and a management and control module, the detection mechanism includes a frame 1, a first camera assembly 2 and a second camera assembly 3, the frame 1 is symmetrically installed with the first camera assembly 2 near the rear end of the mobile vehicle, the second camera assembly 3 is symmetrically installed in the middle of the frame 1, and the two second camera assemblies 3 are located between the two first camera assemblies 2, the first camera assembly 2 and the second camera assembly 3 both include an electric turntable 4, a support rod 5 is rotatably installed on the electric turntable 4, and a camera 6 is installed at the outer end of the support rod 5, a bottom groove 7 is opened on the bottom side of the frame 1 near the rear end of the mobile vehicle, a motor 8 is symmetrically installed on the frame 1 near the rear end of the mobile vehicle, and a rotating arm 9 is installed in the bottom groove 7 at the output end of the motor 8 , a lighting lamp 10 is installed at the outer end of the rotating arm 9, a reducer 11 is installed on the frame 1, and the input end of the reducer 11 is connected to the output end of the adjustment motor, a rotating table 12 is installed at the output end of the reducer 11, and a rotating table 12 is installed on the rotating table 12. When the mobile car moves, the detection mechanism is used to detect and inspect foreign objects in the airport. The electric turntable 4 drives the support rod 5 to rotate, and then rotates the camera 6 located on the frame 1 to the outside of the frame 1, and collects images through the camera 6. At this time, the first camera assembly 2 and the second camera assembly 3 realize multi-directional and comprehensive collection. At the same time, the motor is adjusted to work, and the speed of the reducer 11 is adjusted to realize the rotation of the rotating table 12 and the antenna 13, so that the antenna 13 is in a vertical state during work. The rotating arm 9 is driven to rotate by the motor 8, and then the lighting lamp 10 is rotated to the outside of the frame 1, providing better collection lighting for the camera 6.

[0039] The positioning module receives the data from the positioning module and parses the latitude, longitude and heading information. It uses the geodetic theme forward solution algorithm to calculate the latitude and longitude information of each camera image and the position information of the FOD based on the calibration external parameters.

[0040] The mobile terminal module monitors the connection status and communication status of the mobile terminal device, connects to read the database to obtain data, and writes data processing results;

[0041] The system configuration module includes camera initial value settings and image preview, calibration parameter settings, and airport basic data import. It can adjust the camera exposure value and gain according to the lighting conditions, and set calibration parameters including distance, angle, height, and focal length; basic data settings include plate data and area division;

[0042] Management and control module, performs operations and controls related to project engineering.

[0043] A method for using an object classification device based on image recognition, the specific steps of the method are as follows:

[0044] When S1 and Camera 6 are in FOD detection mode, they can automatically identify foreign objects on the airport pavement in real time and output the image, type, location, and detection time of the foreign object through model-based, data-centric AI detection;

[0045] S102, algorithm design, adopts a single-stage detection algorithm, inputs the original image, and after CNN feature extraction, outputs the target classification and location;

[0046] Changes in feature extraction methods: The FOCUS structure is used to divide the input feature map into four sub-maps, and then these four sub-maps are spliced ​​in the channel dimension to form the output feature map;

[0047] Optimizing detection accuracy: We use an FPN+PAN structure. FPN is a feature map pyramid network that fuses features from different feature maps and then performs predictions on the fused feature maps. This significantly improves the performance of small object detection without increasing the computational complexity of the original model. A PAN is added after FPN, forming a bottom-up pyramid that complements FPN and passes low-level positioning features upwards. The resulting pyramid combines both semantic and positioning information.

[0048] S102, sample data, build data set;

[0049] Data annotation quality: First, ensure the consistency of annotations. All instances of all classes in all images must be labeled. Second, ensure the accuracy of annotations. The bounding box must tightly surround each object, with no space between the object and the bounding box. The error must be within 1 pixel. Finally, no object should be missing a label, and the labels must be accurate.

[0050] Data annotation quantity: more than 10,000 images per category and more than 50,000 instances per category;

[0051] Data diversity: images from different time periods, seasons, weather conditions, lighting conditions, and angles;

[0052] Data balance;

[0053] S103, training strategy;

[0054] Cosine annealing learning rate: By cycling the learning rate between high and low bounds, it gradually decreases at the beginning, reaches a minimum at the maximum number of iterations, and then increases again

[0055] Balance between accuracy and speed: The input image size during training should be consistent with the image size during inference. The image size needs to be balanced between accuracy and speed based on the original image size and target recognition performance.

[0056] S2. The identified targets are divided into three categories based on their hazard level: high-risk, medium-risk, and low-risk. High-risk targets include stones, metal parts, maintenance tools, wires, etc., which are all high-risk regardless of size; medium-risk targets include wood, rubber, cloth, plastic, etc., which are all high-risk if they are too large; low-risk targets include leaves and paper.

[0057] The above are only preferred specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solutions and inventive concepts of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A target object classification device based on image recognition, characterized in that: The invention comprises a mobile vehicle, a detection mechanism installed on the top side of the mobile vehicle, a positioning module installed inside the mobile vehicle, a mobile terminal module, a system configuration module and a management and control module. The detection mechanism comprises a frame (1), a first camera assembly (2) and a second camera assembly (3). The frame (1) is symmetrically mounted with the first camera assembly (2) near the rear end of the mobile vehicle. The middle part of the frame (1) is symmetrically mounted with the second camera assembly (3), and the two second camera assemblies (3) are located between the two first camera assemblies (2). The first camera assembly (2) and the second camera assembly (3) both comprise an electric turntable (4). A support rod (5) is rotatably mounted on the electric turntable (4), and a camera (6) is mounted on the outer end of the support rod (5). The positioning module receives the data from the positioning module and analyzes the latitude, longitude and heading information. It uses the geodetic theme positive solution algorithm to calculate the latitude and longitude information of each camera image and the location information of the FOD according to the calibration external parameters. The mobile terminal module monitors the connection status and communication status of the mobile terminal device, connects to read the database to obtain data, and writes data processing results; System configuration module, including camera initial value setting and image preview, calibration parameter setting, airport basic data import, etc. The camera exposure value and gain can be adjusted according to the lighting conditions, and calibration parameters can be set, including distance, angle, height and focal length; basic data setting, including plate data and regional division; Management and control module performs operations and controls related to project engineering.

2. The airport foreign body detection and inspection device according to claim 1, characterized in that: The frame (1) is provided with a bottom groove (7) at the bottom side close to the rear end of the moving vehicle, and a motor (8) is symmetrically installed on the frame (1) close to the rear end of the moving vehicle.

3. The airport foreign body detection and inspection device according to claim 2, characterized in that: The output end of the motor (8) is located in the bottom groove (7) and is equipped with a rotating arm (9), and the outer end of the rotating arm (9) is equipped with a lighting lamp (10).

4. The airport foreign body detection and inspection device according to claim 3, characterized in that: A reducer (11) is installed on the frame (1), and an input end of the reducer (11) is connected to an output end of the regulating motor.

5. The airport foreign body detection and inspection device according to claim 4, characterized in that: The output end of the reducer (11) is provided with a rotating platform (12), and a rotating platform (12) is provided on the rotating platform (12).

6. The method for using the target classification device based on image recognition according to claim 5, characterized in that: The specific operation steps of the method are as follows: When S1 and camera (6) are in FOD detection mode, they can automatically identify foreign objects on the airport pavement in real time and output the image, type, location, and detection time of the foreign objects through model-based, data-centric AI detection; S102, algorithm design, adopts a single-stage detection algorithm, inputs the original image, and outputs the target classification and location after CNN feature extraction; Changes in feature extraction methods: The FOCUS structure is used to divide the input feature map into four sub-maps, and then these four sub-maps are spliced ​​in the channel dimension to form the output feature map; Optimization of detection accuracy: Adopt FPN+PAN structure. FPN is a feature map pyramid network. It fuses features from different feature maps and makes predictions on the fused feature maps. It greatly improves the performance of small object detection without increasing the computational complexity of the original model. Add a PAN after FPN, a bottom-up pyramid, to supplement FPN and pass the positioning features of the lower layers to the upper layers. The formed pyramid combines both semantic information and positioning information. S102, sample data, constructing a data set; Data labeling quality: First, ensure the consistency of labeling. All instances of all classes in all images must be labeled. Second, ensure the accuracy of labeling. The bounding box must tightly surround each object. There should be no space between the object and the bounding box. The error must be within 1 pixel. Finally, no object should be missing a label and the label must be accurate. Number of data annotations: more than 10,000 images per category and more than 50,000 instances per category; Data diversity: images from different time periods, seasons, weather conditions, lighting conditions, and angles; Data balance; S103, training strategy; Cosine annealing learning rate: By cycling the learning rate between high and low bounds, it gradually decreases at the beginning, reaches a minimum at the maximum number of iterations, and then increases again Balance between accuracy and speed: The input image size during training should be consistent with the image size during inference. The image size needs to balance accuracy and speed based on the size of the original image and the target recognition situation. S2. According to the hazard level of the identified targets, they are divided into three categories: high risk, medium risk and low risk. High risk objects are mainly stones, metal parts, maintenance tools, wires, etc., all of which are high risk regardless of size; medium risk objects are mainly wood, rubber, cloth, plastic, etc., and those that are too large are high risk; low risk objects are mainly leaves and paper.

Images