Vehicle-mounted FOD dynamic detection and positioning method based on airport runway

Abstract

The application relates to the technical field of FOD detection, in particular to a vehicle-mounted FOD dynamic detection and positioning method based on an airport runway. The method generates a target image through target data collected by a vehicle-mounted device, extracts a distorted segment in the target image and constructs a dynamic feature tree. Temporary sub-nodes are generated under the feature nodes in combination with vehicle motion parameters and scanning visual angle information, temporary branches are established through structure simulation, dynamic splicing and closed loop judgment of foreign matter structures are realized, and finally the detection result and the spatial position of the FOD are output. The method improves the recognition accuracy and positioning robustness of the foreign matter target in the distorted form in the image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of FOD detection, in particular to a vehicle-mounted FOD dynamic detection and positioning method based on an airport runway. BACKGROUND

[0002] In the prior art, airport runway foreign object detection mainly relies on image sensors or laser radars and other imaging devices to identify abnormal targets on the runway surface through static or low-speed inspection methods. However, due to the vast area of the airport runway, the small size of foreign objects and their close proximity to the ground, traditional imaging devices have significant limitations in imaging accuracy, occlusion adaptability and weak feature recognition. In particular, in the vehicle-mounted dynamic detection scene using millimeter wave radars, due to the continuous change of the relative angle between the radar beam and the target, the reflection characteristics of the foreign object in the image are prone to distortion phenomena such as stretching, twisting and fragmentation, which makes it impossible to accurately identify the true boundary and spatial position of the target.

[0003] For example, Chinese patent application No. CN115359425A provides a FOD detection system and method based on a double-channel dynamic variable structure neural network. The system includes a shell and an image acquisition device installed inside the shell, two image preprocessing modules, a comparator, an image feature inversion module, an image background removal module, a foreign object positioning module, a foreign object existence recognition module and a foreign object type recognition module. The advantages and beneficial effects of the invention are: it can realize rapid positioning and type recognition of foreign objects in the airport, help airport staff to quickly and accurately remove them, and thus ensure the safe takeoff and landing of flights. In addition, the detection effect is good and the accuracy is high.

[0004] The above prior art all have the problem proposed in the background: it is difficult to deal with the misjudgment and missed judgment problem caused by structural distortion, and there is a lack of modeling and structural restoration mechanism for the target morphological evolution process. To solve the above problems, the present application designs a vehicle-mounted FOD dynamic detection and positioning method based on an airport runway. SUMMARY

[0005] The technical problem to be solved by the present application is to overcome the shortcomings of the prior art and provide a vehicle-mounted FOD dynamic detection and positioning method based on an airport runway. The method generates a target image from the target data collected by the vehicle-mounted device, extracts the distorted segments therein and constructs a dynamic feature tree. In combination with the vehicle motion parameters and scanning angle information, temporary child nodes are generated under the feature nodes, temporary branches are established through structure simulation, the dynamic splicing and closed loop judgment of foreign object structures are realized, and finally the detection results and spatial position of FOD are output. This method improves the recognition accuracy and positioning robustness of foreign object targets in the distorted form in the image.

[0006] To achieve the above object, the application provides the following technical scheme: a vehicle-mounted FOD dynamic detection and positioning method based on an airport runway, applied to a vehicle-mounted device, the vehicle-mounted FOD dynamic detection and positioning method comprises the following steps: generating a target image according to target data collected by the vehicle-mounted device, and obtaining a distortion segment of the target image.

[0007] A dynamic feature tree is constructed, the distortion segment is initialized as a feature node of the dynamic feature tree, and an independent structure branch is generated under each feature node, wherein the number of the independent structure branch is determined according to a local morphological feature of the feature node, at least one temporary child node is generated according to each feature node, and the temporary child node is used to represent a structure part of the target which is not imaged, and the generation of the temporary child node comprises the following steps: generating an edge defect description according to an open contour segment of an edge structure end of the feature node.

[0008] According to the edge defect description, the starting temporary child node of the independent structure branch is generated in combination with vehicle motion parameters and scanning visual angle information of the vehicle-mounted device during collection.

[0009] An iteration operation is performed, the temporary child node at the end in the independent structure branch is taken as input, the next temporary child node is generated according to the input edge defect description in combination with the vehicle motion parameters and the scanning visual angle information, and the next temporary child node is connected to the temporary child node at the end until the next temporary child node is structurally closed, and the iteration is terminated.

[0010] The temporary child nodes are traversed, if there are a pair of child nodes satisfying a preset structure closure condition in the temporary child nodes, a temporary branch is generated between the corresponding feature nodes.

[0011] According to the temporary branch, a FOD detection result is generated, and a spatial position thereof is output.

[0012] The initialization of the distortion segment as the feature node of the dynamic feature tree comprises the following steps: calculating an image structure descriptor of the distortion segment, wherein the image structure descriptor comprises an edge direction distribution, a pixel position set and a local morphological feature of the corresponding distortion segment.

[0013] According to the image structure descriptor, the distortion segment is written into the dynamic feature tree as the feature node.

[0014] The generation of the starting temporary child node of the independent structure branch in combination with the vehicle motion parameters and the scanning visual angle information comprises the following steps: generating a derived structure descriptor according to the image structure descriptor of the feature node and the corresponding local morphological feature in the image structure descriptor.

[0015] According to the derived structure descriptor, the starting temporary child node of the independent structure branch is initialized.

[0016] The generating the derived structure descriptor comprises: if the local morphological feature of the feature node is point-like, calculating a structure extension trend according to the main reflection direction in the edge defect description and the vehicle motion parameter, generating a pixel feature extension region at a preset offset distance along the structure extension trend, simulating the reflection feature and the structure boundary of the feature node in the pixel feature extension region through the edge direction distribution of the feature node, and generating the derived structure descriptor.

[0017] If the local morphological feature of the feature node is line-like, the derived structure descriptor corresponding to the pixel feature aggregation region and the derived structure descriptor corresponding to the pixel feature extension region are generated, and the generation specifically comprises: calculating a bidirectional structure extension trend according to the edge direction consistency in the edge defect description and the scanning view angle information, and respectively generating the pixel feature extension region and the pixel feature aggregation region according to the vehicle motion parameter in each structure extension trend direction.

[0018] The reflection feature and the structure boundary of the feature node are simulated in the pixel feature aggregation region through the edge closeness of the feature node, and the derived structure descriptor corresponding to the pixel feature aggregation region is generated.

[0019] The reflection feature and the structure boundary of the feature node are simulated in the pixel feature extension region through the edge direction distribution of the feature node, and the derived structure descriptor corresponding to the pixel feature extension region is generated.

[0020] If the local morphological feature of the feature node is fragment-like, a convergence vector field is calculated according to the pixel centroid in the edge defect description, the vehicle motion parameter and the scanning view angle information, a pixel feature aggregation region is generated along the center trend direction of the convergence vector field, the reflection feature and the structure boundary of the feature node are simulated in the pixel feature aggregation region through the edge closeness of the feature node, and the derived structure descriptor is generated.

[0021] The generating the temporary branch between the corresponding feature nodes comprises: extracting a temporary child node in an independent structure branch of different feature nodes.

[0022] For any two temporary child nodes under different independent structure branches, whether there is a mapping relationship of structure direction complementarity and edge morphology compatibility is judged according to the derived structure descriptor.

[0023] If the mapping relationship exists, a temporary branch is established to connect the corresponding feature nodes.

[0024] The method further comprises: constructing a structure connection graph according to the temporary branch established between the feature nodes through the temporary child node.

[0025] Whether the structure connection graph is completely closed is judged.

[0026] If the complete closure is not satisfied, a sub-chain set constituting a closed structure is extracted in the structure connection graph.

[0027] In the sub-chain set, a closed sub-chain with the largest number of feature nodes is selected, and the corresponding temporary branch is retained, and other temporary branches are pruned.

[0028] If there are multiple closed sub-chains with the same number of feature nodes, the temporary branch is determined according to the matching confidence of the independent structure branch corresponding to the closed sub-chain.

[0029] Obtaining the distortion segment of the target image includes: processing the target image by a region growing algorithm, expanding and connecting regions in a local range according to pixel gray value similarity, and generating a plurality of candidate segments.

[0030] Extracting a contour edge for each candidate segment, and constructing a region boundary graph according to the edge density distribution and contour closure degree of the contour edge.

[0031] According to the preset judgment condition, the region boundary graph is screened, and the candidate segment screened is taken as the distortion segment.

[0032] The judgment condition for screening the region boundary graph includes one of the following: the region boundary graph has a candidate segment, and the difference between the boundary line length and the perimeter of the minimum circumscribed rectangle thereof is greater than or equal to a first threshold proportion difference. The region boundary graph has a candidate segment, and the proportion of the edge direction distribution outside the main direction is greater than or equal to a second threshold direction dispersion rate. The region boundary graph has a candidate segment, and the boundary closure degree is less than or equal to a third threshold closure index. The region boundary graph has a candidate segment, and the pixel intensity gradient variance of the boundary is greater than or equal to a fourth threshold gradient dispersion value.

[0033] According to the temporary branch, a FOD detection result is generated, and the spatial position thereof is output, including: generating FOD detection information according to the feature nodes connected by the temporary branch.

[0034] According to the vehicle motion parameters and the scanning view angle information corresponding to each feature node, the spatial position of the corresponding feature node in the world coordinate system is calculated.

[0035] The spatial position is multi-point fitted to generate spatial information.

[0036] The FOD detection information and the spatial information are displayed in a display terminal.

[0037] Compared with the prior art, the beneficial effects of the present application are: the present application realizes unified modeling and continuous restoration of different distortion states of FOD in images by constructing a dynamic feature tree and introducing a temporary sub-node and a temporary branch structure expression method. In the process of generating a temporary sub-node, the possible structural extension trend of the target is dynamically deduced in combination with vehicle motion parameters and scanning angle information, and the unimaged area is predicted and expressed through a variety of pixel feature simulation strategies, so that the complete perception of the foreign object structure can still be maintained under the condition of incomplete imaging; further, the temporary branches between the feature nodes are established through the mapping relationship judgment of the derived structure descriptor, effectively solving the problem of structure splitting or intermittent distribution of the target in multiple images. BRIEF DESCRIPTION OF DRAWINGS

[0038] Other characteristics, objects and advantages of the present application will become more apparent after reading the following detailed description of non-limiting embodiments made with reference to the attached drawings:

[0039] Figure 1 An exemplary application scenario for an embodiment of the present application.

[0040] Figure 2 A flowchart of a vehicle-mounted FOD dynamic detection and positioning method based on an airport runway for an embodiment of the present application.

[0041] Figure 3 An initialization structure diagram of a dynamic feature tree for an embodiment of the present application.

[0042] Figure 4 A principle diagram of generating a corresponding independent structure branch starting sub-node for a feature node under a linear shape condition for an embodiment of the present application.

[0043] Figure 5 A principle diagram of generating a temporary branch for an embodiment of the present application.

[0044] Figure 6 A non-closed loop principle diagram of a temporary branch for an embodiment of the present application. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments.

[0046] Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. Those skilled in the art will appreciate from the present disclosure that embodiments described herein can be combined with other embodiments in various ways.

[0047] FOD mentioned in the present application is the full name of Foreign Object Debris, which specifically refers to any foreign debris in a system or device.

[0048] The present application focuses on the detection and positioning of airport runway foreign objects, and is particularly suitable for mobile detection based on vehicle-mounted equipment. It is easy to understand that the characteristics of airport runway FOD detection are as follows: the overall length of the runway is long and the site is open, and the appearance of FOD is random.

[0049] The target size of FOD is usually small, low in structure and close to the ground, and does not have a significant visual effect.

[0050] Limited by the requirements of airport operation and management, FOD detection usually needs to be completed in a short time window, and does not have the conditions for multiple tracking or regional rescan.

[0051] Please refer to Figure 1 , which is a schematic diagram of an exemplary application scenario provided by an embodiment of the present application.

[0052] Figure 1 It is shown that the vehicle-mounted equipment is carried on a mobile detection vehicle on the airport runway, and the vehicle travels at a constant speed or a variable speed along a predetermined path on the airport runway, and real-time target data acquisition is performed by using the forwardly deployed vehicle-mounted equipment.

[0053] Figure 1 It is shown that the vehicle-mounted equipment collects target data and generates corresponding image frames. It is easy to understand that the image features of Frame-t0, Frame-t1 and Frame-t2 in the figure correspond to the features of FOD reflection in three cases respectively, and the specific number of FOD and the feature of FOD reflection in the image are not limited in the present application, but in general, they can be divided into three cases of dot-like, linear tailing and fragmentation, Figure 1 which is only a reference example.

[0054] Figure 1 It is shown that in Frame-t0, the foreign object target first enters the collection coverage area. Since the collection angle is small at this time, the echo of the target is concentrated, and the image presents a dot-like high reflection area.

[0055] Figure 1It is shown that in Frame-t1, the foreign object target is in the middle of the main beam, the collection angle gradually changes, and the target in the image is stretched from a point structure to a linear tail shape, showing obvious deformation.

[0056] Figure 1 It is shown that in Frame-t2, the foreign object target leaves the main beam axis area and enters the multiple scattering area. Due to the irregular structure of the target and the difference in reflection characteristics, it appears as multiple discontinuous bright spots or fragment structures in the image, and each fragment has no obvious connectivity, reflecting the spatial distortion phenomenon.

[0057] In one example, the vehicle-mounted device of the present application can be a forward-looking radar mounted on a patrol vehicle.

[0058] In another example, the vehicle-mounted device of the present application can also be a plurality of radar arrays arranged along the front or both sides of the chassis of the vehicle, which are complementary in direction.

[0059] Although the radar device in the example of the present application itself has a certain target distance and azimuth estimation capability, it can output the approximate spatial position of the target under the current view angle, but due to the characteristics of small size, structure close to the ground and serious reflection distortion of the FOD target, its reflection form in the radar image often presents the phenomena of structure fragmentation, discontinuous intensity and fuzzy boundary. It is difficult to accurately identify the target boundary and complete structure only by the positioning output of the radar.

[0060] As can be appreciated by those skilled in the art, the image generation method in the present application is not limited to radar data reconstruction images, but can also be extended to any image acquisition method that has a view-dependent structure distortion problem in dynamic acquisition.

[0061] It should be noted that the method of the present application is not applicable to the target image acquired by the image sensor. The reason is not only the difference in imaging methods, but also the fact that the target image acquired by the image sensor does not exist in the form of point-like, linear tail and fragmentation. More importantly, in the vehicle-mounted FOD detection of the airport runway, single scanning is the main method, which cannot rely on long-term tracking, and the target size is often extremely small and close to the ground. The image sensor cannot meet the needs of this application scenario.

[0062] It can be understood that the present application is not based on a fixed sensor imaging model, but is aimed at scenarios where there is a scanning path dependence and a nonlinear characteristic of reflection response in the imaging mechanism.

[0063] Next, the airport runway-based vehicle-mounted FOD dynamic detection and positioning method provided by the embodiment of the present application will be introduced in combination with the drawings.

[0064] Figure 2The method shown can be applied to a vehicle-mounted device, including the following S1-S4, and the specific steps are as follows: S1: generating a target image according to target data collected by the vehicle-mounted device, and obtaining a distortion segment of the target image.

[0065] In this embodiment, the target data is derived from a vehicle-mounted device with scanning data, collection frequency, and incident angle sensitivity, preferably a radar device or an equivalent imaging mechanism detection device.

[0066] It can be understood that generating a target image from target data is common knowledge, and this application will not be repeated here.

[0067] Further, due to the limitations of the collection device, the FOD in the target image will appear stretched, trailing, or split into distortion structures, and the distortion segment is a local area collection of such image distortion characteristics.

[0068] It is easy to understand that the prerequisite of this application is that the vehicle-mounted device can accurately detect the FOD, and the difficulty is not in the detection of a single FOD, but in how to determine whether multiple distortion forms are derived from the same FOD target and achieve the structural restoration and accurate determination of the position.

[0069] S2: Construct a dynamic feature tree, initialize the distortion segment as a feature node of the dynamic feature tree, and generate at least one temporary child node according to each feature node in combination with the vehicle motion parameters and scanning view angle information of the vehicle-mounted device during collection.

[0070] In this embodiment, each distortion segment is written into the dynamic feature tree as an independent feature node after calculating the image structure descriptor. The image structure descriptor includes edge direction distribution, pixel position set, and local morphological features.

[0071] Further, the dynamic feature tree allows different distortion segments to establish a structural evolution sub-path, so that originally inexplicable distortion segments have the ability to deduce spatial structures. In combination with the motion direction of the vehicle-mounted device, the current view angle, and the distortion defect trend, the structural extension path or the aggregation convergence path of the feature node is generated under different morphological characteristics.

[0072] S3: Traverse the temporary child nodes, and if there is a pair of child nodes that meet the preset structure closure condition in the temporary child nodes, generate a temporary branch between the corresponding feature nodes.

[0073] In this embodiment, the pair of child nodes that meet the preset structure closure condition are considered to belong to the same FOD in three-dimensional structure, and a temporary branch is constructed between the corresponding feature nodes to generate a closed link.

[0074] S4: generating a FOD detection result according to the temporary branch, and outputting a spatial position thereof.

[0075] In the field of FOD detection, on the one hand, the material and shape of FOD itself are not fixed, and the target reflection characteristics are easily affected by factors such as scanning angle, illumination direction, and radar field edge, resulting in fragmentation, discontinuous structure, or feature distortion in the reflected area in the image. On the other hand, the existing recognition method generally adopts a frame-by-frame processing and independent recognition strategy, lacks the ability to associate and judge the same FOD target in multiple image frames or multiple structural fragments, and is prone to false positives and false negatives.

[0076] The method proposed by the present application models the distorted fragments in each image as feature nodes, and based on motion parameters, scanning angles, and edge defects, generates predictive temporary sub-nodes under the feature nodes to construct possible structural extension paths.

[0077] It can be understood that the present application focuses on the evolution possibility of the structure, simulates the transition logic between point, line and fragmentation, guides the reconstruction process of the target structure, and further identifies the same FOD target in multiple image frames.

[0078] In one example, the specific steps of the present application for obtaining the distorted fragments of the target image are as follows: S1.1: processing the target image by a region growing algorithm, expanding and connecting regions in a local range according to pixel gray value similarity, and generating multiple candidate fragments.

[0079] Specifically, in the airport runway scene, the overall gray value distribution of the background region is relatively smooth, so the present application preliminarily segments the image by the region growing algorithm.

[0080] In this embodiment, the region growing algorithm is based on pixel gray similarity.

[0081] Further, the selected pixel points in the image that meet the initial seed conditions are selected as the starting position, and the seed conditions include but are not limited to brightness intensity exceeding the average background gray threshold and local gradient amplitude being higher than the edge sensitivity threshold.

[0082] Further, the selected seed pixels are gradually expanded and connected in the eight-neighbor direction, and the expansion judgment is based on the set gray similarity threshold and local variance range. When the number of pixels in the expanded region reaches the set lower limit, and the region boundary gradient meets the growth termination condition, the connected domain is determined as a candidate fragment.

[0083] S1.2: extracting the contour edge of each candidate fragment, and constructing a region boundary graph according to the edge density distribution and contour closure degree of the contour edge.

[0084] It can be understood that the edge density distribution reflects the spatial distribution of the number of boundary pixels in a unit area, and the contour closure degree is the ratio of the actual boundary length to the length of the theoretical minimum closed boundary of the region. The extraction of the contour edge can be performed by various edge detection operators, which will not be described herein.

[0085] S1.3: According to the preset judgment condition, the region boundary graph is screened, and the candidate segment screened is taken as the distortion segment.

[0086] In the embodiment, the region boundary graph includes all candidate segments.

[0087] In one example, the judgment condition for screening the region boundary graph includes the following two aspects: in the first aspect, the judgment is performed according to the proportion of the edge direction distribution of a candidate segment existing in the region boundary graph.

[0088] In one case, when the proportion of the edge direction distribution of a candidate segment existing in the region boundary graph is greater than or equal to a second threshold value in the main direction, the candidate segment can be taken as the distortion segment, wherein the second threshold value can be a direction dispersion rate related threshold value, and those skilled in the art can understand that the second threshold value can be obtained by experimental data fitting or historical data statistics.

[0089] It can be understood that when the candidate segment in the image is a real FOD distortion structure, the edge structure thereof usually does not present a regular distribution consistent with the single main direction of the runway background, but is accompanied by a phenomenon of multi-directional mixture and edge texture fracture.

[0090] In another case, when the proportion of the edge direction distribution of a candidate segment existing in the region boundary graph is less than the second threshold value in the main direction, the candidate segment cannot be taken as the distortion segment.

[0091] It can be understood that the introduction of the direction dispersion rate is not an extension of the traditional image feature, but is combined with the real presentation mode and structural distortion mechanism of the FOD target in the radar image, converts the imaging physical characteristics into an image structure identifiable index, and makes the segment judgment have physical consistency.

[0092] In the second aspect, the judgment is performed according to the pixel intensity gradient variance of the boundary of a candidate segment existing in the region boundary graph.

[0093] In one case, when the pixel intensity gradient variance of the boundary of a candidate segment existing in the region boundary graph is greater than or equal to a fourth threshold value, the candidate segment can be taken as the distortion segment, wherein the fourth threshold value can be a gradient dispersion related threshold value, and those skilled in the art can understand that the fourth threshold value can be obtained by experimental data fitting or historical data statistics.

[0094] It can be understood that, since FOD usually has irregular reflection topography, specifically, the pixel gray value at the edge presents nonlinear jump or local sharp fluctuation in the image, at this time, the change of pixel intensity gradient lacks regularity within the boundary range, and the variance value deviates.

[0095] In another case, when the pixel intensity gradient variance of a candidate segment boundary in the region boundary graph is less than a fourth threshold value, the candidate segment boundary cannot be a distortion segment.

[0096] It can be understood that, if the gradient change of the boundary pixels of a certain candidate segment is stable, and the variance value is lower than the set threshold value, it indicates that the region boundary is relatively smooth, and the reflection intensity distribution is uniform, which is usually derived from the background, fixed paving structure or radar measurement redundant area, and can be reasonably excluded.

[0097] Further, the first aspect and the second aspect can be combined to determine whether it is a distortion segment.

[0098] When the first aspect and the second aspect are combined to determine whether it is a distortion segment, if the result based on the first aspect is inconsistent with the result based on the second aspect, the result based on the first aspect is given priority, that is, the judgment priority of the first aspect is higher than that of the second aspect.

[0099] In addition, in addition to the above two aspects, whether it is a distortion segment can also be determined by other manners or in combination with other manners.

[0100] For example, the difference between the boundary line length of a candidate segment and the perimeter of its minimum enclosing rectangle in the region boundary graph is greater than or equal to a first threshold value, wherein the first threshold value can be a proportion-related threshold value, and those skilled in the art can understand that the first threshold value can be obtained by experimental data fitting or historical data statistics.

[0101] For another example, the closure degree of a candidate segment in the region boundary graph is less than or equal to a third threshold value, wherein the third threshold value can be a closure index-related threshold value, and those skilled in the art can understand that the third threshold value can be obtained by experimental data fitting or historical data statistics.

[0102] For example, a dynamic feature tree can be taken as an example, and the following can be referred to for understanding: Figure 3 Figure 3 The initialization structure diagram of the dynamic feature tree of the embodiment of the present application is shown in the following figure: Figure 3 A dynamic feature tree including five feature nodes is shown in the figure, the root node is derived from the initial setting and does not have actual meaning, and the dashed line represents an independent structure branch, wherein the number of the independent structure branch is determined according to the local morphological characteristics of the feature node.

[0103] ​It can be understood that the local morphological features in the present application include three types, namely, point-like, line-like and fragment-like, which correspond to the three distortion conditions described above, respectively.

[0104] Figure 3 It is shown that two feature nodes have two independent structural branches, respectively, and the other three feature nodes have a single independent structural branch. It is easy to understand that the number of independent structural branches in the embodiments of the present application is one to two. It should be noted that the independent structural branch is two only when the local morphological feature of the feature node is line-like.

[0105] It can be understood that, Figure 3 The simplified example diagram is only a simplified example for ease of understanding. The specific dynamic feature tree can further include a larger number of feature nodes. In addition, the dynamic feature tree can further include temporary branch information between the feature nodes, node attribute tables and image structure descriptor mapping relationships. The dynamic feature tree described in the embodiments of the present application is used to more clearly illustrate the technical solutions of the embodiments and does not constitute a limitation on the technical solutions provided by the embodiments of the present application.

[0106] In one example, initializing the distortion segment as a feature node of the dynamic feature tree includes: calculating an image structure descriptor of the distortion segment.

[0107] As can be appreciated by those skilled in the art, the image structure descriptor is used to represent the local structural features of the distortion segment in the image, and its form includes but is not limited to edge direction distribution, pixel position set and local morphological feature. The edge direction distribution can be used to represent the contour trend and structural extension trend of the distortion segment, the pixel position set is used to record the position distribution and topological boundary information of the segment in the image space, and the local morphological feature reflects the point-like, line-like or fragment-like presented by the segment in the vision.

[0108] It can be understood that the calculation of the image structure descriptor belongs to the common method in the field of image analysis and target modeling in the prior art. For example, the edge direction distribution can be calculated by the Canny operator combined with the main direction projection technology, the pixel position set can be calculated by the connected domain marking or contour tracking algorithm, and the local morphological feature can be classified by combining the Hu invariant moment, Zernike moment, circumscribed ellipse morphological ratio, etc. The present application does not repeat the description here.

[0109] It should be noted that the specific composition of the image structure descriptor can be flexibly selected according to the imaging characteristics, sampling accuracy and actual features of the target scene of the deployment device, and the specific mathematical expression form is not limited in the embodiments.

[0110] According to the image structure descriptor, the distorted segment is written into the dynamic feature tree as a feature node, and an independent structure branch is generated under each feature node.

[0111] Specifically, the image structure descriptor, as a carrier of structure information representing the distorted segment, can write the distorted segment into the dynamic feature tree in the form of a node by analyzing its edge direction distribution, pixel position set, and local morphological features.

[0112] It can be understood that each feature node includes a unique node identifier, an image structure descriptor, spatial position information of the node, and a sub-node reference interface related to its structure topological relationship.

[0113] Further, in the structure construction of the dynamic feature tree, the nodes can be inserted into the tree in time sequence or structure priori arrangement, and independent structure branches can be hung under them, or they can be randomly inserted into the tree.

[0114] It should be noted that the feature node writing process can be implemented in the memory structure as a data entity indexed by the image structure descriptor and hung in the corresponding position of the dynamic feature tree. The specific implementation mode can include object-oriented data structure definition, graph structure representation, etc., and the embodiments do not limit the implementation mode.

[0115] In one example, the specific steps of S2 are as follows: S2.1: generating an edge defect description according to the open contour segment of the edge structure end of the feature node.

[0116] Specifically, the open contour segment can be understood as a boundary trend with obvious discontinuity or unclosed contour in the image structure, in other words, there can be a target extended form or a target aggregated form that has not been imaged.

[0117] Further, the edge defect description includes but is not limited to: main reflection direction, used to depict the dominant propagation direction of the reflection signal of the current open contour segment, usually along the physical extension trend of the target structure, to infer the deformation path of the target in the subsequent frame.

[0118] Edge direction consistency, used to reflect the continuity degree of the edge trend near the open segment, which helps to judge whether the current structure has sufficient linearity, and then infer the possible extension form it may maintain subsequently.

[0119] Pixel centroid, used to estimate the geometric center position of the region, thereby providing a starting reference point when performing position offset deduction, calculating the position of the extended path and the target aggregated region.

[0120] In the embodiment, the end of the edge structure in the feature node image can be extracted by the Canny edge detection algorithm, and the open contour segment is located by combining the contour tracking and the breaking analysis algorithm, and the edge defect description is generated by the gradient direction statistics of the broken boundary, the edge distribution fitting and the centroid solving, which will not be described herein.

[0121] S2.2: According to the edge defect description, a temporary sub-node of the starting of the independent structure branch is generated in combination with the vehicle motion parameters and the scanning view angle information.

[0122] In the embodiment, according to the edge defect description generated in S2.1, the vehicle motion parameters of the vehicle-mounted device when collecting the image frame are combined, including the speed, acceleration and collection time interval in the embodiment, and the scanning view angle information of the radar, including the pitch angle, azimuth angle and incident angle, the extension trend and spatial offset distance of the target structure are calculated.

[0123] Further, a prediction area window is set on the extension trend, a group of expected structure features are constructed based on the image structure descriptor, the possible presentation form of the target in the non-imaged area is simulated, and the corresponding derived structure descriptor is generated, the temporary sub-node of the starting of the independent structure branch is generated, and the first node of the structure extension chain is hung under the feature node.

[0124] In one example, the temporary sub-node of the starting of the independent structure branch is generated, including: generating a derived structure descriptor according to the image structure descriptor of the feature node and the corresponding local morphological feature in the image structure descriptor.

[0125] If the local morphological feature of the feature node is point-like, the structure extension trend is calculated according to the main reflection direction in the edge defect description and the vehicle motion parameters, the pixel feature extension area at the corresponding position is generated at the preset offset distance along the structure extension trend, and the reflection feature and structure boundary of the feature node are simulated in the pixel feature extension area through the edge direction distribution of the feature node to generate the derived structure descriptor.

[0126] Specifically, for the feature node with point-like local morphological feature, since its reflection feature usually shows strong echo but lacks edge continuity and directionality in the image, it indicates that the corresponding physical structure only partially enters the main beam of the radar, or only forms effective echo at a narrow angle, so the point-like structure is often the initial reaction of the foreign matter entering the detection range. To infer the possible extension direction, based on the main reflection direction in the edge defect description and the motion parameters such as the current speed direction, heading angle and pitch angle of the vehicle, the structure extension trend can be obtained by the vector superposition model.

[0127] Further, a spatial offset distance is applied in the trend direction, and in the embodiments of the present application, the spatial offset distance can be estimated in combination with the radar frame rate and the vehicle forward distance, a region is demarcated as a pixel feature extension region, and in the region, the direction distribution of the point feature original edge is used for affine transformation simulation to generate new edge segments and a reflection structure model, so as to obtain a derived structure descriptor. The descriptor is hung at the starting sub-node position of the dynamic feature tree as a structure expansion starting point.

[0128] If the local morphological feature of the feature node is linear, the derived structure descriptor corresponding to the pixel feature aggregation region and the derived structure descriptor corresponding to the pixel feature extension region are generated, specifically including: calculating the two-way structure extension trend according to the edge direction consistency in the edge defect description and the scanning angle information, and generating the pixel feature extension region and the pixel feature aggregation region according to the vehicle motion parameters in each structure extension trend direction.

[0129] In the pixel feature aggregation region, the reflection feature and the structure boundary of the feature node are simulated through the edge closure degree of the feature node, and the derived structure descriptor corresponding to the pixel feature aggregation region is generated.

[0130] In the pixel feature extension region, the reflection feature and the structure boundary of the feature node are simulated through the edge direction distribution of the feature node, and the derived structure descriptor corresponding to the pixel feature extension region is generated.

[0131] Specifically, for the feature node with a local morphological feature of a line, the image thereof is a trailing structure with a clear extension direction, and such a structure is usually in the main center of the radar beam or in the middle frame during the crossing. Since the edge direction consistency is high and the reflection band has significant directionality, it is speculated that the structure extension trend needs to consider the possibility of two-way development. Therefore, the possible extension trend directions on both sides of the current structure need to be obtained by comprehensively analyzing the edge direction consistency index obtained through the edge defect description and the scanning angle.

[0132] Further, the forward pixel feature extension region and the reverse pixel feature aggregation region are generated according to the dynamic parameters of the vehicle in the two trend directions respectively, wherein the forward pixel feature extension region is used to simulate the structure extension segment that has not been imaged, and the aggregation region is used to test whether there is an edge re-closing trend. In feature simulation, the image is reconstructed through linear expansion of the original edge direction distribution in the extension region; the aggregation region is constructed with a high-density boundary grid taking the edge closure degree as the core index to fit the possible aggregation morphology of the target end, thereby generating two derived structure descriptors, and two starting sub-nodes are hung on two independent structure branches of the feature node respectively to realize two-way prediction of the dynamic structure.

[0133] If the local morphological feature of the feature node is fragmented, a convergence vector field is calculated according to the pixel centroid in the edge defect description, the vehicle motion parameters and the scanning angle information, a pixel feature aggregation area is generated along the central tendency direction of the convergence vector field, and the reflection feature and the structural boundary of the feature node are simulated in the pixel feature aggregation area according to the edge closure degree of the feature node to generate a derived structural descriptor.

[0134] Specifically, for the feature node with a local morphological feature of fragmentation, it is usually represented by multiple discontinuous high-reflection fragments in the image, and is often in a stage where the target is about to leave the main field of view of the radar or enter the multipath reflection area. The structure is highly discrete and lacks significant directional tendency. Therefore, in such a structure, the target extension path cannot be effectively predicted by only using the traditional directional vector.

[0135] In the embodiment, the pixel response direction field around the pixel centroid in the edge defect description is calculated by extracting the pixel centroid information, combined with the scanning angle and the vehicle motion parameters. On this basis, a convergence vector field modeling mechanism is introduced, and a field theory model based on the minimum energy of the gradient is used to calculate the convergent tendency of multiple fragmented pixel responses, and further estimate the possible aggregation center of the structure. A pixel feature aggregation area is constructed in the central tendency direction, and a new structural description model is constructed in the area combined with the edge closure degree index to simulate the possible compression aggregation state of the target under the current viewing angle, generate a derived structural descriptor and generate a unique starting sub-node.

[0136] It is easy to understand that in different morphological features, the present application specifically includes two pixel feature simulation strategies, specifically including a pixel feature extension area and a pixel feature aggregation area, wherein the pixel feature extension area includes point-shaped evolution into line-shaped and fragmented, and also includes line-shaped evolution into fragmented, and the pixel feature aggregation area includes fragmented evolution into line-shaped and point-shaped, and also includes line-shaped evolution into point-shaped.

[0137] As can be appreciated by those skilled in the art, the two pixel feature simulation strategies provided in the present application are not based on simple classification of the structure of the static image itself, but are modeled for the structural evolution trend of the target in the radar imaging process under different distortion morphologies, to predict the potential structural state of the target in the missing imaging area. In other words, the pixel feature extension area focuses on depicting the spatial extension direction of the target morphology under the current scanning angle and motion parameters, and is suitable for the case where the structure has not been completely imaged or there is partial occlusion; and the pixel feature aggregation area emphasizes more on the integration of discrete or fragmented echo features, simulates the aggregated boundary that the target may form under complex scattering conditions, and is used to improve the structural correlation ability of the edge fragments.

[0138] In the specific application of the present application, the evolution from a point shape to a line shape or a fragment shape is often seen in the state that the foreign object first enters the radar beam but has not completely entered the main field of view, at which time generating an extension region for simulation is helpful to build the initial boundary of the complete structure; while the evolution from a line shape to a fragment shape often appears in the disintegration process of the target after leaving the main beam center area, through the multi-directional expansion of the extension region, the structure fracture path can be simulated in advance. Conversely, if the target currently shows a fragment shape, but there is a certain aggregation trend between multiple fragments (such as consistent direction, continuous edge, and concentrated centroid), through the generation of an aggregation region for convergence simulation, the original line shape or point shape structure can be effectively reconstructed, thereby improving the continuity of structure identification.

[0139] Taking the evolution of a line shape as an example, it can be understood with reference to Figure 4 Figure 4 is a schematic diagram of the principle of generating corresponding independent structure branch starting sub-nodes for a feature node in a line shape according to an embodiment of the present application, Figure 4 shows that the feature node is a structure node initially constructed in a dynamic feature tree, and the image structure descriptor indicates that its shape is a line shape. In combination with the edge defect description of the feature node, vehicle motion parameters, and scanning angle information, the structure extension trend is deduced at both ends, thereby generating two independent structure branches in different directions.

[0140] Figure 4 shows two independent structure branches. The first branch simulates the extension trend formed by further trailing evolution of the target, constructs a pixel feature extension region in the extension direction, and simulates the edge features and reflection shape of the structure in the region, generating a temporary sub-node on the right as a starting sub-node. The second branch simulates the structure aggregation trend of the target due to reflection weakening, angle change, etc., generates a pixel feature aggregation region, and performs edge closing simulation in the region, generating a temporary sub-node on the left as a starting sub-node.

[0141] According to the derived structure descriptor, the starting temporary sub-node of the independent structure branch is initialized.

[0142] S2.3: Perform an iteration operation, taking the temporary sub-node at the end of the independent structure branch as input, generating the next temporary sub-node according to the input edge defect description in combination with the vehicle motion parameters and scanning angle information, and hanging it to the temporary sub-node at the end, until the next temporary sub-node is structure closed, and the iteration is terminated.

[0143] ​In this embodiment, after the initial temporary sub-node is generated, the derived structure description sub-content is based on it, and it is continuously judged whether there is a new open contour segment, if there is, the process of S2.1 and S2.2 is repeated, the next temporary sub-node is continuously generated and is hung to the branch. Each iteration will re-collect the current attitude, displacement and view angle change of the vehicle and other parameters, dynamically correct the predicted offset path, so as to maintain the spatial consistency of the structure extension prediction. When the structure simulation of a temporary sub-node reaches the preset closed threshold, it is considered that the structure is closed, and the continuous growth of the branch is terminated.

[0144] For example, the temporary branch can be understood by referring to Figure 5 Figure 5 The schematic diagram for generating the temporary branch of the embodiment of the present application is shown in Figure 5 Figure 3 Based on

[0145] It can be understood that Figure 5 Three groups of temporary sub-node pairs are shown in the figure, and it is determined by structure similarity analysis at the end of the respective structure branch that they meet the preset structure closure condition, so as to establish a temporary branch between the respective feature nodes, and realize the structure merging between the FOD distortion segments.

[0146] Those skilled in the art can understand how to judge whether there is a mapping relationship of structure direction complementarity, edge form compatibility and reflection feature reducibility, which can be completed by implementing a structure feature matching algorithm, for example, using direction consistency calculation based on edge direction cosine similarity, form matching evaluation based on boundary contour Hausdorff distance, and reflection feature reconstruction measurement based on gray distribution similarity or radar reflection intensity model. The above technical methods have relatively mature public implementation in the field of image structure analysis and target reconstruction, and can adapt to the needs of structure association judgment between temporary sub-nodes in the present application. The present application will not be repeated here.

[0147] In one example, the temporary branch generated between the corresponding feature nodes includes: S3.1: extracting temporary sub-nodes in the independent structure branch of different feature nodes.

[0148] S3.2: For any two temporary sub-nodes under different independent structure branches, according to the derived structure description sub, it is judged whether there is a mapping relationship of structure direction complementarity, edge form compatibility and reflection feature reducibility.

[0149] S3.3: If the mapping relationship exists, a temporary branch is established to connect the corresponding feature nodes.

[0150] ​​It can be understood that when all feature nodes are connected or not connected by temporary branches, there may be Figure 6 the case shown, Figure 6 It is a schematic diagram of the temporary branch unclosed principle of the embodiment of the application, Figure 6 It is shown that feature node A and feature node B establish a temporary branch, and feature node B and feature node C establish a temporary branch, but there is no temporary branch between feature node A and feature node C.

[0151] Figure 6 The case shown is a special case in the principle of the application. The application defaults that the feature nodes establishing temporary branches belong to the same FOD with a high probability, that is, the temporary branches between the feature nodes have and only have a closed loop relationship in the connection relationship.

[0152] And Figure 6 The case shown is that due to abnormal structure inference of a temporary sub-node generated by a certain feature node, or introduction of structure direction drift, edge fragment error merging or reflection feature recognition deviation in the matching process, it establishes temporary branches with other two feature nodes respectively, but cannot form a closed loop structure connection among the three. More specifically, this non-closed loop connection relationship indicates that although B node has structure matching conditions with A and C nodes respectively, A and C lack a common derived structure relationship that can be mapped to each other, indicating that the three do not have a unified spatial structure extension path, and thus cannot be reasonably merged into the same FOD.

[0153] It can be understood that in the structure reasoning logic of the application, temporary branches are not only used for bilateral matching confirmation between feature nodes, but also play a role in verifying the closure of the loop at the structure level. If a certain temporary branch chain cannot form a closed structure, the feature node set connected by the temporary branch chain may have abnormal judgment, which needs to be further screened, pruned or rolled back by strategies such as confidence calculation and connected chain quality evaluation, which specifically includes: constructing a structure connection graph according to the temporary branches established by the feature nodes through temporary sub-nodes.

[0154] Determine whether the structure connection graph is completely closed.

[0155] If it is not completely closed, extract a sub-chain set constituting a closed structure in the structure connection graph.

[0156] In the sub-chain set, select the closed sub-chain with the largest number of feature nodes, retain the corresponding temporary branch, and prune other temporary branches.

[0157] If there are multiple closed sub-chains with the same number of feature nodes, the temporary branches are determined according to the matching confidence of the independent structure branches corresponding to the closed sub-chains.

[0158] The specific steps of S4 are as follows: S4.1: generating FOD detection information according to the feature nodes of the temporary branch connection.

[0159] S4.2: calculating the spatial position of the corresponding feature node in the world coordinate system according to the vehicle motion parameters and the scanning view angle information corresponding to each feature node.

[0160] S4.3: multi-point fitting the spatial position to generate spatial information.

[0161] S4.4: displaying the FOD detection information and the spatial information in the display terminal.

[0162] Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those of ordinary skill in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A vehicle-mounted FOD dynamic detection and positioning method based on an airport runway, applied to a vehicle-mounted device, is characterized in that: The vehicle-mounted FOD dynamic detection and positioning method includes: generating a target image based on the target data collected by the vehicle-mounted device, and obtaining a distorted segment of the target image; A dynamic feature tree is constructed, the distorted fragment is initialized as a feature node of the dynamic feature tree, and independent structural branches are generated under each feature node, wherein the number of the independent structural branches is determined according to the local morphological features of the feature node, and at least one temporary child node is generated according to each feature node, wherein the temporary child node is used to represent a structural portion of the target that is not imaged, and the generation of the temporary child node includes: generating an edge incomplete description according to an open contour segment at an end of the edge structure of the characteristic node; According to the edge defect description, in combination with the vehicle motion parameters and scanning angle information of the vehicle-mounted device during acquisition, a temporary sub-node at the start of the independent structure branch is generated, including: A derived structure descriptor is generated based on the image structure descriptor of the feature node and the corresponding local morphological features in the image structure descriptor, including: if the local morphological feature of the feature node is point-like, the structure extension trend is calculated based on the main reflection direction and vehicle motion parameters in the edge incomplete description, and a pixel feature extension area of ​​the corresponding position is generated at a preset offset distance along the structure extension trend, and the reflection features and structural boundaries of the feature node are simulated by the edge direction distribution of the feature node within the pixel feature extension area to generate a derived structure descriptor; if the local morphological feature of the feature node is linear, a derived structure descriptor corresponding to the pixel feature aggregation area and a derived structure descriptor corresponding to the pixel feature extension area are generated, specifically including: calculating the bidirectional structure extension trend based on the edge direction consistency and scanning perspective information in the edge incomplete description, and generating a derived structure descriptor in each structure extension trend direction. Upward, a pixel feature extension region and a pixel feature aggregation region are generated respectively according to the vehicle motion parameters; within the pixel feature aggregation region, the reflection features and structural boundaries of the feature nodes are simulated by the edge closure of the feature nodes, and a derived structure descriptor corresponding to the pixel feature aggregation region is generated; within the pixel feature extension region, the reflection features and structural boundaries of the feature nodes are simulated by the edge direction distribution of the feature nodes, and a derived structure descriptor corresponding to the pixel feature extension region is generated; if the local morphological features of the feature nodes are fragmented, a convergence vector field is calculated according to the pixel centroid, vehicle motion parameters and scanning view information in the edge incomplete description, and a pixel feature aggregation region is generated along the central trend direction of the convergence vector field. Within the pixel feature aggregation region, the reflection features and structural boundaries of the feature nodes are simulated by the edge closure of the feature nodes, and a derived structure descriptor is generated; Initializing a temporary child node at the start of an independent structure branch according to the derived structure descriptor; Perform an iterative operation, taking the temporary child node at the end of the independent structure branch as input. Based on the input edge fragment description, combined with the vehicle motion parameters and scanning view information, generate the next temporary child node and attach it to the temporary child node at the end until the next temporary child node structure is closed, terminating the iteration. Traversing the temporary child nodes, if there is a child node pair that meets the preset structural closure condition in the temporary child nodes, generating a temporary branch between the corresponding feature nodes; Based on the temporary branches, a FOD detection result is generated and its spatial position is output.

2. The vehicle-mounted FOD dynamic detection and positioning method based on an airport runway according to claim 1 is characterized in that: Initializing the distorted segment as a feature node of the dynamic feature tree includes: Calculating an image structure descriptor of the distorted segment, wherein the image structure descriptor includes an edge direction distribution, a pixel position set, and a local morphological feature of the corresponding distorted segment; According to the image structure descriptor, the distorted segment is written into a dynamic feature tree as a feature node.

3. The vehicle-mounted FOD dynamic detection and positioning method based on an airport runway according to claim 2 is characterized in that: Generating temporary branches between corresponding feature nodes includes: Extract temporary sub-nodes in independent structural branches of different feature nodes; For any two temporary sub-nodes under different independent structural branches, determine whether there is a mapping relationship with complementary structural directions and compatible edge morphologies based on the derived structural descriptors; If the mapping relationship exists, a temporary branch is created to connect the corresponding feature nodes.

4. The vehicle-mounted FOD dynamic detection and positioning method based on an airport runway according to claim 3 is characterized in that: The method further comprises: Construct a structural connection graph based on temporary branches established between feature nodes through temporary child nodes; Determining whether the structural connection diagram is completely closed; If the complete closure is not satisfied, a subchain set constituting a closed structure is extracted from the structure connection graph; In the subchain set, the closed subchain with the largest number of characteristic nodes is selected, the corresponding temporary branches are retained, and the other temporary branches are pruned; If there are multiple closed subchains with the same number of feature nodes, they are screened according to the matching confidence of the independent structural branches corresponding to the closed subchains to determine the temporary branches.

5. The vehicle-mounted FOD dynamic detection and positioning method based on an airport runway according to claim 1 is characterized in that: Obtaining a distorted segment of the target image, including: Processing the target image using a region growing algorithm, expanding the connected region in a local range according to the similarity of pixel grayscale values, and generating multiple candidate segments; Extracting contour edges for each candidate segment, and constructing a region boundary map based on edge density distribution and contour closure of the contour edges; The region boundary map is screened according to a preset judgment condition, and candidate segments that pass the screening are used as distorted segments.

6. The vehicle-mounted FOD dynamic detection and positioning method based on an airport runway according to claim 5 is characterized in that: The judgment condition for screening the region boundary map includes one of the following: The difference between the length of the boundary line of a candidate segment in the region boundary map and the perimeter of its minimum circumscribed rectangle is greater than or equal to a first threshold ratio difference; In the region boundary map, there is a candidate segment whose edge direction distribution outside the main direction has a ratio greater than or equal to a second threshold directional discrete rate; In the region boundary map, there is a candidate segment whose boundary closure degree is less than or equal to a third threshold closure index; In the region boundary map, there is a candidate segment boundary whose pixel intensity gradient variance is greater than or equal to a fourth threshold gradient discrete value.

7. The vehicle-mounted FOD dynamic detection and positioning method based on an airport runway according to claim 1 is characterized in that: Based on the temporary branches, generate FOD detection results and output their spatial positions, including: Generate FOD detection information based on the characteristic nodes connected by temporary branches; According to the vehicle motion parameters and scanning angle information corresponding to each feature node, the spatial position of the corresponding feature node in the world coordinate system is calculated; The spatial positions are subjected to multi-point fitting to generate spatial information.

Images