Method and system for positioning head of driver in aircraft cockpit

Abstract

The invention provides a method and system for positioning the head of a driver in an aircraft cockpit, and relates to the technical field of image processing, and the method comprises the steps: obtaining the head images of the driver in the aircraft cockpit at a plurality of visual angles; extracting a plurality of feature points in the head image of the driver through an FAST algorithm; calculating a Harris response value of each feature point through a Harris corner response function; through a non-maximum suppression algorithm, a feature point with the maximum Harris response value is screened out from the driver head image to serve as a target feature point; calculating the main direction of the target feature point through a centroid method; generating a BRIEF descriptor of the target feature point according to the main direction of the target feature point; and according to the BRIEF descriptors of the target feature points, feature point matching is performed on the target feature points in the head image of the driver under each view angle through an FLANN algorithm, so that the head of the driver in an aircraft cockpit can be positioned.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a method and system for positioning a pilot's head in an aircraft cockpit. Background Art

[0002] As the degree of automation in aircraft cockpits continues to increase, the interaction between pilots and automated systems has become increasingly critical. The driver's head posture information is not only crucial to flight safety, but is also widely used in key systems such as attitude detection, navigation assistance, and target recognition. With the complexity of modern flight missions, the demand for real-time perception of the driver's head posture by automated systems is growing. By accurately and reliably acquiring the driver's head posture information, the automated system can better judge the pilot's attention, posture changes, and even make intelligent auxiliary decisions, which is directly related to flight safety, operational efficiency, and mission success rate. Therefore, real-time and accurate tracking and positioning of the driver's head posture to ensure that the system can adapt to various complex cockpit environments has become an important task in modern aviation technology.

[0003] However, existing technologies have difficulty maintaining the robustness of feature point detection when dealing with frequently changing lighting conditions and complex background interference in the cockpit. It is difficult to ensure the stability of feature point descriptors when dealing with driver head rotation. The use of traditional Euclidean distance matching methods under noise and partial occlusion conditions leads to a decrease in matching accuracy.

[0004] In addition, the real-time processing capabilities of existing technologies are limited. In complex scenarios with high-density feature points, the response speed is insufficient and the driver's head posture data cannot be fed back in a timely manner. Summary of the invention

[0005] In order to solve the technical problems that the traditional method for locating the driver's head in the cockpit of an aircraft is difficult to maintain the robustness of feature point detection when dealing with frequently changing lighting conditions and complex background interference in the cockpit, it is difficult to ensure the stability of the feature point descriptor when processing the rotation of the driver's head, the use of the traditional Euclidean distance matching method under noise and partial occlusion conditions leads to a decrease in matching accuracy and limited real-time processing capability, and the response speed is insufficient in complex scenes with high-density feature points, and the driver's head posture data cannot be fed back in time, the present invention provides a method and system for locating the driver's head in the cockpit of an aircraft.

[0006] The technical solution provided by the embodiment of the present invention is as follows:

[0007] First aspect:

[0008] An embodiment of the present invention provides a method for positioning a pilot's head in an aircraft cockpit, comprising:

[0009] S1: Acquire the pilot's head image in the aircraft cockpit from multiple perspectives;

[0010] S2: extracting multiple feature points from the driver's head image using the FAST algorithm;

[0011] S3: Calculating the Harris response value of each of the feature points by using the Harris corner point response function;

[0012] S4: Using a non-maximum suppression algorithm, a feature point with a maximum Harris response value is selected from the driver's head image as a target feature point;

[0013] S5: Calculate the main direction of the target feature point by using the centroid method;

[0014] S6: generating a BRIEF descriptor of the target feature point according to the main direction of the target feature point;

[0015] S7: According to the BRIEF descriptor of the target feature point, the target feature point in the driver's head image at each viewing angle is matched by the FLANN algorithm to locate the driver's head in the cockpit of the aircraft.

[0016] Second aspect:

[0017] An embodiment of the present invention provides a pilot head positioning system in an aircraft cockpit, comprising: a memory and one or more processors;

[0018] One or more application programs are stored in the memory, and the one or more application programs are suitable for being executed by the one or more processors to implement the above-mentioned method for positioning the pilot's head in the cockpit of an aircraft.

[0019] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0020] In the present invention, a plurality of feature points in the driver's head image are extracted by the FAST algorithm, the Harris corner point response function is used to calculate the Harris response value of each feature point, and the feature point with the maximum Harris response value is screened out in the driver's head image as the target feature point by the non-maximum suppression algorithm. The robustness of feature point detection can be always maintained when dealing with frequently changing lighting conditions and complex background interference in the cockpit. The main direction of the target feature point is calculated by the centroid method. According to the main direction of the target feature point, the BRIEF descriptor of the target feature point is generated. When processing the rotation of the driver's head, the stability of the feature point descriptor can be ensured. According to the BRIEF descriptor of the target feature point, the FLANN algorithm is used to match the target feature points in the driver's head image under various viewing angles, so as to avoid using the traditional Euclidean distance matching method under noise and local occlusion conditions, thereby improving the matching accuracy, having strong real-time processing capability, and having a fast response speed in complex scenes with high-density feature points, so as to be able to timely feedback the driver's head posture data. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A schematic flow chart of a method for locating a pilot's head in an aircraft cockpit provided by an embodiment of the present invention;

[0023] Figure 2 A schematic structural diagram of a pilot head positioning system in an aircraft cockpit provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0025] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0026] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0027] Reference Manual Attached Figure 1 , shows a schematic flow chart of a method for locating a pilot's head in an aircraft cockpit provided by an embodiment of the present invention.

[0028] An embodiment of the present invention provides a method for locating a pilot's head in an aircraft cockpit. The method can be implemented by a pilot's head locating device in the aircraft cockpit. The pilot's head locating device in the aircraft cockpit can be a terminal or a server. The processing flow of the method for locating a pilot's head in an aircraft cockpit can include the following steps:

[0029] S1: Acquire the pilot's head image in the aircraft cockpit from multiple perspectives.

[0030] In the present invention, the head image of the pilot in the aircraft cockpit is obtained from multiple perspectives, which can improve the accuracy and robustness of feature point detection and recognition. Different perspectives provide richer information, which helps to reduce the impact of occlusion and lighting changes, thereby enhancing the pilot's positioning and recognition capabilities. In addition, multi-perspective data can also support more complex three-dimensional reconstruction and analysis, improving the performance and reliability of the overall system.

[0031] S2: Extract multiple feature points from the driver's head image using the FAST algorithm.

[0032] Among them, the FAST (Features from Accelerated Segment Test) algorithm is an efficient feature point detection algorithm used to extract pre-feature points from an image. It identifies feature points by quickly evaluating the brightness changes around pixels. Specifically, it selects a candidate point and detects whether the brightness values ​​of the pixels around it are significantly different from the central pixel. If a certain number (usually 12) of the brightness values ​​of the surrounding pixels change by more than a set threshold, the point is marked as a feature point. The FAST algorithm is widely used because of its fast speed and ability to work efficiently in real-time applications, especially in environments with limited computing resources.

[0033] In the present invention, the FAST algorithm can quickly detect feature points and is suitable for real-time applications, especially in dynamic environments such as cockpits. The algorithm has low requirements for computing resources and is suitable for embedded systems with limited resources. It can complete feature extraction without affecting performance. Due to its characteristics based on brightness changes, the FAST algorithm can still effectively extract feature points under different lighting conditions, improving the stability of the system in complex environments. Accurate feature point extraction provides a reliable basis for subsequent image matching and head positioning, and enhances the accuracy and reliability of overall positioning.

[0034] In a possible implementation, S2 specifically includes sub-steps S201 and S202:

[0035] S201: Optimize the FAST algorithm through adaptive threshold adjustment:

[0036] t=μ I +α·σ I

[0037] Where t represents the brightness threshold after adaptive threshold adjustment, μ I represents the average brightness value of the pixels in the local area of ​​the driver's head image, α represents the adjustment coefficient, and σ I Represents the brightness standard deviation of pixels in the local area of ​​the driver's head image.

[0038] It should be noted that those skilled in the art can set the brightness threshold according to actual needs, and the present invention is not limited thereto.

[0039] In the present invention, the adaptive threshold is dynamically adjusted according to the brightness characteristics of the local area, so that the algorithm can adapt to different lighting conditions and image features, and improve the accuracy of feature point extraction. By introducing the average brightness value and standard deviation, the interference caused by noise and lighting changes can be effectively filtered out, ensuring that the extracted feature points are more stable and reliable. Adaptive adjustment can reduce false detection and missed detection, making the algorithm perform better in complex scenes and improving the performance of overall feature detection.

[0040] S202: Extract multiple feature points from the driver's head image using the optimized FAST algorithm:

[0041] |I(p i )-I(p)|>t

[0042] Among them, I(p i ) represents the brightness value of the i-th pixel in the neighborhood of the preset candidate feature point p, I(p) represents the brightness value of the preset candidate feature point p, and p represents the preset candidate feature point in the driver's head image.

[0043] For example, a candidate feature point is preset in the driver's head image, and a circular neighborhood with a radius of 3 is selected with the preset candidate feature point as the center. The circular neighborhood contains 16 pixels, i.e., i=16. When extracting feature points through the optimized FAST algorithm, the brightness value I(p i ) is compared with the brightness value I(p) of the central pixel, i.e., the preset candidate feature point. If there are at least 12 consecutive pixels with brightness values ​​I(p) in the circular neighborhood, i ) and the brightness value I(p) of the central pixel, i.e., the preset candidate feature point, exceeds the brightness threshold t after the set adaptive threshold is adjusted, then the pixel is marked as a feature point.

[0044] In the present invention, by comparing only the limited pixels around the preset candidate feature points, the feature points are quickly determined, the amount of calculation is reduced, the processing speed is improved, and it is suitable for real-time applications. The optimized algorithm can work effectively in complex scenes, especially in cockpit environments with large lighting changes, which enhances the robustness of the system. The adaptive threshold is used to compare the brightness difference, so that the algorithm can more accurately identify significant feature points and reduce the false detection rate. The extraction of high-quality feature points lays a solid foundation for subsequent image matching and analysis, thereby improving the positioning and recognition performance of the entire system.

[0045] S3: Calculate the Harris response value of each feature point through the Harris corner point response function.

[0046] Among them, the Harris corner response function is an algorithm for detecting corners in an image. It evaluates the significance of feature points by analyzing the gradient information of the local area of ​​the image. Specifically, the algorithm calculates a combination of the determinant and trace of a Hessian matrix to form a response value R. The value of R can characterize the feature strength of the image at that point: a high value indicates a corner, and a low value indicates an edge or a flat area. The advantage of the Harris response function is that it is robust to noise and can provide a good corner location effect. It is widely used in image processing and computer vision tasks.

[0047] In the present invention, the Harris corner response function can effectively identify corners in an image by analyzing local gradient information. These corners are usually the best candidates for feature points and provide a strong feature description. The algorithm has good robustness to noise and can resist the influence of image noise and illumination changes to a certain extent, thereby improving the stability of feature point extraction. The Harris response value can directly reflect the significance of the feature point. A high response value indicates that the point has a strong feature, which is convenient for subsequent screening and processing.

[0048] In a possible implementation, S3 is specifically:

[0049] According to the following formula, the Harris response value of each feature point is calculated through the Harris corner response function:

[0050] R = det(H) - k (trace(H)) 2

[0051]

[0052] Where R represents the Harris response value of the feature point, det(H) represents the determinant of the Hessian matrix H, H represents the Hessian matrix of the driver's head image, k represents a constant, trace(H) represents the trace of the Hessian matrix H, and Lxx represents the second-order derivative of the driver's head image in the x direction, L xy represents the mixed second-order derivative of the driver's head image in the x and y directions, L yy Represents the second-order derivative of the driver's head image in the y direction.

[0053] In the present invention, the combination of the determinant and the trace of the Hessian matrix can effectively capture the gradient changes in the local area of ​​the image, making the significance evaluation of the feature points more accurate. By calculating the determinant and the trace, the Harris response value can suppress the influence of noise to a certain extent and improve the stability and reliability of the feature points. The introduction of the constant k makes the algorithm flexible and can be adjusted according to the specific application scenario to optimize the effect of pre-feature point detection, so as to adapt to the needs of different environments. A high response value R corresponds to a strong feature point. This clear feature strength calibration helps to screen out the most important feature points in the subsequent processing steps and improve the efficiency of feature matching.

[0054] S4: Through the non-maximum suppression algorithm, the feature points with the maximum Harris response value are screened out in the driver's head image as target feature points.

[0055] Among them, the non-maximum suppression algorithm is a technique for extracting local maxima, which is often used for feature point detection in image processing. Its main goal is to filter out the most significant points from the detected features. Specifically, the algorithm compares the response values ​​in the neighborhood of the feature point, retains only those points with higher response values ​​than the surrounding points, and suppresses other relatively low response values. This process helps to eliminate redundant features, making the final extracted feature points more accurate and have stronger descriptive capabilities, and is often used in algorithms such as Harris corner detection.

[0056] In the present invention, the non-maximum suppression algorithm can effectively eliminate the redundancy of multiple adjacent feature points, making the final selected feature points more unique and avoiding unnecessary complexity caused by repeated feature points. By retaining only the point with the maximum response value in the neighborhood, it is ensured that the selected feature point has strong significance and stability in the image, thereby enhancing the description ability of the feature point.

[0057] S5: Calculate the main direction of the target feature point by the centroid method.

[0058] Among them, the centroid method is a method for calculating the center point (center of mass) of a region in an image, which is usually used for direction calculation of feature points or regional analysis. In image processing, the centroid method determines the position of the center of mass by calculating the weighted average position of pixels in a specific area, and the weight is usually the brightness value or other characteristic value of the pixel. This method can effectively find the center of gravity of the target area, help extract shape features or perform subsequent image analysis tasks. The centroid method is simple and intuitive, and is widely used in morphological processing, target tracking, pattern recognition and other fields.

[0059] In the present invention, the centroid method can effectively identify the center of gravity of feature points, and then accurately calculate their main direction, thereby improving the description ability of feature points. By considering the weighted average of neighborhood pixels, the centroid method has a certain robustness to noise and illumination changes, and can provide stable direction estimation. Through accurate main direction estimation, a good foundation is provided for subsequent feature matching and image registration, improving the efficiency of the overall system.

[0060] In a possible implementation, S5 specifically includes sub-steps S501 and S502:

[0061] S501: Calculate the horizontal gradient and vertical gradient of the neighborhood of the target feature point respectively:

[0062] G x (x,y)=I(x+1,y)-I(x-1,y)

[0063] G y (x,y)=I(x,y+1)-I(x,y-1)

[0064] Among them, G x (x, y) represents the horizontal gradient of the pixel point (x, y) in the neighborhood of the target feature point, I(x+1, y) represents the brightness value of the pixel point (x+1, y) in the neighborhood of the target feature point, I(x-1, y) represents the brightness value of the pixel point (x-1, y) in the neighborhood of the target feature point, G y (x,y) represents the vertical gradient of the pixel point (x,y) in the neighborhood of the target feature point, I(x,y+1) represents the brightness value of the pixel point (x,y+1) in the neighborhood of the target feature point, I(x,y-1) represents the brightness value of the pixel point (x,y-1) in the neighborhood of the target feature point, x represents the horizontal coordinate of the pixel point, and y represents the vertical coordinate of the pixel point.

[0065] S502: Taking the brightness values ​​of the pixels in the neighborhood of the target feature point as weights, weighted summing is performed on the horizontal gradient and the vertical gradient in the neighborhood of the target feature point to calculate the main direction of the target feature point:

[0066]

[0067] Among them, θ represents the main direction of the target feature point, atan2() represents the two-parameter inverse tangent function, and I(x,y) represents the brightness value of the pixel point (x,y) in the neighborhood of the target feature point.

[0068] In the present invention, the horizontal and vertical gradients are calculated separately, so that the calculation of the main direction is more accurate and can reflect the true direction of the feature point. Using the brightness value as a weight helps to highlight the influence of important areas, enhance the reliability of direction estimation, and reduce noise interference. By weighted summing of the gradients, the pixel information in the neighborhood is fully utilized, local features can be better captured, and the analysis depth can be improved.

[0069] S6: Generate a BRIEF descriptor of the target feature point according to the main direction of the target feature point.

[0070] Among them, the BRIEF (Binary Robust Invariant Scalable Keypoints) descriptor is a binary feature descriptor for describing image feature points, which aims to provide fast and effective feature description. BRIEF generates a descriptor by randomly selecting a pair of pixels around the feature point and comparing the brightness. The generated descriptor is a binary string that reflects the brightness relationship of these pixels. This descriptor has rotation invariance and certain scale invariance, and is suitable for feature matching and recognition tasks. It is widely used in real-time vision systems due to its high computational efficiency and low storage requirements.

[0071] In the present invention, the BRIEF descriptor generates a binary string by simple brightness comparison, which has fast calculation speed and is very suitable for real-time applications. The main direction is used to ensure the consistency of the descriptor when the image is rotated, which improves the accuracy of feature matching. BRIEF has good robustness to illumination changes and noise, and enhances the identifiability of feature points under different conditions. The generated descriptor can be matched and compared quickly, and is suitable for feature recognition and tracking in real-time visual systems.

[0072] In a possible implementation, S6 specifically includes sub-steps S601 to S603:

[0073] S601: Randomly select multiple pixel pairs in the neighborhood of the target feature point.

[0074] S602: Rotate the pixel positions in each pixel pair to the main direction of the target feature point:

[0075]

[0076] Wherein, R(θ) represents a rotation matrix with the main direction θ of the target feature point as the rotation angle, cosθ represents the cosine value of the main direction of the target feature point, and sinθ represents the sine value of the main direction of the target feature point.

[0077] S603: Generate a BRIEF descriptor of the target feature point by comparing the brightness values ​​of each rotated pixel pair:

[0078]

[0079] Where d(p) represents the BRIEF descriptor of the target feature point, n represents the total number of pixel pairs after rotation, and BRIEF(p i ) represents the brightness comparison result of the i-th pixel pair, BRIEF(p) represents the brightness comparison result of the pixel pair, I(p x ) represents the pixel point p in the pixel pair x The brightness value, I(p y ) represents the pixel point p in the pixel pair y The brightness value of the

[0080] It should be noted that the FAST algorithm and the BRIEF descriptor are both components of the ORB algorithm. ORB uses FAST for fast feature point detection and uses the directionally normalized BRIEF descriptor to characterize these feature points.

[0081] Among them, the ORB (Oriented FAST and Rotated BRIEF) algorithm is a comprehensive method for feature point detection and description. It combines the FAST feature point detection algorithm and the BRIEF descriptor, and improves the robustness of the descriptor by normalizing the direction of the feature points. ORB first uses the FAST algorithm to quickly detect feature points, then achieves rotation invariance by calculating the main direction of the feature points, and finally generates a BRIEF descriptor to represent these feature points. The ORB algorithm achieves a good balance between speed and accuracy and is widely used in fields such as real-time vision systems and robot navigation.

[0082] In the present invention, by randomly selecting pixel pairs and rotating to the main direction, the BRIEF descriptor can be quickly generated and is suitable for real-time processing needs. The main direction of the target feature point is used to ensure that the descriptor remains consistent at different angles, thereby improving the accuracy and robustness of feature matching. The random selection of pixel pairs allows the descriptor to be diverse in different images, thereby better adapting to different image features. Brightness value comparison and binary output simplify the feature description process, reduce computational complexity, and help accelerate feature matching. Combining the advantages of FAST and BRIEF, the ORB algorithm provides a fast and accurate feature point detection and description solution, which can effectively improve the overall performance of the image processing system.

[0083] S7: According to the BRIEF descriptor of the target feature point, the target feature point in the driver's head image under each viewing angle is matched by the FLANN algorithm to locate the driver's head in the cockpit of the aircraft.

[0084] Among them, the FLANN (Fast Library for Approximate Nearest Neighbors) algorithm is an algorithm library for fast approximate nearest neighbor search, which is particularly suitable for feature point matching of high-dimensional data. FLANN optimizes the search process by constructing a variety of index structures (such as KD tree, K-means tree, etc.), so that points similar to the query point can be efficiently found on large-scale data sets. The algorithm places special emphasis on speed and flexibility, and can select the most suitable indexing method according to data characteristics. It is widely used in computer vision, machine learning, image retrieval and other fields.

[0085] In the present invention, the FLANN algorithm is designed for fast approximate nearest neighbor search, which can quickly find matching features in large-scale feature point data sets, thereby accelerating the positioning process. FLANN can select the most suitable index structure according to the characteristics of the data, flexibly respond to the feature point matching requirements in different scenarios, and improve matching efficiency. Its fast matching speed makes FLANN particularly suitable for real-time visual systems, such as the positioning of the pilot's head in the cockpit of an aircraft, and can meet the needs of fast response.

[0086] In a possible implementation, S7 specifically includes sub-steps S701 to S703:

[0087] S701: construct a KD tree or a K-means tree through the FLANN algorithm, perform feature point matching on target feature points in the driver's head image at various viewing angles, and obtain multiple candidate matching points.

[0088] Among them, KD tree (k-dimensional tree) is a tree data structure for storing data points in k-dimensional space, mainly used for fast search and nearest neighbor search of multidimensional data. KD tree recursively divides the data set in different dimensions to form a binary tree structure, where each node represents a data point, and divides the data into left subtree and right subtree according to the median of a certain dimension. This structure makes range query and nearest neighbor search in high-dimensional space more efficient, and is widely used in computer graphics, machine learning, pattern recognition and other fields.

[0089] Among them, K-means tree is a tree data structure based on K-means clustering algorithm, which is used for fast search and matching of high-dimensional data. It forms a tree structure by dividing data points into K clusters, each cluster is represented by its centroid. During the construction process, K initial centroids are first randomly selected, then data points are assigned to the nearest centroids, and the centroids are iteratively updated until convergence. K-means tree can effectively perform approximate nearest neighbor search and range query, and is suitable for applications that need to process large-scale data sets, such as image retrieval, machine learning, and data mining.

[0090] In the present invention, KD tree and K-means tree realize efficient nearest neighbor search through hierarchical structure, significantly reduce the time complexity of feature point matching, and are suitable for real-time systems. KD tree recursively divides the data space so that irrelevant areas can be quickly excluded when searching for feature points, thereby improving search efficiency. K-means tree uses the idea of ​​clustering to divide data points into different clusters, reducing the amount of data that needs to be compared, thereby accelerating the matching process. FLANN automatically selects the appropriate tree structure according to the characteristics of the data, so that the feature point matching can be optimized according to the actual situation, improving the overall performance of the algorithm.

[0091] In a possible implementation, in S701, a KD tree is constructed by using the FLANN algorithm, and feature point matching is performed on target feature points in the driver's head image at each viewing angle to obtain multiple candidate matching points, which specifically includes sub-steps S701A to S701F:

[0092] S701A: Determine the division dimension of the target feature points:

[0093] X KD = argmax i Var(X i )

[0094]

[0095] Among them, X KD Indicates the division dimension of the target feature points, argmax iIt means to find the variance Var(X i ) the largest i-th dimension, Var(X i ) represents the variance of the i-th dimension, m represents the total number of target feature points, X ij represents the variance of the jth target feature point in the i-th dimension, μ i Represents the mean variance of the target feature points in the i-th dimension.

[0096] It should be noted that the dimension that maximizes the variance is used to divide the target feature points in the driver's head image at each viewing angle, because the target feature points on this dimension are most widely distributed. Choosing this dimension for division can make the distribution of target feature points in different subspaces more uniform, thereby improving the efficiency of division.

[0097] S701B: According to the median of the partition dimension of each target feature point, the target feature point is divided into a left subtree and a right subtree to construct a KD tree:

[0098]

[0099] in, Represents the division dimension X of the target feature points KD of the median.

[0100] Specifically, the KD tree is constructed by recursively dividing the target feature points in the driver's head image from each perspective in each dimension to form a tree structure. When dividing, one dimension is selected and split according to the median of the dimension to form the left subtree and right subtree of the KD tree.

[0101] S701C: According to the coordinates of the preset query point, recursively traverse the KD tree to determine whether the value of the preset query point in the current dimension is less than the value of the current partition dimension of the target feature point. If so, search the left subtree until a leaf node is found. Otherwise, search the right subtree until a leaf node is found.

[0102] Specifically, after the query point is preset, the KD tree starts from the root node and recursively traverses along the tree according to the coordinates of the query point until a leaf node is found. The traversal method is to compare the value of each dimension of the query point with the partition dimension of the node. If the value of the query point in this dimension is less than the value of the current node, the left subtree is searched. Otherwise, the right subtree is searched.

[0103] S701D: Linearly search the target feature points stored in the leaf nodes through the KD tree, calculate the Euclidean distance between the target feature points stored in the leaf nodes and the preset query points, and record the target feature points stored in the leaf nodes closest to the preset query points as the nearest neighbor points.

[0104]

[0105] Among them, d(p,q) represents the Euclidean distance between the target feature point stored in the leaf node and the preset query point, p represents the preset query point, q represents the target feature point stored in the leaf node, k represents the division dimension of the target feature point stored in the leaf node, and p represents the preset query point. i Represents the value of the preset query point in the i-th dimension, q i Represents the value of the target feature point stored in the leaf node in the i-th dimension.

[0106] Specifically, after reaching a leaf node, the KD tree linearly searches for the target feature points stored in the node, calculates the Euclidean distance between these feature points and the query point, and retains the feature point with the closest distance as the nearest neighbor point.

[0107] S701E: Backtrack to the parent node of the leaf node. When the Euclidean distance between the target feature point stored in the parent node or other leaf nodes and the preset query point is closer than the current shortest distance, update the nearest neighbor point and continue backtracking until the root node.

[0108] S701F: When backtracking to the root node and no parent node or other leaf node is found whose Euclidean distance between the target feature point and the preset query point is closer than the current shortest distance, the updated nearest neighbor point is output as a candidate matching point.

[0109] In the present invention, the dimension is divided by selecting the maximum variance to ensure the uniform distribution of the target feature points in each subspace, thereby improving the efficiency of tree construction and making the subsequent search process faster. The recursive partitioning mechanism of the KD tree can efficiently manage and search high-dimensional data, reduce the complexity of feature point matching, and can quickly locate relevant features, especially in multi-view situations. Linear search of the target feature points stored in the leaf nodes, combined with the Euclidean distance calculation, can accurately find the feature points closest to the query point and improve the matching accuracy. Through the backtracking and update mechanism, the nearest neighbor points are updated in time during the search process to ensure that the candidate matching points finally output are optimal, thereby improving the robustness of the matching. Through effective partitioning and search strategies, unnecessary calculations and storage are reduced, the consumption of system resources is reduced, and the overall performance is improved.

[0110] In a possible implementation, S701 uses the FLANN algorithm to construct a K-means tree, performs feature point matching on target feature points in the driver's head image at each viewing angle, and obtains multiple candidate matching points, specifically including sub-steps S701G to S701K:

[0111] S701G: Through the K-means clustering algorithm, K target data points are randomly selected as the initial centroids in the driver's head image at each viewing angle to construct a K-means tree.

[0112] Among them, the K-means clustering algorithm is a commonly used unsupervised learning method for dividing a data set into K clusters to minimize the distance between points within the cluster. The basic steps of the algorithm include randomly selecting K initial centroids, then assigning each data point to the cluster with the nearest centroid, and then updating the centroid of each cluster to the average of all its members. This process is repeated until the centroid no longer changes significantly or the preset number of iterations is reached. The K-means algorithm is simple and efficient, and is widely used in fields such as data mining, pattern recognition, and image processing.

[0113] S701H: Calculate the distance between each target feature point and the centroid, assign the target feature point to the cluster with the nearest centroid, and update the cluster:

[0114]

[0115] in, Represents target feature points The distance from the center of mass μ, k' represents the total number of target feature points, w L Represents the lighting weight factor, μ j represents the centroid of the jth cluster, represents the i-th target feature point, w A represents the angle weight factor, θ μ Represents the angle information of the center of mass, Represents the angle information of the target feature point.

[0116] S701I: Calculate the centroid of the updated cluster:

[0117]

[0118] Among them, C j represents the jth cluster, |C j | represents the number of target data points contained in the jth cluster, and X' represents the feature vector of the target feature point in the cluster.

[0119] S701J: Repeat substeps S701H to S701I until the centroid of the cluster no longer changes or the maximum number of iterations is reached.

[0120] S701K: Output the centroid of each cluster as a candidate matching point.

[0121] In the present invention, the K-means tree performs well in processing data noise and outliers, can ensure the robustness of feature point matching, and is more effective in complex environments. The K-means algorithm can quickly divide feature points into different clusters, reduce the amount of calculation in the subsequent matching process, and make feature point matching more efficient. K-means is suitable for processing large-scale data sets, and can effectively identify and aggregate feature points in driver head images under multiple perspectives, ensuring the comprehensiveness of matching. The algorithm can flexibly select the number of clusters K according to specific needs, adapt to the distribution of feature points in different scenarios, and optimize matching accuracy. The introduction of illumination and angle weight factors makes distance calculation more physically meaningful, enhances adaptability to environmental changes (such as illumination changes and perspective changes), and improves matching accuracy. By continuously updating the centroid of the cluster, it is ensured that the algorithm can dynamically adapt to changes in data, and improves the stability and reliability of clustering.

[0122] S702: Calculate the Hamming distance between the BRIEF descriptors of each candidate matching point.

[0123] In the present invention, the BRIEF descriptor is in binary format, and the Hamming distance can effectively process this data, avoiding the complexity and errors caused by floating-point operations. The Hamming distance is highly robust to noise and small image transformations (such as rotation and scaling), and can effectively identify the similarity of feature points. By calculating the Hamming distance, similar and dissimilar feature points can be clearly distinguished, improving the accuracy of matching.

[0124] In a possible implementation manner, S702 specifically includes:

[0125] The Hamming distance between the BRIEF descriptors of each candidate matching point is calculated according to the following formula:

[0126]

[0127] Among them, d H (p',q') represents the Hamming distance between the BRIEF descriptors of each candidate matching point, n' represents the total number of bits of the BRIEF descriptor, and w N represents the noise weight factor, p′ i represents the i-th bit in the BRIEF descriptor of the candidate matching point p', ⊕ represents the XOR operation, q' i The i-th bit in the BRIEF descriptor of the candidate matching point q', w o represents the occlusion weight factor, O i Binary flag indicating occlusion.

[0128] Among them, the Hamming distance is an indicator used to measure the difference between two strings of equal length, specifically expressed as the number of different characters at the same position. It is often used in information theory, coding theory, and computer science, especially in the comparison of binary data. The Hamming distance can effectively evaluate the similarity and difference of data. For example, in feature matching, error detection and correction, calculating the Hamming distance between two binary strings or feature descriptors can help determine their degree of similarity.

[0129] Specifically, the BRIEF descriptor generated by the ORB algorithm is binary, and the Hamming distance is an effective measurement method. However, considering the actual situation in the aircraft cockpit, the matching accuracy of the BRIEF descriptor may be affected by some environmental factors, such as noise, partial occlusion, etc. Therefore, we can introduce noise weight factors and occlusion weight factors in the Hamming distance matching process, so that the system can match feature points more robustly in a complex cockpit environment.

[0130] The noise weight factor is used to adjust the weight of some unreliable bits in a noisy environment. The occlusion weight factor is used to reduce the impact of partially occluded areas. When partial occlusion of a feature point is detected (for example, a part of the helmet is covered), the system can use O i A binary flag indicating occlusion, giving an additional weighted correction to the matching results.

[0131] In the present invention, the introduction of noise and occlusion weight factors makes feature point matching more robust in complex environments and can effectively cope with the impact of noise and partial occlusion. Through weighted calculation, the system can more accurately identify and adjust unreliable bits, thereby improving the reliability and accuracy of matching and reducing false matches. In practical applications, the pilot's head may be partially occluded. Considering the occlusion factor can reduce misjudgments caused by occlusion during the matching process. Through the improved Hamming distance calculation method, the accuracy of matching can be significantly improved while maintaining high efficiency, which is suitable for the needs of real-time systems.

[0132] S703: Screen out candidate matching points with the smallest Hamming distance to locate the pilot's head in the cockpit of the aircraft.

[0133] It should be noted that those skilled in the art can set the positions of the preset query points and the preset candidate points according to actual needs, and the present invention is not limited thereto.

[0134] In the present invention, by selecting the matching point with the smallest Hamming distance, the system can ensure that the most similar feature point is selected, thereby improving the positioning accuracy of the driver's head. The matching point with the smallest Hamming distance usually means that the point has high consistency under different viewing angles, which can effectively reduce the positioning error caused by environmental changes (such as lighting, occlusion, etc.).

[0135] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0136] In the present invention, a plurality of feature points in the driver's head image are extracted by the FAST algorithm, the Harris corner point response function is used to calculate the Harris response value of each feature point, and the feature point with the maximum Harris response value is screened out in the driver's head image as the target feature point by the non-maximum suppression algorithm. The robustness of feature point detection can be always maintained when dealing with frequently changing lighting conditions and complex background interference in the cockpit. The main direction of the target feature point is calculated by the centroid method. According to the main direction of the target feature point, the BRIEF descriptor of the target feature point is generated. When processing the rotation of the driver's head, the stability of the feature point descriptor can be ensured. According to the BRIEF descriptor of the target feature point, the FLANN algorithm is used to match the target feature points in the driver's head image under various viewing angles, so as to avoid using the traditional Euclidean distance matching method under noise and local occlusion conditions, thereby improving the matching accuracy, having strong real-time processing capability, and having a fast response speed in complex scenes with high-density feature points, so as to be able to timely feedback the driver's head posture data.

[0137] Reference Manual Attached Figure 2 , showing a schematic structural diagram of a pilot head positioning system in an aircraft cockpit provided by the present invention.

[0138] The present invention further provides a pilot head positioning system 30 in an aircraft cockpit, comprising: a memory 303 and one or more processors 301 .

[0139] One or more application programs are stored in the memory 303 , and the one or more application programs are suitable for being executed by the one or more processors 301 to implement the method for locating the pilot's head in the cockpit of an aircraft as described in the method embodiment.

[0140] The pilot head positioning system 30 in the cockpit of an aircraft comprises: a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, for example, via a bus 302.

[0141] The structure of the pilot head positioning system 30 in the cockpit of the aircraft does not constitute a limitation to the embodiment of the present invention.

[0142] Processor 301 may be a CPU, a general purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. Processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0143] The bus 302 may include a path to transmit information between the above components. The bus 302 may be a PCI bus or an EISA bus, etc. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0144] The memory 303 can be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an EEPROM, a CD-ROM or other optical disk storage, an optical disk storage (including a compressed optical disk, a laser disk, an optical disk, a digital versatile disk, a Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0145] It should be noted that the pilot head positioning system 30 in the aircraft cockpit can implement the above-mentioned pilot head positioning method in the aircraft cockpit and can achieve the same or similar technical effects. To avoid repetition, the present invention will not go into details.

[0146] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0147] In the present invention, a plurality of feature points in the driver's head image are extracted by the FAST algorithm, the Harris corner point response function is used to calculate the Harris response value of each feature point, and the feature point with the maximum Harris response value is screened out in the driver's head image as the target feature point by the non-maximum suppression algorithm. The robustness of feature point detection can be always maintained when dealing with frequently changing lighting conditions and complex background interference in the cockpit. The main direction of the target feature point is calculated by the centroid method. According to the main direction of the target feature point, the BRIEF descriptor of the target feature point is generated. When processing the rotation of the driver's head, the stability of the feature point descriptor can be ensured. According to the BRIEF descriptor of the target feature point, the FLANN algorithm is used to match the target feature points in the driver's head image under various viewing angles, so as to avoid using the traditional Euclidean distance matching method under noise and local occlusion conditions, thereby improving the matching accuracy, having strong real-time processing capability, and having a fast response speed in complex scenes with high-density feature points, so as to be able to timely feedback the driver's head posture data.

[0148] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program can be loaded and executed by a processor to implement the method for locating a pilot's head in an aircraft cockpit as described in the first aspect.

[0149] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0150] There are a few points to note:

[0151] (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention, and other structures may refer to the general design.

[0152] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of the layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.

[0153] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.

[0154] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for positioning a pilot's head in an aircraft cockpit, characterized in that: include: S1: Acquire the pilot's head image in the aircraft cockpit from multiple perspectives; S2: extracting multiple feature points from the driver's head image using the FAST algorithm; S3: Calculating the Harris response value of each of the feature points by using the Harris corner point response function; S4: Using a non-maximum suppression algorithm, a feature point with a maximum Harris response value is selected from the driver's head image as a target feature point; S5: Calculate the main direction of the target feature point by using the centroid method; S6: generating a BRIEF descriptor of the target feature point according to the main direction of the target feature point; S7: According to the BRIEF descriptor of the target feature point, the target feature point in the driver's head image at each viewing angle is matched by the FLANN algorithm to locate the driver's head in the cockpit of the aircraft.

2. The method for locating the pilot's head in an aircraft cockpit according to claim 1, characterized in that: The S2 specifically includes: S201: Optimize the FAST algorithm through adaptive threshold adjustment: t=μ I +a·s I Where t represents the brightness threshold after adaptive threshold adjustment, μ I represents the average brightness value of the pixels in the local area of ​​the driver's head image, α represents the adjustment coefficient, and σ I Represents the brightness standard deviation of pixels in the local area of ​​the driver's head image; S202: extracting multiple feature points from the driver's head image using the optimized FAST algorithm: |I(p i )-I(p)|>t Among them, I(p i ) represents the brightness value of the i-th pixel in the neighborhood of the preset candidate feature point p, I(p) represents the brightness value of the preset candidate feature point p, and p represents the preset candidate feature point in the driver's head image.

3. The method for locating the pilot's head in an aircraft cockpit according to claim 1, characterized in that: The S3 is specifically: The Harris response value of each feature point is calculated by the Harris corner point response function according to the following formula: R=det(H)-k·(trace(H)) 2 Where R represents the Harris response value of the feature point, det(H) represents the determinant of the Hessian matrix H, H represents the Hessian matrix of the driver's head image, k represents a constant, trace(H) represents the trace of the Hessian matrix H, and L xx represents the second-order derivative of the driver's head image in the x direction, L xy represents the mixed second-order derivative of the driver's head image in the x and y directions, L yy Represents the second-order derivative of the driver's head image in the y direction.

4. The method for locating the pilot's head in an aircraft cockpit according to claim 1, characterized in that: The S5 specifically includes: S501: Calculate the horizontal gradient and vertical gradient of the neighborhood of the target feature point respectively: G x (x,y)=I(x+1,y)-I(x-1,y) G y (x,y)=I(x,y+1)-I(x,y-1) Among them, G x (x, y) represents the horizontal gradient of the pixel point (x, y) in the neighborhood of the target feature point, I(x+1, y) represents the brightness value of the pixel point (x+1, y) in the neighborhood of the target feature point, I(x-1, y) represents the brightness value of the pixel point (x-1, y) in the neighborhood of the target feature point, G y (x,y) represents the vertical gradient of the pixel point (x,y) in the neighborhood of the target feature point, I(x,y+1) represents the brightness value of the pixel point (x,y+1) in the neighborhood of the target feature point, I(x,y-1) represents the brightness value of the pixel point (x,y-1) in the neighborhood of the target feature point, x represents the horizontal coordinate of the pixel point, and y represents the vertical coordinate of the pixel point; S502: Taking the brightness values ​​of the pixels in the neighborhood of the target feature point as weights, weighted summing is performed on the horizontal gradient and the vertical gradient in the neighborhood of the target feature point to calculate the main direction of the target feature point: Among them, θ represents the main direction of the target feature point, atan2() represents the two-parameter inverse tangent function, and I(x,y) represents the brightness value of the pixel point (x,y) in the neighborhood of the target feature point.

5. The method for locating the pilot's head in an aircraft cockpit according to claim 1, characterized in that: The S6 specifically includes: S601: randomly selecting a plurality of pixel pairs in the neighborhood of the target feature point; S602: Rotate the pixel positions in each pixel pair to the main direction of the target feature point: Wherein, R(θ) represents the rotation matrix with the main direction θ of the target feature point as the rotation angle, cosθ represents the cosine value of the main direction of the target feature point, and sinθ represents the sine value of the main direction of the target feature point; S603: Generate a BRIEF descriptor of the target feature point by comparing the brightness values ​​of each rotated pixel pair: Where d(p) represents the BRIEF descriptor of the target feature point, n represents the total number of pixel pairs after rotation, and BRIEF(p i ) represents the brightness comparison result of the i-th pixel pair, BRIEF(p) represents the brightness comparison result of the pixel pair, I(p x ) represents the pixel point p in the pixel pair x The brightness value, I(p y ) represents the pixel point p in the pixel pair y The brightness value of the 6. The method for locating the pilot's head in an aircraft cockpit according to claim 1, characterized in that: The S7 specifically includes: S701: constructing a KD tree or a K-means tree through the FLANN algorithm, performing feature point matching on target feature points in the driver's head image at each viewing angle, and obtaining a plurality of candidate matching points; S702: Calculate the Hamming distance between the BRIEF descriptors of each of the candidate matching points; S703: Screen out candidate matching points with the smallest Hamming distance to locate the pilot's head in the cockpit of the aircraft.

7. The method for locating the pilot's head in an aircraft cockpit according to claim 6, characterized in that: In S701, a KD tree is constructed by using the FLANN algorithm to match the target feature points in the driver's head image at each viewing angle to obtain multiple candidate matching points, specifically including: S701A: Divide the target feature points in the driver's head image at each viewing angle into dimensions to determine the division dimensions of the target feature points: X KD =argmax i Var(X i ) Among them, X KD Indicates the division dimension of the target feature points, argmax i It means to find the variance Var(X i ) the largest i-th dimension, Var(X i ) represents the variance of the i-th dimension, m represents the total number of target feature points, X ij represents the variance of the jth target feature point in the i-th dimension, μ i Represents the variance mean of the target feature points in the i-th dimension; S701B: According to the median of the partition dimension of each target feature point, the target feature point is divided into a left subtree and a right subtree to construct the KD tree: in, Represents the division dimension X of the target feature points KD the median; S701C: recursively traverse the KD tree according to the coordinates of the preset query point to determine whether the value of the preset query point in the current dimension is less than the value of the current division dimension of the target feature point; if so, search the left subtree until a leaf node is found; otherwise, search the right subtree until a leaf node is found; S701D: linearly search the target feature points stored in the leaf nodes through the KD tree, calculate the Euclidean distance between the target feature points stored in the leaf nodes and the preset query point, and record the target feature points stored in the leaf nodes closest to the preset query point as the nearest neighbor points; Among them, d(p,q) represents the Euclidean distance between the target feature point stored in the leaf node and the preset query point, p represents the preset query point, q represents the target feature point stored in the leaf node, k represents the division dimension of the target feature point stored in the leaf node, and p represents the preset query point. i Represents the value of the preset query point in the i-th dimension, q i Represents the value of the target feature point stored in the leaf node in the i-th dimension; S701E: Backtracking to the parent node of the leaf node, when the Euclidean distance between the target feature point stored in the parent node or other leaf nodes and the preset query point is closer than the current shortest distance, updating the nearest neighbor point and continuing to backtrack until the root node; S701F: When tracing back to the root node and no Euclidean distance between the target feature point stored in any parent node or other leaf node and the preset query point is found to be closer than the current shortest distance, the updated nearest neighbor point is output as the candidate matching point.

8. The method for locating the pilot's head in an aircraft cockpit according to claim 6, characterized in that: In S701, the K-means tree is constructed by the FLANN algorithm to match the target feature points in the driver's head image at each viewing angle to obtain multiple candidate matching points, which specifically include: S701G: Using the K-means clustering algorithm, K target data points are randomly selected as initial centroids in the driver head images at each viewing angle to construct a K-means tree; S701H: Calculate the distance between each target feature point and the centroid, and assign the target feature point to the cluster where the nearest centroid is located, and update the cluster: in, Represents target feature points The distance from the center of mass μ, k' represents the total number of target feature points, w L Represents the lighting weight factor, μ j represents the centroid of the jth cluster, represents the i-th target feature point, w A represents the angle weight factor, θ μ Represents the angle information of the center of mass, Represents the angle information of the target feature point; S701I: Calculate the centroid of the updated cluster: Among them, C j represents the jth cluster, |C j | represents the number of target data points contained in the jth cluster, and X' represents the feature vector of the target feature point in the cluster; S701J: Repeat substeps S701H to S701I until the centroid of the cluster no longer changes or the maximum number of iterations is reached; S701K: Output the centroid of each cluster as the candidate matching point.

9. The method for locating the pilot's head in an aircraft cockpit according to claim 6, characterized in that: The S702 is specifically as follows: The Hamming distance between the BRIEF descriptors of the candidate matching points is calculated according to the following formula: Among them, d H (p',q') represents the Hamming distance between the BRIEF descriptors of each candidate matching point, n' represents the total number of bits of the BRIEF descriptor, and w N represents the noise weight factor, p′ i The i-th bit in the BRIEF descriptor representing the candidate matching point p', represents the XOR operation, q′ i The i-th bit in the BRIEF descriptor of the candidate matching point q', w o represents the occlusion weight factor, O i Binary flag indicating occlusion.

10. A pilot head positioning system in an aircraft cockpit, characterized in that: include: memory and one or more processors; One or more application programs are stored in the memory, and the one or more application programs are suitable for being executed by the one or more processors to implement the method for locating a pilot's head in an aircraft cockpit according to any one of claims 1 to 9.

Images