Image matching method and device, equipment, medium and program product

Abstract

The embodiment of the invention discloses an image matching method and device, equipment, a medium and a program product, and the method comprises the steps: obtaining an image data set which comprises an airplane docking component template, and the airplane docking component template comprises pose labeling information; the method comprises the following steps: acquiring a component image of a to-be-docked airplane docking component, and determining feature points in the component image; determining a target docking component template corresponding to the component image according to the feature points and the pose labeling information; and adjusting the target docking component template according to the pose information of the aircraft docking component to be docked. According to the technical scheme provided by the invention, the accuracy of determining the position and posture of the to-be-docked airplane docking parts can be improved, and then the docking accuracy of the parts is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to an image matching method, device, equipment, medium and program product. BACKGROUND

[0002] Aircraft major component docking is a critical step in modern aviation manufacturing. It not only improves production efficiency and reduces cost, but also enhances the flexibility and maintainability of the aircraft through modular design. This technology enables precise docking of components produced in different regions, ensuring the integrity and safety of the aircraft structure, while also facilitating future upgrades and modifications.

[0003] Due to the large size of aircraft components, it is difficult to determine the position and attitude of the docking components in real time during the docking process. In the prior art, the position and attitude are generally positioned manually when docking aircraft components. However, due to the limited field of view of the human eye, there is a large error in determining the attitude and position of the aircraft components, which ultimately leads to low docking accuracy of the components to be docked. SUMMARY

[0004] The present application provides an image matching method, device, equipment, medium and program product, which can accurately position the position and attitude of the aircraft docking component to be docked, and improve the accuracy of aircraft docking.

[0005] In a first aspect, the present application provides an image matching method, comprising:

[0006] obtaining an image data set, wherein the image data set includes an aircraft docking component template, and the aircraft docking component template includes pose annotation information;

[0007] obtaining a component image of an aircraft docking component to be docked, and determining feature points in the component image;

[0008] determining a target docking component template corresponding to the component image according to the feature points and the pose annotation information;

[0009] adjusting the target docking component template according to the pose information of the aircraft docking component to be docked.

[0010] In a second aspect, the present application provides an image matching device, comprising:

[0011] a data set acquisition module for acquiring an image data set, wherein the image data set includes an aircraft docking component template, and the aircraft docking component template includes pose annotation information;

[0012] The feature point determination module is configured to acquire a component image of an aircraft docking component to be docked, and determine feature points in the component image.

[0013] The template determination module is configured to determine a target docking component template corresponding to the component image according to the feature points and the pose annotation information.

[0014] The pose adjustment module is configured to adjust the target docking component template according to the pose information of the aircraft docking component to be docked.

[0015] In a third aspect, an electronic device is provided, and the electronic device includes:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; and

[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the image matching method according to any one of the embodiments of the present application.

[0019] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores computer instructions for enabling a processor to implement the image matching method according to any one of the embodiments of the present application.

[0020] In a fifth aspect, a computer program product is provided, and the computer program product includes a computer program, and the computer program, when executed by a processor, implements the image matching method according to any one of the embodiments of the present application.

[0021] An embodiment of the present invention provides an image matching method, apparatus, device, medium, and program product. The method includes: obtaining an image data set, wherein the image data set includes an aircraft docking component template, and the aircraft docking component template includes posture annotation information; obtaining a component image of the aircraft docking component to be docked, and determining feature points in the component image; determining a target docking component template corresponding to the component image based on the feature points and the posture annotation information; and adjusting the target docking component template based on the posture information of the aircraft docking component to be docked. Specifically, the target docking component template corresponding to the component image can be accurately determined based on the feature points of the component image of the aircraft docking component to be docked and the posture annotation information of the aircraft docking component template, and then the target docking component template can be adjusted based on the posture information of the aircraft docking component to be docked, so that the target docking component template has the same posture as the aircraft docking component to be docked. The method of the embodiment of the present invention can make the target docking component template and the aircraft docking component to be docked have the same posture, and then simulate the posture of the aircraft docking component to be docked through the target docking component template, so as to facilitate real-time observation of the position and posture of the aircraft docking component to be docked, thereby improving the accuracy of determining the posture of the aircraft docking component to be docked, and thus improving the accuracy of component docking. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 A flowchart of an image matching method provided in Example 1 of the present invention;

[0024] Figure 2 A flowchart of an image matching method provided in Example 2 of the present invention;

[0025] Figure 3 A flowchart of an image matching method provided in Example 3 of the present invention;

[0026] Figure 4 A flowchart of an image matching method provided in Embodiment 4 of the present invention;

[0027] Figure 5 A schematic structural diagram of an image matching device provided in a fifth embodiment of the present invention;

[0028] Figure 6 This is a structural diagram of an electronic device provided in Example 6 of the present invention. DETAILED DESCRIPTION

[0029] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0030] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0031] It should be noted that in the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the technical solutions comply with relevant laws and regulations and do not violate public order and good customs.

[0032] Embodiment one

[0033] Figure 1 A flowchart of an image matching method provided by the first embodiment of the present application is provided, which can be applied to the docking of aircraft major components, and can be particularly applied to the pose determination of aircraft major components during the docking of aircraft major components. The method can be executed by an image matching device, which can be composed of software and / or hardware and configured in a computer or a server.

[0034] As shown in Figure 1 , comprising:

[0035] Step 110, acquiring an image data set, wherein the image data set includes an aircraft docking component template, and the aircraft docking component template includes pose annotation information.

[0036] Wherein, the image dataset is an image set of the aircraft docking component. The aircraft docking component template is an image template of the aircraft docking component, such as a 3D image template. The pose annotation information of the aircraft docking component template is information annotated on the aircraft docking component template, used to represent the position information and attitude information of the aircraft docking component. The pose annotation information also includes coordinate information of key points, feature points and edges, texture and color information, etc. of the aircraft docking component.

[0037] Step 120, obtaining a component image of the aircraft docking component to be docked, and determining feature points in the component image.

[0038] Wherein, the component image can be a real-time image of the aircraft docking component to be docked. For example, the component image can be multiple images of the aircraft docking component to be docked at different angles. Alternatively, the component image can be multiple local images of the aircraft docking component to be docked, etc. The feature points of the component image can represent the positions that are significant and / or unique in the component image. Under different viewing angles and lighting conditions, the feature points of the component image remain unchanged. For example, the feature points of the component image usually represent the positions of high texture, corner points or edges in the component image. Optionally, the description information of the feature points of the component image includes at least one of position information, direction information, attitude information, and texture information. The feature points in the component image can be determined by a feature point detection algorithm. The corresponding aircraft docking component to be docked can be determined through the feature points of the component image.

[0039] Specifically, the complete image of the aircraft docking component to be docked can be determined by splicing images of the aircraft docking component to be docked at different angles, and then the feature points of the complete image can be determined through feature extraction of the complete image.

[0040] Step 130, determining a target docking component template corresponding to the component image according to the feature points and the pose annotation information.

[0041] Wherein, the target docking component template is a virtual image used to represent the aircraft component to be docked in the component image. The target docking component template can be obtained by matching the description information of the feature points with the position annotation information.

[0042] Specifically, the coordinate information, direction information, color and texture information contained in the feature points can be matched with the pose annotation information of the aircraft docking component template. Then, the aircraft docking component template that matches successfully can be determined as the target docking component template corresponding to the component image.

[0043] Step 140, adjusting the target docking component template according to the pose information of the aircraft docking component to be docked.

[0044] Specifically, since the position and posture of the aircraft docking component to be docked will change in real time during the docking process, the target docking component template can be adjusted through the pose information of the aircraft docking component to be docked, so as to ensure that the pose information of the target docking component template is the same as the pose information of the aircraft docking component to be docked. Therefore, the pose information of the aircraft docking component to be docked can be accurately determined through the pose information of the target docking component template.

[0045] Optionally, step 140 comprises:

[0046] superimposing the target docking component template on the component image.

[0047] Specifically, after the target docking component template corresponding to the aircraft docking component to be docked in the image data set is determined, the two can be image-corresponded through superimposition operation, and then the pose information of the aircraft docking component to be docked can be represented by the pose information of the target docking component template through the method of the embodiment of the application.

[0048] determining the position and direction of the aircraft docking component to be docked according to the feature points in the component image.

[0049] Specifically, the feature points in the component image can be connected and combined to determine the pose information, such as position, posture, reverse direction and rotation angle, of the aircraft docking component to be docked.

[0050] updating the position and direction of the target docking component template according to the position and direction of the aircraft docking component to be docked.

[0051] Specifically, the position and direction of the target docking component template can be updated through the position and direction of the aircraft docking component to be docked, so as to ensure that the position and direction of the target docking component template are the same as the position and direction of the aircraft docking component to be docked, and then the position and direction of the aircraft docking component to be docked can be accurately simulated according to the position and direction of the target docking component template.

[0052] Exemplarily, the target docking component template can be displayed on the display screen, so that the worker can completely understand the progress of the docking work.

[0053] The embodiment of the present application provides a kind of image matching method, the method comprises: obtaining image dataset, wherein the image dataset includes aircraft docking component template, the aircraft docking component template includes pose annotation information;The component image of aircraft docking component to be docked is acquired, and the feature point in the component image is determined;According to the feature point and pose annotation information, the target docking component template corresponding to the component image is determined;According to the pose information of the aircraft docking component to be docked, the target docking component template is adjusted.The specific, through the feature point of the component image of aircraft docking component to be docked and the pose annotation information of aircraft docking component template, the target docking component template corresponding to the component image can be accurately determined, and then the target docking component template can be adjusted according to the pose information of the aircraft docking component to be docked, so that the target docking component template and the pose of the aircraft docking component to be docked are same.The method of the embodiment of the present application can make the target docking component template and the pose of the aircraft docking component to be docked be same, and then the pose of the aircraft docking component to be docked is simulated by the target docking component template, to improve the accuracy of the pose of the aircraft docking component to be docked, and then the precision of component docking is improved.

[0054] Embodiment two

[0055] Figure 2 A flow chart of the image matching method for determining the image dataset provided in the second embodiment of the present application, the present embodiment is based on the above embodiments, and further limits the determination mode of image dataset.

[0056] As shown in Figure 2 , comprising:

[0057] Step 201, obtaining the pose information and texture information of the aircraft docking component template.

[0058] Among them, the pose information represents the position information and attitude information of the aircraft docking component template, the position information can be the spatial position of the aircraft docking component template, spatial coordinates, etc.;The attitude information can include the deflection angle, orientation and action information of the component, etc. Texture information is used to represent the surface feature information of the aircraft docking component template, such as the texture and color information of the outer surface, etc.

[0059] Step 202, adjusting the pose information and / or texture information of the aircraft docking component template to obtain the adjusted aircraft docking component template.

[0060] Specifically, since the pose information and / or texture information of the aircraft docking component may change in real time during the aircraft docking process, the pose information and texture information of a single aircraft docking component template cannot be used throughout the docking process. Therefore, the aircraft docking component template with different pose information and / or texture information can be obtained by adjusting the pose information and / or texture information of the aircraft docking component template.

[0061] Specifically, since the aircraft docking component template itself can be a 3D template, the adjusted aircraft docking component template can be obtained by 3D fine-tuning, for example, adjusting the orientation angle of the aircraft docking component template, adjusting the spatial position of the aircraft docking component template, adjusting the color of the aircraft docking component template, and the like. Further, a plurality of adjusted derivative templates of the aircraft docking component template are obtained.

[0062] Obtain configuration parameters, wherein the configuration parameters include at least one of the attributes such as color, position, and rotation angle. Adjust the aircraft docking component template according to the configuration parameters to obtain the adjusted aircraft docking component template.

[0063] Step 203, determining the aircraft docking component template according to the aircraft docking component template and the adjusted aircraft docking component template.

[0064] Specifically, the aircraft docking component template and the adjusted aircraft docking component template can be uniformly named as the aircraft docking component template. Therefore, the aircraft docking component template includes the diversified position, pose, texture, and color of the aircraft docking component, so that the component image of the aircraft docking component to be docked can be matched more accurately in the subsequent docking process.

[0065] For example, a 3D model of the aircraft docking component can be generated by 3D drawing software, and then the model can be captured by an RGB image through a perception camera, and then the color, position, and rotation angle of the aircraft docking component in the scene can be adjusted by using a randomization framework to generate a variety of component images of the aircraft docking component, thereby improving the diversity and complexity of the generated data set.

[0066] Step 204, performing pose information labeling on the aircraft docking component template to obtain the image data set.

[0067] The pose information labeling is a labeling method for marking each point on the aircraft docking component template and adding corresponding labels. For example, the label content can be the type of each point on the aircraft docking component template, such as component boundary, internal space, and key points (for example, marked points on the 3D model).

[0068] Specifically, the aircraft docking component template itself only includes image information, in order to ensure the integrity and accuracy of data acquisition when the image is matched, the aircraft docking component template can be marked with pose information, indicating the type, color, region to be docked and contour key points of the aircraft docking component template and other information.

[0069] Optionally, step 204 comprises:

[0070] A label asset library is created, wherein the label asset library includes labels of the aircraft docking component template, and the labels include pose information; the labels are attached to the aircraft docking component template, and the image data set is determined according to the labeled aircraft docking component template.

[0071] Specifically, the label asset library includes various label information of the aircraft docking component template, such as position information, color information, texture information and type information. By labeling the aircraft docking component template with labels, the information amount of the template can be improved, and thus the comprehensiveness of information acquisition and the matching accuracy can be improved in the matching process.

[0072] For example, the position information of the aircraft docking component template can be labeled by computer vision labeling technology. For example, the pose information labeling includes 2D and 3D bounding boxes, instance and semantic segmentation and key points. Optionally, when the pose information of the aircraft docking template is labeled, a label component needs to be added to the container corresponding to the aircraft docking template and other containers inheriting the container. A label configuration asset is created, which includes label information of all labels to be acquired in the image data set, and additional information for determining the pose of the 3D model. By increasing the labels, the pose information of the template can be accurately captured in the data set.

[0073] The embodiment of the application provides a kind of image data set determination method, by generating aircraft docking component template, and the aircraft docking component template generated is diversified adjusted, obtains the aircraft docking component template after adjustment, respectively to the aircraft docking component template generated and the aircraft docking component template after adjustment is marked with pose information, obtains image data set, can improve the diversity and complexity of image data set, and, by position information marking can ensure that the pose information of aircraft docking component template is accurately acquired in image data set.

[0074] Embodiment three

[0075] Figure 3 A flow chart of an image matching method provided by the third embodiment of the application is provided. Based on the above-mentioned embodiments, the third embodiment of the application specifically limits to acquire a component image of the aircraft docking component to be docked, and to determine feature points in the component image.

[0076] As Figure 3As shown, the method comprises:

[0077] Step 301, acquiring an image data set, wherein the image data set comprises an aircraft docking component template, and the aircraft docking component template comprises pose annotation information.

[0078] Step 302, acquiring a local image of the aircraft docking component to be docked, and splicing the local images to obtain the component image.

[0079] Specifically, due to the problem of the shooting angle of the aircraft docking component to be docked and the excessively large size of the component itself, a single picture only includes a local image of the aircraft docking component to be docked, therefore, a complete component image of the aircraft docking component to be docked needs to be obtained according to multiple local images of the aircraft docking component to be docked.

[0080] Illustratively, the local image can be a 2D image, and the component image can be a 3D image generated by splicing multiple 2D images.

[0081] Step 303, determining a feature point in the component image according to pixel gray scale information of the component image.

[0082] Wherein, the pixel gray scale information can represent the gray scale and brightness information of each point of the component image. Further, the key points of the aircraft docking component are generally points with high texture and obvious changes in brightness information, therefore, the feature point in the component image can be determined through the pixel gray scale information of the component image.

[0083] Illustratively, the determination of the feature point can be realized through the Harris corner point detection algorithm, which detects the corner points by calculating the pixel gray scale changes in different directions of the image. The result of Harris corner point detection is usually a set of corner point coordinates and corresponding corner point intensity values. These corner point coordinates can be used for subsequent image processing tasks, such as target tracking and splicing images. Harris corner point detection uses the following formula to evaluate the corner point properties of each pixel point in the image:

[0084] R = det(M) - k(tr(M)) 2 ;

[0085] Wherein, M is a matrix containing the gradient information of the pixels within the window, det(M) represents the determinant of M, tr(M) represents the trace of M, and k is a constant used to adjust the sensitivity of the corner points. When the value of R is greater than a given threshold, it is considered to be a corner point. Harris corner point detection can be performed at different scales and directions for multi-scale detection to capture corner points of different sizes and directions. It needs to be explained that the extracted corner points are the feature points.

[0086] Step 304, determining a target local image region in the component image according to the feature points, and determining feature point description information according to image features of the target local image region.

[0087] In the method, the target local image region can be a region corresponding to the feature points, and the feature point description information can represent image features of the target local image region, such as at least one of the position, texture, brightness and the like of a pixel.

[0088] Specifically, the image features of the target local image region can be determined by the feature point description information, which can be used to determine key information of the aircraft docking component to be docked, and the feature point description information can be used as identification information of the aircraft docking component to be docked to improve the matching accuracy and success rate of the aircraft docking component to be docked and the aircraft docking component template in the image matching process.

[0089] For example, the SIFT (Scale-invariant feature transform) algorithm can be used to extract features of the target local image region of the aircraft docking component template and generate corresponding feature point description information, and the feature point description information contains key features of the target local image region.

[0090] Step 305, determining a target docking component template corresponding to the component image according to the feature point description information and the pose annotation information.

[0091] Specifically, the feature point description information includes key information of the aircraft docking component to be docked, such as docking key points, poses, positions, colors and texture information of the aircraft docking component to be docked, and the pose annotation information includes key information of the aircraft docking component template, such as docking key points, poses, positions, colors and texture information of the aircraft docking component template. Therefore, the feature point description information and the pose annotation information can be matched to determine the target docking component template corresponding to the component image, i.e., the target docking component template corresponding to the aircraft docking component to be docked. The matching method can be a threshold matching method of Hamming distance or a similarity matching method.

[0092] Step 306, adjusting the target docking component template according to the pose information of the aircraft docking component to be docked.

[0093] The method provided by the embodiment of the application can make the target docking component template have the same pose as the aircraft docking component to be docked, and then determine the pose of the aircraft docking component to be docked through the target docking component template, so as to improve the accuracy of the pose determination of the aircraft docking component to be docked and improve the accuracy of the component docking.

[0094] Embodiment Four

[0095] Figure 4 A flow chart of an image matching method provided for Embodiment Four of the present application is based on the above-mentioned embodiments, and further limits determining the target docking component template corresponding to the component image according to the feature point description information and the pose annotation information, as shown in the following. Figure 4

[0096] Step 410: Obtain an image dataset, wherein the image dataset comprises aircraft docking component templates, and the aircraft docking component templates comprise pose annotation information.

[0097] Step 420: Obtain a component image of an aircraft docking component to be docked, and determine feature points in the component image.

[0098] Step 430: Determine a target local image region in the component image according to the feature points, and determine feature point description information according to image features of the target local image region.

[0099] Step 440: For any aircraft docking component template, determine key point description information according to pose annotation information of the current aircraft docking component template, wherein the key points are pixel points associated with component docking on the aircraft docking component template.

[0100] The pose annotation information comprises pose information and texture information of each pixel point on the aircraft docking component template. Further, the pixel points comprise key points associated with component docking on the aircraft docking component template, and the key points can be used to determine position information and texture information of the component associated with the docking work.

[0101] Step 450: Determine an information deviation amount according to the feature point description information and the key point description information.

[0102] Specifically, the feature point description information comprises key information of the aircraft docking component to be docked, such as docking key points, attitude, position, color, and texture information of the aircraft docking component to be docked.

[0103] Specifically, by performing deviation calculation on the feature point description information and the key point description information of the aircraft docking component to be docked, the information deviation amount of the feature point description information and the key point description information can be determined. If the information deviation amount is large and greater than a target threshold, it indicates that the current aircraft docking component template does not correspond to the aircraft docking component to be docked, and belongs to a different aircraft docking component.

[0104] Step 460: If the information deviation amount is less than the target threshold, determine that the current aircraft docking component template is the target docking component template. ​

[0105] Specifically, if the information deviation amount is less than the target threshold value, it indicates that the current aircraft docking component template matches the aircraft docking component to be docked successfully, and the current aircraft docking component template can be used as the target docking component template.

[0106] For example, the information deviation amount can be calculated by Hamming distance measurement, and the specific formula is as follows:

[0107] D(p)={d1,d2,…,d n};

[0108] D H (p,q)=∑|p i -q i |。

[0109] Where D p is the feature point description information of the aircraft docking component to be docked, and D H (p,q) is the Hamming distance between the feature point description information of the aircraft docking component to be docked and the key point description information. The target threshold value of Hamming distance is an important decision because it will affect the performance of the system. If the target threshold value is too high, the system may return too many irrelevant images, resulting in false positives. If the target threshold value is low, the system may miss some relevant images, resulting in false negatives. Therefore, the determination process of the target threshold value itself is an important step in image matching.

[0110] For example, the determination of the target threshold value can use a fruit fly optimization algorithm. The fruit fly optimization algorithm constantly updates the position information of the fruit fly during the iteration process, and finally makes the fruit fly reach the food to determine the optimal position.

[0111] The determination of the target threshold value includes:

[0112] Obtain the position initial value and adjustment parameter of each candidate threshold value, wherein the adjustment parameter is used to adjust the position of each candidate threshold value.

[0113] Each fruit fly has a preset position initial value, which can represent the position of an initial candidate threshold value. The adjustment parameter is used to adjust the position of each candidate threshold value, and the adjustment parameter can include adjustment step and adjustment direction, etc.

[0114] According to the position initial value and adjustment parameter of each candidate threshold value, the candidate threshold value is determined.

[0115] Specifically, in each iteration process, the new position of the fruit fly is determined by the position initial value and the adjustment parameter of the fruit fly, and then the candidate threshold value can be determined by the new position and the position initial value. For example, the candidate threshold value can be the distance between the new position and the position initial value.

[0116] perform image matching on the preset picture set according to the candidate threshold value, to obtain matching accuracy of the candidate threshold value.

[0117] The preset picture set includes a plurality of image groups, and each image group includes at least two images. The images in the same image group are matched, and the images in different image groups are not matched. For example, the preset picture set includes pictures of an aircraft part to be docked and a template of an aircraft docking part.

[0118] Further, the pictures of the aircraft part to be docked and the template of the aircraft docking part can be matched by the candidate threshold value, to determine a matching result, and the matching accuracy is determined according to the matching result.

[0119] For example, the fruit fly optimization algorithm includes setting an initial position of each fruit fly, InitX, and assigning a step size and a direction for each fruit fly to search for an optimal position. X i = X + X RANDOM ; Y i = Y + Y RANDOM . Wherein, X RANDOM is the updated position of the fruit fly on the X-axis in each iteration, Y RANDOM is the updated position of the fruit fly on the Y-axis in each iteration, including the update step size and the update direction.

[0120] Since the position of the optimal threshold value cannot be known, the distance Dist i between the existing position of each fruit fly and the origin is calculated first. i Dist

[0121] Specifically, for each candidate threshold value determined at present, image matching is performed on the pictures in the preset picture set based on the candidate threshold value. For example, the Hamming distance between different pictures can be calculated. If the Hamming distance is less than the candidate threshold value, it is determined that the two pictures are matched, otherwise, it is determined that the two pictures are not matched. The performance index and the matching accuracy corresponding to each candidate threshold value are calculated according to the matching result corresponding to the candidate threshold value. The performance index includes accuracy and recall rate. The matching result can include the following parameters: TP, indicating that a non-matching picture is correctly identified as a non-matching result (an abnormal state is correctly identified as an abnormality); FP, indicating that a normal picture is incorrectly identified as a non-matching result (a normal state is incorrectly identified as an abnormality); TN, indicating that a normal picture is correctly identified as a template picture (a normal state is correctly identified as normal); and FN, indicating that a non-matching picture is incorrectly matched as a template picture (an abnormal state is incorrectly identified as normal).

[0122] Further, the accuracy Precision, the recall rate Recall and the matching accuracy can be calculated according to the above parameters:

[0123]

[0124] wherein, Precision represents the proportion of correctly identified abnormal conditions to all identified abnormal conditions; Recall represents the proportion of correctly identified abnormal conditions to all actual abnormal conditions;

[0125] Function is the matching precision.

[0126] determining the target threshold according to the matching precision of each candidate threshold.

[0127] Specifically, in each iteration, the matching precision of each candidate threshold is determined. If the maximum matching precision in the current iteration is greater than the maximum matching precision in the last iteration, the optimal candidate threshold is determined according to the candidate threshold corresponding to the maximum matching precision in the current iteration, otherwise the optimal candidate threshold is determined according to the candidate threshold corresponding to the maximum matching precision in the last iteration. Therefore, if the iteration converges or reaches the maximum number of iterations, the target threshold is determined according to the optimal candidate threshold. The target threshold is the candidate threshold with the highest matching precision in all iterations.

[0128] Step 470, adjusting the target docking component template according to the pose information of the aircraft docking component to be docked.

[0129] The embodiment of the application provides an image matching method, which can accurately determine the value of the target threshold in the matching process, thereby increasing the accuracy of picture matching. The method of the embodiment of the application can make the pose of the target docking component template and the pose of the aircraft docking component to be docked the same, thereby simulating the pose of the aircraft docking component to be docked through the target docking component template, improving the accuracy of the pose determination of the aircraft docking component to be docked, and improving the accuracy of component docking.

[0130] Embodiment five

[0131] Figure 5 A structural schematic diagram of an image matching device provided by the embodiment five of the application is shown in FIG. 5. Figure 5 As shown in the figure, the device comprises:

[0132] The data set acquisition module 510 is configured to acquire an image data set, wherein the image data set comprises an aircraft docking component template, and the aircraft docking component template comprises pose annotation information.

[0133] The feature point determination module 520 is configured to acquire a component image of an aircraft docking component to be docked, and determine feature points in the component image.

[0134] The template determination module 530 is configured to determine a target docking component template corresponding to the component image according to the feature points and the pose annotation information.

[0135] The pose adjustment module 540 is configured to adjust the target docking component template according to the pose information of the aircraft docking component to be docked.

[0136] The image matching device provided by the embodiment of the present application can accurately determine the target docking component template corresponding to the component image of the aircraft docking component to be docked according to the feature points of the component image of the aircraft docking component to be docked and the pose annotation information of the aircraft docking component template, and then adjust the target docking component template according to the pose information of the aircraft docking component to be docked, so that the target docking component template has the same pose as the aircraft docking component to be docked. The device of the embodiment of the present application can make the target docking component template have the same pose as the aircraft docking component to be docked, and then determine the pose of the aircraft docking component to be docked through the target docking component template, so as to improve the accuracy of the pose determination of the aircraft docking component to be docked, and further improve the accuracy of the component docking.

[0137] Optionally, the method for determining the image data set comprises:

[0138] The information acquisition unit is configured to acquire the pose information and the texture information of the aircraft docking component template.

[0139] The adjustment unit is configured to adjust the pose information and / or the texture information of the aircraft docking component template to obtain an adjusted aircraft docking component template.

[0140] The combination unit is configured to determine the aircraft docking component template according to the aircraft docking component template and the adjusted aircraft docking component template.

[0141] The annotation unit is configured to perform pose information annotation on the aircraft docking component template to obtain the image data set.

[0142] Optionally, the annotation unit is specifically configured to create a label asset library, wherein the label asset library comprises labels of the aircraft docking component template, and the labels comprise pose information; the labels are attached to the aircraft docking component template, and the image data set is determined according to the annotated aircraft docking component template.

[0143] Optionally, the feature point determination module 520 comprises:

[0144] a splicing unit configured to acquire local images of the aircraft docking components to be docked, and splice the local images to obtain the component images;

[0145] a feature point determination unit configured to determine feature points in the component images according to pixel grayscale information of the component images.

[0146] Optionally, the template determination module 530 comprises:

[0147] a feature point description information determination unit configured to determine a target local image region in the component images according to the feature points, and determine feature point description information according to image features of the target local image region;

[0148] a matching unit configured to determine a target docking component template corresponding to the component images according to the feature point description information and the pose annotation information.

[0149] Optionally, the matching unit comprises:

[0150] an annotation subunit configured to determine key point description information according to pose annotation information of a current aircraft docking component template for any aircraft docking component template, wherein the key points are pixel points associated with component docking on the aircraft docking component template;

[0151] an information deviation amount determination subunit configured to determine an information deviation amount according to the feature point description information and the key point description information;

[0152] a judgment subunit configured to determine that the current aircraft docking component template is the target docking component template if the information deviation amount is less than a target threshold.

[0153] Optionally, the feature point determination module 520 further comprises a target threshold determination unit configured to determine a target threshold, comprising:

[0154] an acquisition subunit configured to acquire position initial values and adjustment parameters of each candidate threshold, wherein the adjustment parameters are used to adjust positions of the candidate thresholds;

[0155] a current value determination subunit configured to determine the candidate thresholds according to the position initial values and the adjustment parameters of the candidate thresholds;

[0156] a matching precision determination subunit configured to perform image matching on a preset picture set according to the candidate thresholds to obtain matching precisions of the candidate thresholds;

[0157] a target threshold determination subunit configured to determine the target threshold according to the matching precisions of the candidate thresholds.

[0158] Optionally, the pose adjustment module 540 comprises:

[0159] a superimposition unit configured to superimpose the target docking component template onto the component image;

[0160] a determination unit configured to determine the position and orientation of the aircraft docking component to be docked according to the feature points in the component image;

[0161] an updating unit configured to update the position and orientation of the target docking component template according to the position and orientation of the aircraft docking component to be docked.

[0162] The image matching device provided by the embodiment of the present application can execute the image matching method provided by any embodiment of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0163] Embodiment six

[0164] Figure 6 A structural schematic diagram of an electronic device 10 that can be used to implement embodiment five of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0165] As shown in Figure 6 The electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0166] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0167] The processor 11 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the image matching method.

[0168] In some embodiments, the image matching method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the image matching method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the image matching method by any other appropriate means, such as by means of firmware.

[0169] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0170] Computer programs for implementing the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer program, when executed, can cause instructions defined in the flow charts and / or block diagrams to be implemented. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, and partially on a remote machine or entirely on a remote machine or server.

[0171] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0172] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0173] The systems and techniques described herein can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described herein, or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0174] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0175] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, and the present disclosure is not limited in this regard.

[0176] The specific embodiments described above are not intended to limit the scope of the present disclosure. Those skilled in the art will understand that various modifications, combinations, sub-combinations, and alternatives can be made to the specific embodiments without departing from the spirit and principles of the present disclosure. Any further modifications, equivalents, and / or alternatives come within the scope of the present disclosure as recited by the claims.

Claims

1. An image matching method, characterized in that: include: Acquire an image dataset, wherein the image dataset includes an aircraft docking component template, and the aircraft docking component template includes pose annotation information; Acquire a component image of a docking component of an aircraft to be docked, and determine feature points in the component image; Determining a target docking component template corresponding to the component image based on the feature points and the pose annotation information; The target docking component template is adjusted according to the posture information of the aircraft docking component to be docked.

2. The method according to claim 1, characterized in that The method for determining the image dataset includes: Obtain the pose information and texture information of the aircraft docking component template; Adjusting the posture information and / or texture information of the aircraft docking component template to obtain an adjusted aircraft docking component template; determining the aircraft docking component template according to the aircraft docking component template and the adjusted aircraft docking component template; Position and posture information is annotated on the aircraft docking component template to obtain the image dataset.

3. The method according to claim 2, characterized in that The step of labeling the aircraft docking component template with position information to obtain the image dataset includes: Creating a label asset library, wherein the label asset library includes labels of the aircraft docking component templates, and the labels include pose information; The label is attached to the aircraft docking component template, and the image dataset is determined based on the labeled aircraft docking component template.

4. The method according to claim 1, wherein The step of obtaining a component image of the aircraft docking component to be docked and determining feature points in the component image includes: Acquiring partial images of the aircraft docking component to be docked, and stitching the partial images to obtain the component image; Determine feature points in the component image according to pixel grayscale information of the component image.

5. The method according to claim 4, characterized in that Determining the target docking component template corresponding to the component image according to the feature points and the pose annotation information includes: determining a target local image region in the component image according to the feature points, and determining feature point description information according to image features of the target local image region; A target docking component template corresponding to the component image is determined according to the feature point description information and the pose annotation information.

6. The method according to claim 5, characterized in that The step of determining a target docking component template corresponding to the component image based on the feature point description information and the pose annotation information includes: For any aircraft docking component template, determine key point description information based on the pose annotation information of the current aircraft docking component template, wherein the key point is a pixel point associated with the component docking on the aircraft docking component template; Determining an information deviation amount based on the feature point description information and the key point description information; If the information deviation is less than the target threshold, the current aircraft docking component template is determined to be the target docking component template.

7. The method according to claim 6, characterized in that Methods for determining the target threshold include: Obtaining an initial position value and an adjustment parameter of each candidate threshold value, wherein the adjustment parameter is used to adjust the position of each candidate threshold value; Determining each candidate threshold value according to the position initial value and adjustment parameter of each candidate threshold value; Perform image matching on a preset picture set according to the candidate thresholds to obtain a matching accuracy of each candidate threshold; The target threshold is determined according to the matching accuracy of each candidate threshold.

8. The method according to claim 1, characterized in that The adjusting the target docking component template according to the posture information of the docking component of the aircraft to be docked includes: superimposing the target docking component template onto the component image; Determining the position and orientation of the aircraft docking component to be docked according to the feature points in the component image; The position and orientation of the target docking component template are updated according to the position and orientation of the aircraft docking component to be docked.

9. An image matching device, characterized in that: include: A data set acquisition module, configured to acquire an image data set, wherein the image data set includes an aircraft docking component template, and the aircraft docking component template includes pose annotation information; A feature point determination module is used to obtain a component image of the aircraft docking component to be docked and determine feature points in the component image; A template determination module is used to determine a target docking component template corresponding to the component image based on the feature points and posture annotation information; The posture adjustment module is used to adjust the target docking component template according to the posture information of the aircraft docking component to be docked.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the image matching method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image matching method according to any one of claims 1 to 8 when executed.

12. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the image matching method according to any one of claims 1 to 8.

Images