Aircraft component self-adaptive in-place method, device, equipment and medium

Abstract

The invention discloses an aircraft component self-adaptive in-place method, device and equipment and a medium. The aircraft component self-adaptive in-position method comprises the following steps: acquiring an aircraft component ball socket image; based on the target detection model, detecting an aircraft component ball socket image to obtain a target ball socket joint image, and performing image preprocessing on the target ball socket joint image to obtain a ball socket joint circle center pixel coordinate; according to the initial positioning coordinates of the ball head and the ball socket, the pixel coordinates of the circle center of the ball socket joint and the coordinate system transformation matrix, the ball head feeding amount of the positioner is calculated; and controlling the positioner ball head to completely enter the ball socket according to the feeding amount of the positioner ball head and the force-position hybrid control model. According to the technical scheme, the ball head can be automatically guided to be accurately inserted into the ball socket in the racking process of the large aircraft component, manual intervention is not needed, the safety is high, the racking efficiency is improved, and damage to the component can be avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of aircraft assembly, and in particular, to a method, device, equipment and medium for the adaptive positioning of aircraft components. Background Art

[0002] As a continuation of the assembly of components, the docking and general assembly of large components is a complex production process of assembling large sections such as the fuselage, wings, tail wings and engines into a complete aircraft product. Its assembly quality and efficiency directly affect the final manufacturing quality and mass production delivery capacity of the complete aircraft.

[0003] At present, a three-degree-of-freedom transfer positioner is used for the process of loading and unloading components before docking. First, multiple positioners automatically move to the vicinity of the corresponding support ball socket joints of the large component under the guidance of a digital measurement device. Then, the ball head at the end of the positioner is inserted into the corresponding support ball socket joint and jacks up the large component under manual guidance to complete the loading of the large component. Finally, the transfer positioner transports the large component to the docking station and performs positioning and attitude adjustment in cooperation.

[0004] However, there are problems such as large positioning errors, low efficiency, and still certain safety risks when manually guiding the ball head at the end of the transfer positioner to insert into the ball socket. And during the positioning process of large sections, due to uncertain factors such as geometric manufacturing errors and positioning errors, excessive contact extrusion forces will be generated on the contact surface between the process joint ball socket and the positioner ball head, which easily leads to component pulling deformation and stress, and even causes component damage. Summary of the Invention

[0005] The present invention provides a method, device, equipment and medium for the adaptive positioning of aircraft components to solve the problems of low efficiency, large errors, low safety and easy component damage caused by manually guiding the ball head to insert into the ball socket during the loading process of large aircraft components.

[0006] According to one aspect of the present invention, a method for the adaptive positioning of aircraft components is provided, including:

[0007] Obtain an image of the ball socket of the aircraft component;

[0008] Based on the target detection model, detect the image of the ball socket of the aircraft component to obtain an image of the target ball socket joint, and perform image preprocessing on the image of the target ball socket joint to obtain the pixel coordinates of the center of the ball socket joint;

[0009] Calculate the feed amount of the positioner ball head according to the initial positioning coordinates of the ball head and ball socket, the pixel coordinates of the center of the ball socket joint, and the coordinate transformation matrix;

[0010] Control the positioner ball head to completely enter the ball socket according to the feed amount of the positioner ball head and the force-position hybrid control model.

[0011] According to another aspect of the present invention, there is provided an aircraft component adaptive positioning device, comprising:

[0012] An image acquisition module for acquiring the image of the ball socket of the aircraft component;

[0013] A pixel coordinate determination module for detecting the image of the ball socket of the aircraft component based on an object detection model to obtain an image of the target ball socket joint, and performing image preprocessing on the image of the target ball socket joint to obtain the pixel coordinates of the center of the ball socket joint;

[0014] A feed amount determination module for calculating the feed amount of the ball head of the locator according to the initial positioning coordinates of the ball head and the ball socket, the pixel coordinates of the center of the ball socket joint, and the coordinate system transformation matrix;

[0015] A positioning control module for controlling the ball head of the locator to fully enter the ball socket according to the feed amount of the ball head of the locator and the force-position hybrid control model.

[0016] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the aircraft component adaptive positioning method according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing a processor to implement the aircraft component adaptive positioning method according to any embodiment of the present invention when executed.

[0021] The technical solution of the embodiment of the present invention is to obtain the spherical socket image of the aircraft component, and then based on the target detection model, detect the spherical socket image of the aircraft component to obtain the target spherical socket joint image, and perform image preprocessing on the target spherical socket joint image to obtain the pixel coordinates of the center of the spherical socket joint. Furthermore, according to the initial positioning coordinates of the ball head and spherical socket, the pixel coordinates of the center of the spherical socket joint, and the coordinate system transformation matrix, calculate the feed amount of the locator ball head. Further, according to the feed amount of the locator ball head and the force-position hybrid control model, control the locator ball head to fully enter the spherical socket. In this solution, through the target detection network and image preprocessing technology, the accurate position of the center of the spherical socket joint can be excavated, so as to maximize the elimination of the alignment error between the locator ball head and the spherical socket based on the initial positioning coordinates of the ball head and spherical socket and the coordinate system transformation matrix, and obtain a more accurate feed amount of the locator ball head. And through the force-position hybrid control model, it can ensure that the locator ball head is in full contact with the spherical socket without damaging the component, solving the problems of low efficiency, large error, low safety and easy component damage existing in the process of manually guiding the ball head to insert into the spherical socket during the process of mounting large aircraft components. It can automatically guide the ball head to accurately insert into the spherical socket during the process of mounting large aircraft components, without manual intervention, with strong safety, improving the mounting efficiency, and avoiding component damage.

[0022] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0024] Figure 1 It is a flowchart of a method for self-adaptive positioning of aircraft components provided in Embodiment 1 of the present invention;

[0025] Figure 2 It is a flowchart of a method for self-adaptive positioning of aircraft components provided in Embodiment 2 of the present invention;

[0026] Figure 3 It is a logic diagram of a self-adaptive positioning of aircraft components provided in Embodiment 2 of the present invention;

[0027] Figure 4 It is a schematic structural diagram of a device for self-adaptive positioning of aircraft components provided in Embodiment 3 of the present invention;

[0028] Figure 5The schematic structural diagram of an electronic device that can be used to implement the embodiments of the present invention is shown. Detailed implementation manners

[0029] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0030] It should be noted that the terms "target" and other terms in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0031] Embodiment 1

[0032] Figure 1 The flowchart of a method for self-adaptive positioning of aircraft components provided in Embodiment 1 of the present invention. This embodiment is applicable to the situation of efficiently and safely placing aircraft components on the rack. This method can be executed by an aircraft component self-adaptive positioning device, which can be implemented in the form of hardware and / or software, and the aircraft component self-adaptive positioning device can be configured in an electronic device. The electronic device can include, but is not limited to, a transfer locator or a computer terminal, etc. As Figure 1 shown, the method includes:

[0033] Step 110: Obtain the image of the ball socket of the aircraft component.

[0034] Among them, the image of the ball socket of the aircraft component can be an image of the ball socket on the photographed aircraft component.

[0035] In the embodiment of the present invention, the transfer locator can drive the ball head of the locator to move near the ball socket of the aircraft component. At this time, the ball socket of the aircraft component appears within the field of view of the image capture component on the transfer locator, so as to take a picture of the ball socket of the aircraft component through the image capture component to obtain the image of the ball socket of the aircraft component captured by the image capture component.

[0036] Among them, the transfer locator may include an automated guided vehicle and a locator, and the locator is a 3-degree-of-freedom (X-Y-Z) system. During the operation of the transfer locator, the image capturing component fixed thereon has no movement in the vertical direction.

[0037] In an alternative embodiment of the present invention, obtaining the ball socket image of the aircraft component may include: moving to the image acquisition position according to the iGPS and the rough positioning path planning data, and collecting the ball socket image of the aircraft component through the locator camera.

[0038] Among them, the rough positioning path planning data may be path data for guiding the transfer locator to drive the ball head to move near the ball socket of the aircraft component (that is, within a certain error range from the ball socket of the aircraft component). The image acquisition position may be used to represent the photographing position of the locator camera.

[0039] In the embodiment of the present invention, the rough positioning path planning data may be determined according to the ball socket position of the aircraft component and the position of the transfer locator, so as to control the transfer locator to move to the corresponding position according to the rough positioning path planning data through iGPS (indoor GPS, indoor global positioning system), so as to photograph the ball socket of the aircraft component through the locator camera, and further obtain the ball socket image of the aircraft component taken by the locator camera.

[0040] Step 120: Based on the target detection model, detect the ball socket image of the aircraft component to obtain the target ball socket joint image, and perform image preprocessing on the target ball socket joint image to obtain the pixel coordinates of the center of the ball socket joint.

[0041] Among them, the target detection model may be a model with target detection function. The target ball socket joint image may be the region of interest identified from the ball socket image of the aircraft component based on the target detection model, that is, the ball socket joint part of the aircraft component locked and framed by the target detection model in the ball socket image of the aircraft component. The pixel coordinates of the center of the ball socket joint may be used to represent the position of the center of the ball socket joint in the pixel coordinate system.

[0042] In the embodiment of the present invention, the ball socket image of the aircraft component may be input into the target detection model, so as to lock and identify the ball socket of the ball socket image of the aircraft component through the target detection model to obtain the target ball socket joint image, and then perform image preprocessing on the target ball socket joint image according to the preset image processing method to obtain the pixel coordinates of the center of the ball socket joint.

[0043] Step 130: Calculate the feed amount of the locator ball head according to the initial positioning coordinates of the ball head and the ball socket, the pixel coordinates of the center of the ball socket joint, and the coordinate system transformation matrix.

[0044] Among them, the initial positioning coordinates of the ball head and socket can be the coordinates of the ball head in the base coordinate system of the locator when the ball head and socket are aligned (the axis of the end cylindrical surface of the actuator where the ball head is located is aligned with the axis of the cylindrical surface of the socket). The coordinate system transformation matrix can be used to convert the pixel coordinate system where the center pixel coordinate of the ball socket joint is located and the base coordinate system of the locator. Optionally, the coordinate system transformation matrix can be determined based on the hand-eye calibration method. Exemplarily, the estimateAffine3D() function in the Opencv library can be used to calculate the coordinate system transformation matrix. The feed of the locator ball head can be the movement amount of the locator ball head aligned with the socket.

[0045] In the embodiment of the present invention, based on the center pixel coordinate of the ball socket joint and the coordinate system transformation matrix, the coordinate of the center of the ball socket joint in the base coordinate system of the locator can be determined, and the coordinate of the center of the ball socket joint in the base coordinate system of the locator can be calibrated by using the initial positioning coordinates of the ball head and socket to obtain the feed of the locator ball head.

[0046] Step 140: Control the locator ball head to completely enter the socket according to the feed of the locator ball head and the force-position hybrid control model.

[0047] Among them, the force-position hybrid control model can be a feedback control model that uses position control in the vertical direction (which can be understood as the Z direction) and performs force control in the orthogonal plane of the vertical direction (which can be understood as the plane formed by the X and Y directions) during the process of the locator ball head coming into contact with and positioning in the socket on the aircraft component.

[0048] In the embodiment of the present invention, the movement of the transfer locator can be controlled by the feed of the locator ball head, and after reaching the corresponding position (that is, the locator ball head is aligned with the socket on the aircraft component in the vertical direction), the locator ball head can be controlled to enter the socket through the force-position hybrid control model. Through the force-position hybrid control model, when the Z-direction force reaches the set threshold, it can be determined that the locator ball head and the socket are in full contact, and there are errors in the manufacturing of the aircraft component, and forces will be generated in the X and Y directions during the insertion process. Through the force-position hybrid control model, the locator ball head can be controlled to move in the opposite direction until the force range is satisfied, preventing damage to the aircraft component, realizing the adaptive positioning of the locator ball head, without manual intervention, while meeting the positioning accuracy, avoiding component damage caused by excessive contact force, and greatly improving the quality and efficiency of the docking of large components.

[0049] The technical solution of the embodiment of the present invention is to obtain the spherical socket image of the aircraft component, and then based on the target detection model, detect the spherical socket image of the aircraft component to obtain the target spherical socket joint image, and perform image preprocessing on the target spherical socket joint image to obtain the pixel coordinates of the center of the spherical socket joint. Furthermore, according to the initial positioning coordinates of the ball and spherical socket, the pixel coordinates of the center of the spherical socket joint, and the coordinate system transformation matrix, calculate the feed amount of the locator ball head. Further, according to the feed amount of the locator ball head and the force-position hybrid control model, control the locator ball head to fully enter the spherical socket. In this solution, through the target detection network and image preprocessing technology, the accurate position of the center of the spherical socket joint can be excavated, so as to eliminate the alignment error between the locator ball head and the spherical socket to the greatest extent based on the initial positioning coordinates of the ball and spherical socket and the coordinate system transformation matrix, and obtain a more accurate feed amount of the locator ball head. And through the force-position hybrid control model, it can ensure that the locator ball head is in full contact with the spherical socket without damaging the component, solving the problems of low efficiency, large error, low safety and easy component damage existing in the manual guidance of the ball head into the spherical socket during the process of mounting large aircraft components. It can automatically guide the ball head to accurately insert into the spherical socket during the process of mounting large aircraft components, without manual intervention, with strong safety, improving the mounting efficiency, and avoiding component damage.

[0050] Embodiment 2

[0051] Figure 2 The flowchart of an aircraft component adaptive positioning method provided by Embodiment 2 of the present invention. This embodiment is specific based on the above embodiment, and gives a specific and optional implementation manner for performing image preprocessing on the target spherical socket joint image to obtain the pixel coordinates of the center of the spherical socket joint. As Figure 2 shown, the method includes:

[0052] Step 210, obtain the spherical socket image of the aircraft component.

[0053] Step 220, based on the target detection model, detect the spherical socket image of the aircraft component to obtain the target spherical socket joint image, and obtain the camera distortion coefficient.

[0054] Among them, the camera distortion coefficient can be data describing the distortion of the lens in the camera.

[0055] In the embodiment of the present invention, after obtaining the target spherical socket joint image, the camera distortion coefficient of the camera for shooting the spherical socket image of the aircraft component can be further obtained based on the Zhang Zhengyou calibration method.

[0056] In an optional embodiment of the present invention, the target detection model can be a model constructed based on yolov5, attention mechanism and BiFPN.

[0057] Among them, YOLOv5 (i.e., a single-stage detection model) can include four modules: Input, Backbone, Neck, and Head. The functions of the aforementioned four modules are as follows: The Input module is used to preprocess the image into data that can be recognized and processed by the Backbone module. The Backbone module consists of parts such as CBS, C3, and SPPF, and is used to extract image features. The Neck module is PAFPN (Path Aggregation Network with Feature Pyramid Network, an object detection network based on the cascaded attention mechanism), which can deeply fuse multi-scale features. The Head module classifies and locates the multi-scale features fused by the Neck part, thereby completing the entire object detection task. YOLOv5 has various variants such as s, m, and l. To meet the real-time requirement, the smaller-sized YOLOv5 is used in this solution.

[0058] In the embodiment of the present invention, based on the YOLOv5 as the basic model, a CA attention mechanism module can be inserted between the SPPF part of the Backbone module and the immediately adjacent previous C3 part to improve the detection effect of the model. At the same time, to highlight the proportion of important information in the fusion process and suppress unimportant information, PAFPN is replaced with BiFPN (Bidirectional Feature Pyramid Network, a weighted bidirectional feature pyramid network) to obtain an object detection model.

[0059] Optionally, a camera can be used to record a ball socket video to obtain the original video data. To maintain the diversity of the dataset, it can be recorded from multiple angles, at different distances, and in different lighting environments, and similar-shaped objects can be added for interference to improve the robustness of the model. By recording in different backgrounds, the object features can be enhanced. Video frames in the original video data are intercepted at fixed frame intervals and saved as color images. The dataset is labeled using the LabelImg annotation tool, and the labeled data is saved in the.txt format that can be used by the YOLO model. To prevent model overfitting, the sample set can also be processed through data augmentation methods such as cropping, adjusting brightness and color, and flipping. Then, the sample set is divided into a training set, a validation set, and a test set according to a ratio, and the object detection model is trained, validated, and tested using the training set, the validation set, and the test set to obtain an object detection model that meets the accuracy requirements.

[0060] Step 230: Based on the camera distortion coefficient, eliminate the distortion of the target ball socket joint image to obtain the ball socket joint image to be detected.

[0061] Among them, the ball socket joint image to be detected can be an image obtained by eliminating the distortion of the target ball socket joint image based on the camera distortion coefficient.

[0062] In an embodiment of the present invention, the distortion of the target ball socket joint image can be eliminated by using the camera distortion coefficient to obtain the ball socket joint image to be detected.

[0063] Step 240: Determine the pixel coordinates of the center of the ball socket joint according to the Hough gradient method and the ball socket joint image to be detected.

[0064] In an embodiment of the present invention, the Hough gradient method can be used to detect and estimate the center of the ball socket joint in the ball socket joint image to be detected, and the pixel coordinates of the center of the ball socket joint are obtained.

[0065] In an optional embodiment of the present invention, performing image preprocessing on the target ball socket joint image to obtain the pixel coordinates of the center of the ball socket joint may include: determining candidate center pixel points according to the Hough gradient method and the ball socket joint image to be detected; determining the pixel coordinates of the center of the ball socket joint according to the accumulation voting mechanism and the candidate center pixel points.

[0066] Among them, the candidate center pixel points may be the pixel points corresponding to the center of the circle determined from the ball socket joint image to be detected based on the Hough gradient method in the ball socket joint image to be detected.

[0067] In an embodiment of the present invention, at least one center of the circle can be detected from the ball socket joint image to be detected based on the Hough gradient method, and the pixel points corresponding to the detected centers of the circle are used as candidate center pixel points, so as to vote for the candidate center pixel points through an accumulator, and further use the coordinates of the center pixel point with the highest number of votes as the pixel coordinates of the center of the ball socket joint.

[0068] Exemplarily, the voting mechanism of the accumulator is as follows: draw a line along the gradient direction of the edge pixel point, and set an accumulator at the pixel points passed by the line segment. When the line segment passes through this pixel point, vote, that is, add 1 to the accumulator. The pixel point corresponding to the accumulator with the most votes is the center of the circle, and the distance from all edge points to this point is the radius.

[0069] Step 250: Calculate the feed amount of the locator ball head according to the initial positioning coordinates of the ball head and the ball socket joint, the pixel coordinates of the center of the ball socket joint, and the coordinate system transformation matrix.

[0070] In an optional embodiment of the present invention, calculating the feed amount of the locator ball head according to the initial positioning coordinates of the ball head and the ball socket joint, the pixel coordinates of the center of the ball socket joint, and the coordinate system transformation matrix may include: multiplying the pixel coordinates of the center of the ball socket joint by the coordinate system transformation matrix to obtain the coordinates to be calibrated; determining the feed amount of the locator ball head according to the coordinates to be calibrated and the initial positioning coordinates of the ball head and the ball socket joint.

[0071] Among them, the coordinates to be calibrated can be used to represent the position of the pixel coordinates of the center of the ball socket joint in the base coordinate system of the locator.

[0072] In an embodiment of the present invention, the pixel coordinates of the center of the ball socket joint can be multiplied by the coordinate system transformation matrix to obtain the coordinates to be calibrated, and the coordinates to be calibrated are added to the initial positioning coordinates of the ball head and the ball socket to obtain the feed amount of the locator ball head.

[0073] Step 260: Control the locator ball head to completely enter the ball socket according to the feed amount of the locator ball head and the force-position hybrid control model.

[0074] In an alternative embodiment of the present invention, controlling the locator ball head to completely enter the ball socket according to the feed amount of the locator ball head and the force-position hybrid control model may include: moving the locator ball head to below the ball socket according to the feed amount of the locator ball head, and performing contact force feedback control on the locator ball head according to the force-position hybrid control model, so that the locator ball head completely enters the ball socket.

[0075] In an embodiment of the present invention, the locator ball head can be moved to be aligned below the ball socket of the aircraft component by the transfer locator according to the feed amount of the locator ball head, so as to give the movement speed of the locator ball head in the Z direction, and contact force feedback control is performed on the locator ball head in combination with the force-position hybrid control model, so that the locator ball head completely enters the ball socket.

[0076] Exemplarily, the hand-eye calibration method is as follows: First, attach a calibration board (such as a checkerboard calibration board) to the plane where the contour of the center of the ball socket is located (to ensure the accuracy of the scaling factor of the hand-eye calibration, the thickness of the calibration board approaches 0). The industrial computer in the transfer locator moves in the horizontal direction (XOY plane) of the locator base coordinate system, and the camera moves passively to ensure that the calibration board always appears within the camera's field of view. The transfer locator needs to take a picture of the calibration board every time it moves to a position, and at the same time, take the coordinates read from the upper computer of the transfer locator as the coordinates of the locator ball head. The coordinate serial number of the GUI interface corresponds one-to-one with the serial number of the calibration board picture. Then, the corner points of the calibration board are analyzed to obtain the position coordinates of the calibration board in the pixel coordinate system. The above coordinate pairs are divided proportionally to complete the production of calibration data, and the estimateAffine3D() function in the Opencv library is used to calculate the coordinate system transformation matrix M (including translation, rotation, and scaling). Let S B be the locator base coordinate system, S P be the pixel coordinate system, S T be the locator industrial computer coordinate system, and the three satisfy the following relationship: S B =S T +t, S B =MS P . Among them, t is the translation vector, that is, the coordinates of the locator ball head in S B when the locator ball head is aligned with the ball socket.

[0077] In a specific example, the controller transfer function of the force-position hybrid control model is: Among them, M c , B c are settable parameters. The Z-direction movement speed in the unconstrained movement state is: The final speed will converge to By F d The expected value of sets B c The speed in the unconstrained state. After contact occurs, the deviation between the assembly force and the set value is: Among them, F d is the threshold of the Z-direction force, and k E represents the environmental stiffness.

[0078] In a specific example, such as Figure 3 shown, the transfer locator can obtain the ball socket image of the aircraft component through the camera, and can calibrate the camera through the checkerboard calibration plate to determine the internal parameters of the target detection model. Then, based on the internal parameters and the ball socket image of the aircraft component, target detection and center detection of the circle are performed, and combined with the coordinate system transformation matrix, the position deviation between the ball head of the locator and the ball socket of the aircraft component is calculated, that is, the feed amount of the ball head of the locator is determined. Thus, the position deviation is sent to the control system terminal through the transfer locator, and the force-position hybrid control model in the control system terminal calculates the motion signal and sends the motion signal to the transfer locator, so that the transfer locator pushes the ball head of the locator to completely enter the ball socket of the aircraft component according to the motion signal.

[0079] The technical solution of the embodiment of the present invention, by obtaining the ball socket image of the aircraft component, and then based on the target detection model, detecting the ball socket image of the aircraft component to obtain the target ball socket joint image, and obtaining the camera distortion coefficient. Thus, based on the camera distortion coefficient, the distortion of the target ball socket joint image is eliminated to obtain the ball socket joint image to be detected, and according to the Hough gradient method and the ball socket joint image to be detected, the pixel coordinates of the center of the ball socket joint are determined. Further, according to the initial positioning coordinates of the ball head and the ball socket, the pixel coordinates of the center of the ball socket joint, and the coordinate system transformation matrix, the feed amount of the ball head of the locator is calculated, and according to the feed amount of the ball head of the locator and the force-position hybrid control model, the ball head of the locator is controlled to completely enter the ball socket. In this solution, through the target detection network and the image preprocessing technology, the accurate position of the center of the ball socket joint can be mined, so as to maximize the elimination of the alignment error between the ball head of the locator and the ball socket based on the initial positioning coordinates of the ball head and the ball socket and the coordinate system transformation matrix, and obtain a more accurate feed amount of the ball head of the locator. And through the force-position hybrid control model, it can ensure that the ball head of the locator is in full contact with the ball socket without damaging the component, solving the problems of low efficiency, large error, low safety and easy component damage existing in the process of manually guiding the ball head to insert into the ball socket during the process of the large aircraft component being placed on the shelf. It can automatically guide the ball head to accurately insert into the ball socket during the process of the large aircraft component being placed on the shelf, without manual intervention, with strong safety, improving the placement efficiency, and can avoid component damage.

[0080] Embodiment III

[0081] Figure 4 The following is a schematic structural diagram of an aircraft component adaptive positioning device provided in Embodiment III of the present invention. As Figure 4 shown, the device includes:

[0082] An image acquisition module 310, configured to acquire an image of the ball socket of the aircraft component;

[0083] A pixel coordinate determination module 320, configured to detect the image of the ball socket of the aircraft component based on a target detection model to obtain an image of the target ball socket joint, and perform image preprocessing on the image of the target ball socket joint to obtain the pixel coordinates of the center of the ball socket joint;

[0084] A feed amount determination module 330, configured to calculate the feed amount of the ball head of the locator according to the initial positioning coordinates of the ball head and ball socket, the pixel coordinates of the center of the ball socket joint, and the coordinate system transformation matrix;

[0085] A positioning control module 340, configured to control the ball head of the locator to completely enter the ball socket according to the feed amount of the ball head of the locator and the force-position hybrid control model.

[0086] The technical solution of the embodiment of the present invention obtains an image of the ball socket of the aircraft component, and thus detects the image of the ball socket of the aircraft component based on a target detection model to obtain an image of the target ball socket joint, and performs image preprocessing on the image of the target ball socket joint to obtain the pixel coordinates of the center of the ball socket joint. Furthermore, according to the initial positioning coordinates of the ball head and ball socket, the pixel coordinates of the center of the ball socket joint, and the coordinate system transformation matrix, the feed amount of the ball head of the locator is calculated. Further, according to the feed amount of the ball head of the locator and the force-position hybrid control model, the ball head of the locator is controlled to completely enter the ball socket. In this solution, through the target detection network and image preprocessing technology, the precise position of the center of the ball socket joint can be excavated, so as to eliminate the alignment error between the ball head of the locator and the ball socket to the greatest extent based on the initial positioning coordinates of the ball head and ball socket and the coordinate system transformation matrix, and obtain a more accurate feed amount of the ball head of the locator. And through the force-position hybrid control model, it can be ensured that the ball head of the locator is in full contact with the ball socket without damaging the component, solving the problems of low efficiency, large error, low safety, and easy component damage existing in the process of manually guiding the ball head to insert into the ball socket during the process of lifting large aircraft components. It can automatically guide the ball head to accurately insert into the ball socket during the process of lifting large aircraft components, without manual intervention, with strong safety, improving the lifting efficiency, and avoiding component damage.

[0087] Optionally, the target detection model is a model constructed based on the single-stage detection model yolov5, the attention mechanism, and the weighted bidirectional feature pyramid network BiFPN.

[0088] Optionally, the pixel coordinate determination module 320 is configured to obtain the camera distortion coefficient; based on the camera distortion coefficient, eliminate the distortion of the target ball socket joint image to obtain the ball socket joint image to be detected; and determine the pixel coordinates of the center of the ball socket joint according to the Hough gradient method and the ball socket joint image to be detected.

[0089] Optionally, the pixel coordinate determination module 320 is configured to determine candidate center pixel points according to the Hough gradient method and the ball socket joint image to be detected; and determine the pixel coordinates of the center of the ball socket joint according to the cumulative voting mechanism and the candidate center pixel points.

[0090] Optionally, the feed amount determination module 330 is configured to multiply the pixel coordinates of the center of the ball socket joint by the coordinate system transformation matrix to obtain the coordinates to be calibrated; and determine the feed amount of the ball head of the locator according to the coordinates to be calibrated and the initial positioning coordinates of the ball head and the ball socket.

[0091] Optionally, the positioning control module 340 is configured to move the ball head of the locator to below the ball socket according to the feed amount of the ball head of the locator, and perform contact force feedback control on the ball head of the locator according to the force-position hybrid control model, so that the ball head of the locator completely enters the ball socket.

[0092] Optionally, the image acquisition module 310 is configured to move to the image acquisition position according to the indoor global positioning system iGPS and the rough positioning path planning data, and acquire the ball socket image of the aircraft component through the locator camera.

[0093] The aircraft component adaptive positioning device provided by the embodiments of the present invention can execute the aircraft component adaptive positioning method provided by any embodiment of the present invention, and has corresponding function modules and beneficial effects for executing the method.

[0094] Embodiment 4

[0095] Figure 5 FIG. shows a schematic structural diagram of an electronic device that can be used to implement the embodiments of the present invention. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0096] As Figure 5As shown, the electronic device 10 includes at least one processor 11 and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. The memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0097] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disc, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0098] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include but are not limited to a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the aircraft component adaptive positioning method.

[0099] In some embodiments, the aircraft component adaptive positioning method can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the aircraft component adaptive positioning method described above can be executed. Alternatively, in other embodiments, the processor 11 can be configured to execute the aircraft component adaptive positioning method in any other appropriate manner (e.g., by means of firmware).

[0100] The various embodiments of the systems and techniques described above in this specification can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from, and transmits data and instructions to, a storage system, at least one input device, and at least one output device.

[0101] The computer programs for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The computer programs can be executed entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

[0102] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0103] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the electronic device. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0104] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0105] The computing system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system and solves the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0106] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is made herein.

[0107] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An adaptive positioning method for aircraft components, characterized in that, it includes: Obtain the spherical socket image of the aircraft component; Based on the target detection model, detect the spherical socket image of the aircraft component to obtain the target spherical socket joint image, and perform image preprocessing on the target spherical socket joint image to obtain the pixel coordinates of the center of the spherical socket joint; Calculate the feed amount of the locator ball head according to the initial positioning coordinates of the ball head and spherical socket, the pixel coordinates of the center of the spherical socket joint, and the coordinate system transformation matrix; Control the locator ball head to fully enter the spherical socket according to the feed amount of the locator ball head and the force-position hybrid control model.

2. The method according to claim 1, characterized in that, The target detection model is a model constructed based on the single-stage detection model yolov5, the attention mechanism, and the weighted bidirectional feature pyramid network BiFPN.

3. The method according to claim 1, characterized in that, The image preprocessing of the target spherical socket joint image to obtain the pixel coordinates of the center of the spherical socket joint includes: Obtain the camera distortion coefficient; Based on the camera distortion coefficient, eliminate the distortion of the target spherical socket joint image to obtain the spherical socket joint image to be detected; Determine the pixel coordinates of the center of the spherical socket joint according to the Hough gradient method and the spherical socket joint image to be detected.

4. The method according to claim 3, characterized in that, The determination of the pixel coordinates of the center of the spherical socket joint according to the Hough gradient method and the spherical socket joint image to be detected includes: Determine the candidate center pixel points according to the Hough gradient method and the spherical socket joint image to be detected; Determine the pixel coordinates of the center of the spherical socket joint according to the cumulative voting mechanism and the candidate center pixel points.

5. The method according to claim 1, characterized in that, The calculation of the feed amount of the locator ball head according to the initial positioning coordinates of the ball head and spherical socket, the pixel coordinates of the center of the spherical socket joint, and the coordinate system transformation matrix includes: Multiply the pixel coordinates of the center of the spherical socket joint by the coordinate system transformation matrix to obtain the coordinates to be calibrated; Determine the feed amount of the locator ball head according to the coordinates to be calibrated and the initial positioning coordinates of the ball head and spherical socket.

6. The method according to claim 1, characterized in that, The control of the locator ball head to fully enter the spherical socket according to the feed amount of the locator ball head and the force-position hybrid control model includes: Move the locator ball head to below the spherical socket according to the feed amount of the locator ball head, and perform contact force feedback control on the locator ball head according to the force-position hybrid control model so that the locator ball head fully enters the spherical socket.

7. The method according to claim 1, characterized in that, The obtaining of the spherical socket image of the aircraft component includes: According to the indoor global positioning system iGPS and the rough positioning path planning data, move to the image acquisition position, and collect the spherical socket image of the aircraft component through the locator camera.

8. An adaptive positioning device for aircraft components, characterized in that, it includes: An image acquisition module for obtaining the spherical socket image of the aircraft component; A pixel coordinate determination module, configured to detect the ball socket image of the aircraft component based on a target detection model, obtain a target ball socket joint image, and perform image preprocessing on the target ball socket joint image to obtain the pixel coordinates of the center of the ball socket joint; A feed amount determination module, configured to calculate the feed amount of the ball head of the locator according to the initial positioning coordinates of the ball head and the ball socket, the pixel coordinates of the center of the ball socket joint, and a coordinate system transformation matrix; An in-position control module, configured to control the ball head of the locator to completely enter the ball socket according to the feed amount of the ball head of the locator and a force-position hybrid control model.

9. An electronic device, characterized in that, the electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the aircraft component adaptive in-position method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, the computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the aircraft component adaptive in-position method according to any one of claims 1-7 when executed by a processor.

Images