Hole making operation method and device, hole making robot and storage medium

Abstract

The invention discloses a hole making operation method and device, a hole making robot and a storage medium, and relates to the technical field of aircraft assembling.The method comprises the steps that image information of a reference hole is obtained through a visual function assembly, and position information of a to-be-made hole is obtained according to the image information of the reference hole; according to the position information of the to-be-formed hole, a normal correction assembly is moved to the position above the to-be-formed hole, so that a normal correction posture is obtained through the normal correction assembly; and according to the normal correction posture and the position information of the to-be-drilled hole, drilling operation is completed through a drilling assembly. According to the technical scheme provided by the embodiment of the invention, the labor cost consumed by drilling is saved, the misoperation phenomenon caused by manual drilling is avoided, meanwhile, the position adjustment and posture adjustment during drilling operation are realized, the accurate position and posture of the drilling main shaft are ensured, the drilling process is greatly improved, and the drilling quality of the drilling robot is improved.

Description

Technical Field

[0001] The invention relates to the technical field of aircraft assembly, and in particular to a hole making method, a hole making device, a hole making robot and a storage medium. Background Art

[0002] In the aircraft manufacturing industry, the labor of aircraft assembly accounts for a large proportion, and riveting, as the most widely used assembly method, requires hole making as its primary process. Therefore, hole making operations in aircraft fuselages and component positions have become an indispensable manufacturing process in the aircraft manufacturing process.

[0003] In the prior art, for drilling holes in aircraft fuselages and component locations, a hole-making robot is usually controlled to reach each hole-making point in sequence based on a preset hole-making trajectory and a preset working position, and the hole-making operation is performed at the designated working position of each hole-making point, or a semi-manual participation method is used, in which the staff completes the addressing and positioning operations of the hole-making robot.

[0004] However, in the above-mentioned hole-making method, there are often large deviations in the working position and body posture of the hole-making robot, which often makes it difficult for the hole-making quality to meet the assembly requirements. The semi-manual participation method requires a lot of manpower costs, and manual operation errors often occur, which in turn affects the hole-making quality. Summary of the invention

[0005] The invention provides a hole making operation method, a device, a hole making robot and a storage medium to solve the problem of poor hole making quality of the hole making robot.

[0006] According to one aspect of the present invention, there is provided a hole making method, comprising:

[0007] Acquire image information of the reference hole through the visual function component, and acquire position information of the hole to be made according to the image information of the reference hole;

[0008] According to the position information of the hole to be made, the normal correction component is moved to the top of the hole to be made, so as to obtain the normal correction posture through the normal correction component;

[0009] According to the normal correction posture and the position information of the hole to be made, the hole making operation is completed by the punching assembly.

[0010] If the visual function component includes a binocular vision component; the method of obtaining the image information of the reference hole through the visual function component and obtaining the position information of the hole to be made according to the image information of the reference hole includes: obtaining the image information of the reference hole through the binocular vision component and obtaining the three-dimensional coordinates of the hole to be made according to the image information of the reference hole. Therefore, when the normal correction component is moved above the hole to be made, the three-dimensional coordinates of the hole to be made can not only make the normal correction component above the hole to be made, but also maintain the depth distance in the vertical direction, providing a detection space for the normal correction component to obtain the normal correction posture.

[0011] The method of obtaining the image information of the reference hole through the binocular vision component and obtaining the three-dimensional coordinates of the hole to be made based on the image information of the reference hole includes: obtaining a set of associated feature points through the binocular vision component; wherein the set of associated feature points includes multiple associated feature points; obtaining the depth information of each associated feature point, and obtaining the three-dimensional coordinates of each associated feature point based on the depth information of each associated feature point; obtaining the three-dimensional coordinates of the reference hole based on the three-dimensional coordinates of each associated feature point and the obtained spatial circle based on the fitting, and obtaining the three-dimensional coordinates of the hole to be made based on the three-dimensional coordinates of the reference hole. Thus, the accurate three-dimensional coordinates of the reference hole are obtained through the above technical solution, which further improves the positioning accuracy of the hole-making robot.

[0012] If the normal correction component includes a cross laser emitter; the normal correction component is moved to the top of the hole to be made according to the position information of the hole to be made, so as to obtain the normal correction posture through the normal correction component, including: according to the position information of the hole to be made, the cross laser emitter is moved to the top of the hole to be made, so as to obtain the normal correction posture through the cross laser emitter. Thus, the normal correction posture is obtained through the cross laser emitter, which simplifies the terminal structure of the hole-making robot compared to the multiple laser displacement sensors installed at the position of the through-hole device, and avoids the structural error during installation.

[0013] According to the position information of the hole to be made, the cross laser emitter is moved to the top of the hole to be made, so as to obtain the normal correction posture through the cross laser emitter, including: obtaining the straight line equation group constituting the cross laser stripes according to the cross laser stripes emitted by the cross laser emitter; obtaining the fitting space plane where the hole to be made is located according to the straight line equation group, and obtaining the normal vector of the hole to be made according to the fitting space plane. Thus, the accurate normal correction posture is obtained through the above technical solution, and the posture adjustment accuracy of the hole-making robot is further improved.

[0014] If the normal correction component includes a cross laser emitter and a laser displacement sensor; the normal correction component is moved to the top of the hole to be made according to the position information of the hole to be made, so as to obtain the normal correction posture through the normal correction component, including: according to the position information of the hole to be made, the cross laser emitter is moved to the top of the hole to be made, so as to obtain the initial correction posture through the cross laser emitter; according to the initial correction posture and the position information of the hole to be made, the laser displacement sensor is moved to the top of the hole to be made, so as to obtain the normal correction posture through the laser displacement sensor. Thus, through the posture adjustment twice before and after, the posture accuracy of the punching spindle when performing the hole making operation is further ensured, and the hole making process of the hole making robot is improved.

[0015] According to another aspect of the present invention, there is provided a hole making device, comprising:

[0016] A position information acquisition module is used to acquire image information of the reference hole through a visual function component, and acquire position information of the hole to be made based on the image information of the reference hole;

[0017] A normal correction posture acquisition module, used for moving the normal correction component to the top of the hole to be made according to the position information of the hole to be made, so as to acquire the normal correction posture through the normal correction component;

[0018] The hole making operation execution module is used to complete the hole making operation through the punching component according to the normal correction posture and the position information of the hole to be made.

[0019] According to another aspect of the present invention, a hole-making robot is provided, comprising: a visual function component, a normal correction component and a punching component; the visual function component comprises a monocular vision component or a binocular vision component; the normal correction component comprises a cross laser emitter or a laser displacement sensor; the hole-making robot is used to perform the hole-making operation method described in Example 1 of the present invention.

[0020] According to another aspect of the present invention, a hole-making robot is provided, comprising: a visual function component, a normal correction component and a punching component; the visual function component comprises a monocular vision component or a binocular vision component; the normal correction component comprises a cross laser emitter and a laser displacement sensor; the hole-making robot is used to perform the hole-making operation method described in Example 2 of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the hole-making method described in any embodiment of the present invention when executed by a processor.

[0022] The technical solution of the embodiment of the present invention obtains the image information of the reference hole through the visual function component, and obtains the position information of the hole to be made based on the image information of the reference hole; according to the position information of the hole to be made, the normal correction component is moved to the top of the hole to be made, so as to obtain the normal correction posture through the normal correction component; according to the normal correction posture and the position information of the hole to be made, the hole making operation is completed by the punching component. It not only saves the manpower cost of hole making and avoids the misoperation phenomenon of manual hole making, but also realizes the position adjustment and posture adjustment during the hole making operation, ensures the accurate position and posture of the punching spindle, greatly improves the hole making process, and improves the hole making quality of the hole making robot.

[0023] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1A is a flow chart of a hole making method provided according to the first embodiment of the present invention;

[0026] Figure 1B is a structural diagram of a punching assembly provided according to Embodiment 1 of the present invention;

[0027] Figure 1C is a schematic diagram of the positions of the punching spindle and the holes to be punched according to the first embodiment of the present invention;

[0028] Figure 1D is a structural diagram of another punching assembly provided according to Embodiment 1 of the present invention;

[0029] Figure 2A is a flow chart of another hole making method provided according to the second embodiment of the present invention;

[0030] Figure 2B is a structural diagram of a punching assembly according to the second embodiment of the present invention;

[0031] Figure 3 This is a structural block diagram of a hole-making operation device provided according to Embodiment 3 of the present invention;

[0032] Figure 4is a structural block diagram of a hole-making robot provided according to a fourth embodiment of the present invention;

[0033] Figure 5 It is a structural block diagram of a hole-making robot provided according to Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0036] Embodiment 1

[0037] Figure 1A This is a flowchart of a hole-making method provided in the first embodiment of the present invention. This embodiment is applicable to the case where a hole-making robot including a visual function component and a normal correction component completes the hole-making operation. The method can be executed by a hole-making operation device, which can be implemented in the form of hardware and / or software. The hole-making operation device is configured in the hole-making robot, and the hole-making robot can be used in aircraft assembly operations to punch holes in aircraft fuselages or aircraft parts. Figure 1A As shown, the method includes:

[0038] S101, obtaining image information of a reference hole through a visual function component, and obtaining position information of a hole to be made according to the image information of the reference hole.

[0039] The reference hole is a standard hole prefabricated on the aircraft surface by manual labor or other methods. When performing aircraft assembly operations, a reference hole can be pre-drilled at one end of the aircraft by pre-operation, and the reference hole is used as the starting point for hole making. The hole making operations of other holes (i.e. holes to be made) are completed in sequence along the preset trajectory. The visual function component is a functional component of the hole making robot for image acquisition, image processing and image recognition.

[0040] Taking the visual function component including the monocular vision component as an example, after the hole-making robot obtains the image information of the reference hole through the monocular camera at the reference hole position, the image information is first post-processed, for example, sharpening processing, contrast adjustment processing, saturation adjustment processing, blurring processing, de-reflection processing and de-noising processing; secondly, the image information is edge extracted based on an adaptive threshold; then the region of interest (ROI) is further extracted from the edge extraction result; then the Histogram of Oriented Gradient (HOG) feature of the image information is obtained according to the extracted region of interest; finally, the HOG feature is obtained, and the position information of the reference hole in the reference hole image is determined by recognizing the HOG feature.

[0041] Since the reference hole serves as the starting point for drilling each hole to be made, and the other holes to be made are all located on the preset trajectory, after obtaining the position information of the reference hole, the position information of each hole to be made can be obtained according to the preset trajectory; in particular, the hole-making robot can only obtain a two-dimensional image through a monocular camera, so the position information of the reference hole obtained is also a two-dimensional coordinate, that is, from the viewing angle of the visual function component, the horizontal and vertical coordinates of the reference hole can be obtained, but the depth information in the vertical direction cannot be obtained. On this basis, the position information of the holes to be made obtained is obviously also a two-dimensional coordinate.

[0042] Optionally, in an embodiment of the present invention, if the visual function component includes a binocular vision component; the obtaining of the image information of the reference hole through the visual function component, and obtaining the position information of the hole to be made according to the image information of the reference hole, includes: obtaining the image information of the reference hole through the binocular vision component, and obtaining the three-dimensional coordinates of the hole to be made according to the image information of the reference hole. The binocular vision component includes two monocular cameras, which fuse the images taken by the two monocular cameras at the same time, and can obtain the depth information in the vertical direction while obtaining the plane coordinates of the reference hole, that is, the depth information between the reference hole and the visual function component, that is, the three-dimensional coordinates of the reference hole, and then the three-dimensional coordinates of the hole to be made can be obtained according to the three-dimensional coordinates of the reference hole. Therefore, when the normal correction component is moved to the top of the hole to be made, the three-dimensional coordinates of the hole to be made can not only make the normal correction component above the hole to be made, but also maintain the depth distance in the vertical direction, providing a detection space for the normal correction component to obtain the normal correction posture.

[0043] Optionally, in an embodiment of the present invention, acquiring the image information of the reference hole through a binocular vision component and acquiring the three-dimensional coordinates of the hole to be made based on the image information of the reference hole includes: acquiring a set of associated feature points through a binocular vision component; wherein the set of associated feature points includes multiple associated feature points; acquiring depth information of each associated feature point, and acquiring the three-dimensional coordinates of each associated feature point based on the depth information of each associated feature point; acquiring the three-dimensional coordinates of the reference hole based on the three-dimensional coordinates of each associated feature point and the spatial circle obtained by fitting, and acquiring the three-dimensional coordinates of the hole to be made based on the three-dimensional coordinates of the reference hole.

[0044] Specifically, the binocular vision component is composed of two symmetrically placed monocular cameras, namely, a first monocular camera and a second monocular camera; the first image and the second image at the same time are respectively acquired through the first monocular camera and the second monocular camera; after the first image and the second image are processed as above and the HOG features are acquired, N pairs of feature points with similar features on the same horizontal line in the first image and the second image are first acquired, namely, associated feature points A i and B i Among them, A i is the feature point in the first image, B i is the feature point in the second image, i represents the i-th pair of feature points, i=1, 2, ..., N; the value of N can be configured as needed; secondly, the disparity between the two corresponding feature points is obtained, that is, d i =X Ai -X Bi ; Among them, X Ai is the feature point A in the first image i The horizontal axis, X Bi is the feature point B in the second image i The horizontal axis of .

[0045] Then according to the stereo matching relationship The depth value Z of each feature point can be obtained i ; Wherein, b is the distance between the optical centers of the first monocular camera and the second monocular camera, and f is the focal length of the first monocular camera and the second monocular camera); then according to the depth information Z of each feature point i , and the pinhole camera model Get the three-dimensional coordinates (X i ,Y i ,Z i ), where x c and c is the pixel coordinate in the camera coordinate system, and K is the camera intrinsic parameter matrix.

[0046] Finally, since the reference hole is usually located on a curved surface with a small curvature, the curved surface can be approximated as a plane for processing, and the edge of the reference hole is circular, according to the three-dimensional coordinates (X i ,Y i ,Z i ) is fitted into a space circle, and the coordinates of the center of the circle (X o ,Y o ,Z o ), the center coordinate (X o ,Y o ,Z o ) is the three-dimensional coordinate of the center point of the reference hole. Thus, the accurate three-dimensional coordinate of the reference hole is obtained through the above technical solution, which further improves the positioning accuracy of the hole-making robot.

[0047] S102 . According to the position information of the hole to be made, move the normal correction component to the top of the hole to be made, so as to obtain a normal correction posture through the normal correction component.

[0048] The normal direction is the direction of the normal line. In the embodiment of the present invention, the normal direction is the direction perpendicular to the plane where the hole to be made is located, and is also the feeding direction of the punching spindle of the punching assembly (i.e., the punching direction); based on the position information of the hole to be made obtained in the above technical solution, the normal correction assembly can be moved above the hole to be made; in the embodiment of the present invention, the normal correction assembly can include a laser displacement sensor; Figure 1BTaking the example of a normal correction component including four laser displacement sensors, the punching spindle of the punching component extends out through the through-hole device in the front to perform the punching operation, and the laser displacement sensors are located around the through-hole device; in particular, when the normal correction component is a laser displacement sensor, since the laser displacement sensors are located around the through-hole device, moving the normal correction component to the top of the hole to be made actually moves the punching spindle to the top of the hole to be made.

[0049] like Figure 1C As shown, there are four laser displacement sensors A, B, C, and D installed perpendicular to the plane in the feeding direction of the punching spindle T, and with the help of the above four laser displacement sensors, the normal vector of the plane A1B1C1D1 where the hole position T1 to be made is located is measured, and the normal vector reflects the distance of the hole to be made relative to the laser displacement sensor in the vertical direction, as well as the attitude angle of the hole to be made relative to the laser displacement sensor. At the same time, according to the physical structure of the hole-making robot, the relative position of the laser displacement sensor and the punching spindle can be known, thereby determining the depth distance of the punching spindle from the hole to be made in the depth direction, as well as the attitude angle of the punching spindle relative to the plane where the hole to be made is located. In particular, if the visual function component includes a binocular vision component, the depth information between the binocular vision component and the hole to be made has been obtained through the binocular vision component, and at this time, the attitude angle of the punching spindle relative to the plane where the hole to be made is located can be obtained through the laser displacement sensor.

[0050] Optionally, in an embodiment of the present invention, if the normal correction component includes a cross laser emitter; the step of moving the normal correction component to the top of the hole to be made according to the position information of the hole to be made, so as to obtain the normal correction posture through the normal correction component, includes: moving the cross laser emitter to the top of the hole to be made according to the position information of the hole to be made, so as to obtain the normal correction posture through the cross laser emitter. The normal correction component may also include a cross laser emitter, such as Figure 1D As shown, the cross laser emitter and the visual function component are both installed above the punching component, that is, when the punching component performs the punching operation vertically downward in a vertical posture, the visual function component and the cross laser emitter are both located on the left or right side of the punching component. The hole-making robot can also detect the normal vector of the position of the hole to be made through the cross laser stripes emitted by the cross laser emitter in the direction of the hole to be made. The normal vector reflects the depth distance between the cross laser emitter and the hole to be made in the depth direction, as well as the attitude angle of the cross laser emitter relative to the plane where the hole to be made is located. The normal correction attitude is thus obtained through the cross laser emitter. Compared with multiple laser displacement sensors installed at the position of the through-hole device, the terminal structure of the hole-making robot is simplified, and structural errors during installation are avoided.

[0051] Optionally, in an embodiment of the present invention, the cross laser emitter is moved to the top of the hole to be made according to the position information of the hole to be made, so as to obtain a normal correction posture through the cross laser emitter, including: obtaining a group of straight line equations constituting the cross laser stripes according to the cross laser stripes emitted by the cross laser emitter; obtaining a fitting spatial plane where the hole to be made is located according to the group of straight line equations, and obtaining the normal vector of the hole to be made according to the fitting spatial plane.

[0052] Specifically, based on the position information of the hole to be made obtained in the above technical solution, the center of the cross laser emitter is first moved above the position of the hole to be made, and then the three-dimensional coordinate information (x i ,y i ,z i ), and extract the cross laser stripes according to the image post-processing method, and then use the Hough transform method to fit the two straight line equations of the cross line A1x+B1y+C1=0 and A2x+B2y+C2=0. The coordinates of the intersection of the above two straight line equations are and The coordinates of the intersection point are also the coordinates of the center of the hole to be made.

[0053] After obtaining the center position of the hole to be made, the spatial plane Ax+By+Cz+D=0 is fitted near the center position of the hole to be made using the multiple three-dimensional points located near the center point on the two straight stripes included in the cross laser stripes. Finally, the normal of the spatial plane Ax+By+Cz+D=0 can be obtained by fitting the spatial plane using the random sampling consensus (RANSAC) method. This finding is the normal vector of the position of the hole to be made. Therefore, the accurate normal correction posture is obtained through the above technical solution, which further improves the posture adjustment accuracy of the hole-making robot.

[0054] S103, completing the hole making operation by a punching assembly according to the normal correction posture and the position information of the hole to be made.

[0055] As described in the above technical solution, if the normal correction component is a laser displacement sensor, then the punching spindle is facing the hole to be made. After the attitude angle is adjusted according to the normal correction posture, the punching operation can be directly performed through the punching component; if the normal correction component is a cross laser emitter, after the attitude angle is adjusted according to the current normal correction posture, the cross laser emitter is facing the hole to be made, and then according to the physical structure between the cross laser emitter and the punching spindle, the punching spindle is moved to the current position of the cross laser emitter with the same posture, that is, the position and attitude adjustment of the punching spindle is completed, and the punching operation can be directly performed through the punching component.

[0056] The technical solution of the embodiment of the present invention obtains the image information of the reference hole through the visual function component, and obtains the position information of the hole to be made based on the image information of the reference hole; according to the position information of the hole to be made, the normal correction component is moved to the top of the hole to be made, so as to obtain the normal correction posture through the normal correction component; according to the normal correction posture and the position information of the hole to be made, the hole making operation is completed by the punching component. It not only saves the manpower cost of hole making and avoids the misoperation phenomenon of manual hole making, but also realizes the position adjustment and posture adjustment during the hole making operation, ensures the accurate position and posture of the punching spindle, greatly improves the hole making process, and improves the hole making quality of the hole making robot.

[0057] Embodiment 2

[0058] Figure 2A This is a flow chart of a hole making method provided in the second embodiment of the present invention. In this embodiment, the posture adjustment is performed in sequence by a cross laser emitter and a laser displacement sensor. Figure 2A As shown, the method includes:

[0059] S201, obtaining image information of a reference hole through a visual function component, and obtaining position information of a hole to be made according to the image information of the reference hole.

[0060] S202, according to the position information of the hole to be made, move the cross laser emitter to the top of the hole to be made, so as to obtain an initial correction posture through the cross laser emitter.

[0061] like Figure 2B As shown, after the position information of the hole to be made is obtained through the visual function component, the cross laser transmitter is moved above the hole to be made, and the initial posture angle is determined by the initial correction posture obtained by the cross laser transmitter.

[0062] S203, according to the initial correction posture and the position information of the hole to be made, move the laser displacement sensor to the top of the hole to be made, so as to obtain the normal correction posture through the laser displacement sensor.

[0063] According to the above-mentioned initial attitude angle and the position information of the hole to be made, the laser displacement sensor and the punching spindle are moved together to the top of the hole to be made so that the punching spindle is directly opposite the hole to be made. At this time, the attitude adjustment angle is determined by the normal correction attitude obtained by the laser displacement sensor, that is, the attitude fine-tuning of the punching spindle is performed at this time to further ensure the accuracy of the attitude of the punching spindle.

[0064] S204, completing the hole making operation by a punching assembly according to the normal correction posture and the position information of the hole to be made.

[0065] After the punching spindle has completed the posture fine-tuning, the hole-making operation can be performed directly.

[0066] The technical solution of the embodiment of the present invention is that after obtaining the position information of the hole to be made, the cross laser emitter is first moved above the hole to be made to obtain the initial correction posture through the cross laser emitter, and then according to the initial correction posture and the position information of the hole to be made, the laser displacement sensor is moved above the hole to be made to obtain the normal correction posture through the laser displacement sensor. In this way, through the two posture adjustments, the posture accuracy of the punching spindle when performing the hole-making operation is further ensured, thereby improving the hole-making process of the hole-making robot.

[0067] Embodiment 3

[0068] Figure 3 : is a structural block diagram of a hole-making operation device provided in Embodiment 3 of the present invention, and the device specifically comprises:

[0069] The position information acquisition module 301 is used to acquire the image information of the reference hole through the visual function component, and acquire the position information of the hole to be made according to the image information of the reference hole;

[0070] The normal correction posture acquisition module 302 is used to move the normal correction component to the top of the hole to be made according to the position information of the hole to be made, so as to acquire the normal correction posture through the normal correction component;

[0071] The hole making operation execution module 303 is used to complete the hole making operation through the punching assembly according to the normal correction posture and the position information of the hole to be made.

[0072] The technical solution of the embodiment of the present invention obtains the image information of the reference hole through the visual function component, and obtains the position information of the hole to be made based on the image information of the reference hole; according to the position information of the hole to be made, the normal correction component is moved to the top of the hole to be made, so as to obtain the normal correction posture through the normal correction component; according to the normal correction posture and the position information of the hole to be made, the hole making operation is completed by the punching component. It not only saves the manpower cost of hole making and avoids the misoperation phenomenon of manual hole making, but also realizes the position adjustment and posture adjustment during the hole making operation, ensures the accurate position and posture of the punching spindle, greatly improves the hole making process, and improves the hole making quality of the hole making robot.

[0073] Optionally, if the visual function component includes a binocular vision component;

[0074] The position information acquisition module 301 is specifically used to acquire the image information of the reference hole through the binocular vision component, and acquire the three-dimensional coordinates of the hole to be made according to the image information of the reference hole.

[0075] Optionally, the position information acquisition module 301 is also specifically used to obtain a set of associated feature points through a binocular vision component; wherein the set of associated feature points includes multiple associated feature points; obtain the depth information of each associated feature point, and obtain the three-dimensional coordinates of each associated feature point based on the depth information of each associated feature point; obtain the three-dimensional coordinates of the reference hole based on the three-dimensional coordinates of each associated feature point and the spatial circle obtained by fitting, and obtain the three-dimensional coordinates of the hole to be made based on the three-dimensional coordinates of the reference hole.

[0076] Optionally, if the normal correction component includes a cross laser emitter;

[0077] The normal correction posture acquisition module 302 is specifically used to move the cross laser emitter to the top of the hole to be made according to the position information of the hole to be made, so as to acquire the normal correction posture through the cross laser emitter.

[0078] Optionally, the normal correction posture acquisition module 302 is also specifically used to obtain a group of straight line equations constituting the cross laser stripes based on the cross laser stripes emitted by the cross laser transmitter; obtain the fitting space plane where the hole to be made is located based on the group of straight line equations, and obtain the normal vector of the hole to be made based on the fitting space plane.

[0079] Optionally, if the normal correction component includes a cross laser transmitter and a laser displacement sensor;

[0080] The normal correction posture acquisition module 302 is also specifically used to move the cross laser emitter to the top of the hole to be made according to the position information of the hole to be made, so as to obtain the initial correction posture through the cross laser emitter; according to the initial correction posture and the position information of the hole to be made, move the laser displacement sensor to the top of the hole to be made, so as to obtain the normal correction posture through the laser displacement sensor.

[0081] The above device can execute the hole making method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not described in detail in this embodiment, please refer to the hole making method provided by any embodiment of the present invention.

[0082] Embodiment 4

[0083] Figure 4 A structural schematic diagram of a hole-making robot provided in Example 4 of the present invention, the hole-making robot includes: a visual function component 401, a normal correction component 402 and a punching component 403; the visual function component 401 includes a monocular vision component or a binocular vision component; the normal correction component 402 includes a cross laser emitter or a laser displacement sensor; the hole-making robot is used to perform the hole-making operation method described in Example 1.

[0084] The technical solution of the embodiment of the present invention obtains the image information of the reference hole through the visual function component, and obtains the position information of the hole to be made according to the image information of the reference hole; according to the position information of the hole to be made, the normal correction component is moved to the top of the hole to be made, so as to obtain the normal correction posture through the normal correction component. This simplifies the hardware structure of the hole-making robot while realizing the automated execution of the hole-making operation.

[0085] Embodiment 5

[0086] Figure 5 A structural schematic diagram of a hole-making robot provided in Example 5 of the present invention, the hole-making robot includes: a visual function component 501, a normal correction component 502 and a punching component 503; the visual function component 501 includes a monocular vision component or a binocular vision component; the normal correction component 502 includes a cross laser emitter 504 and a laser displacement sensor 505; the hole-making robot is used to perform the hole-making operation method described in Example 2.

[0087] The technical solution of the embodiment of the present invention is that after obtaining the position information of the hole to be made, the cross laser emitter is first moved above the hole to be made to obtain the initial correction posture through the cross laser emitter, and then according to the initial correction posture and the position information of the hole to be made, the laser displacement sensor is moved above the hole to be made to obtain the normal correction posture through the laser displacement sensor, and the posture adjustment of the drilling spindle is completed by the cross laser emitter and the laser displacement sensor, which further ensures the accuracy of the posture of the drilling spindle when performing the hole making operation and improves the hole making process of the hole making robot.

[0088] Embodiment 6

[0089] In some embodiments, the hole making operation method can be implemented as a computer program, which is tangibly contained in a computer readable storage medium, such as a storage unit. In some embodiments, part or all of the computer program can be loaded and / or installed on a heterogeneous hardware accelerator via a ROM and / or a communication unit. When the computer program is loaded into the RAM and executed by the processor, one or more steps of the hole making operation method described above can be performed. Alternatively, in other embodiments, the processor can be configured to perform the hole making operation method in any other appropriate manner (for example, by means of firmware).

[0090] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0091] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0092] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0093] To provide interaction with a user, the systems and techniques described herein may be implemented on a heterogeneous hardware accelerator having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) through which a user can provide input to the heterogeneous hardware accelerator. Other types of devices may also be used to provide interaction with a user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0094] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0095] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0096] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0097] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A hole making method, characterized in that: include: Acquire image information of the reference hole through the visual function component, and acquire position information of the hole to be made according to the image information of the reference hole; According to the position information of the hole to be made, the normal correction component is moved to the top of the hole to be made, so as to obtain the normal correction posture through the normal correction component; According to the normal correction posture and the position information of the hole to be made, the hole making operation is completed by the punching assembly.

2. The method according to claim 1, characterized in that If the visual function component includes a binocular vision component; The method of obtaining the image information of the reference hole through the visual function component and obtaining the position information of the hole to be made according to the image information of the reference hole includes: The image information of the reference hole is obtained through the binocular vision component, and the three-dimensional coordinates of the hole to be made are obtained based on the image information of the reference hole.

3. The method according to claim 2, characterized in that The method of obtaining the image information of the reference hole through the binocular vision component and obtaining the three-dimensional coordinates of the hole to be made according to the image information of the reference hole includes: Acquire a set of associated feature points through a binocular vision component; wherein the set of associated feature points includes a plurality of associated feature points; Obtaining depth information of each associated feature point, and obtaining three-dimensional coordinates of each associated feature point according to the depth information of each associated feature point; According to the three-dimensional coordinates of each associated feature point, based on the space circle obtained by fitting, the three-dimensional coordinates of the reference hole are obtained, and the three-dimensional coordinates of the hole to be made are obtained according to the three-dimensional coordinates of the reference hole.

4. The method according to claim 1, characterized in that: If the normal correction component includes a cross laser transmitter; The step of moving the normal correction component to the top of the hole to be made according to the position information of the hole to be made, so as to obtain the normal correction posture through the normal correction component, comprises: According to the position information of the hole to be made, the cross laser transmitter is moved above the hole to be made, so as to obtain the normal correction posture through the cross laser transmitter.

5. The method according to claim 4, characterized in that The method of moving the cross laser emitter to the top of the hole to be made according to the position information of the hole to be made, so as to obtain the normal correction posture through the cross laser emitter, comprises: According to the cross laser stripes emitted by the cross laser transmitter, a group of straight line equations constituting the cross laser stripes are obtained; The fitting space plane where the hole to be made is located is obtained according to the linear equation group, and the normal vector of the hole to be made is obtained according to the fitting space plane.

6. The method according to claim 1, characterized in that If the normal correction component includes a cross laser transmitter and a laser displacement sensor; The step of moving the normal correction component to the top of the hole to be made according to the position information of the hole to be made, so as to obtain the normal correction posture through the normal correction component, comprises: According to the position information of the hole to be made, the cross laser transmitter is moved to the top of the hole to be made, so as to obtain an initial correction posture through the cross laser transmitter; According to the initial correction posture and the position information of the hole to be made, the laser displacement sensor is moved above the hole to be made, so as to obtain the normal correction posture through the laser displacement sensor.

7. A hole making device, characterized in that: include: A position information acquisition module is used to acquire image information of the reference hole through a visual function component, and acquire position information of the hole to be made based on the image information of the reference hole; A normal correction posture acquisition module, used to move the normal correction component to the top of the hole to be made according to the position information of the hole to be made, so as to acquire the normal correction posture through the normal correction component; The hole making operation execution module is used to complete the hole making operation through the punching component according to the normal correction posture and the position information of the hole to be made.

8. A hole-making robot, characterized in that: The hole-making robot comprises: a visual function component, a normal correction component and a punching component; the visual function component comprises a monocular vision component or a binocular vision component; the normal correction component comprises a cross laser emitter or a laser displacement sensor; the hole-making robot is used to perform the hole-making operation method described in any one of claims 1-5.

9. A hole-making robot, characterized in that: The hole-making robot includes: a visual function component, a normal correction component and a punching component; the visual function component includes a monocular vision component or a binocular vision component; the normal correction component includes a cross laser emitter and a laser displacement sensor; the hole-making robot is used to perform the hole-making operation method described in claim 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the hole-making method according to any one of claims 1 to 6 when executed.

Images