Cross-shafting multi-camera calibration method and system in large product guide scene, equipment and medium

Abstract

The invention provides a cross-shafting multi-camera calibration method and system in a large product guide scene, equipment and a medium. The method comprises the following steps: controlling a plurality of upper cameras to acquire a first film image; associating the plurality of upper camera coordinates according to the first film image; calibrating a plurality of lower cameras by using a second film; one upper camera is controlled to collect a second film image; associating the upper camera and the plurality of lower cameras into the same servo module according to a second film image acquired by the upper camera; and in combination with the coordinate relationship of the plurality of upper cameras, obtaining the coordinate relationship of the plurality of upper cameras and the plurality of lower cameras. According to the invention, the calibration of multiple cameras is realized by using two different specifications of films, and compared with the traditional separate calibration of a single camera, the method has the advantages of convenient operation, short flow time and high calibration precision, and meets the requirements of high product yield, few abnormal alarms and high production utilization in the factory production process.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a cross-axis multi-camera calibration method, system, equipment, and medium in a large product guidance scenario. Background Art

[0002] In the field of machine vision, the assembly of large products usually requires taking pictures with multiple cameras, moving the servo module multiple times, and aligning multiple times to achieve functions. If the equipment has abnormal conditions, such as collisions, loose camera screws, etc., the machine assembly accuracy will become low, or even the assembly function cannot be achieved. In this case, production needs to be resumed in the shortest possible time. The biggest difficulty in resuming production is that multiple cameras need to be calibrated separately, the steps are relatively cumbersome, and the compensation value needs to be adjusted after calibration, which takes a long time.

[0003] Therefore, it is necessary to optimize the cross-axis multi-camera calibration method in large product guidance scenarios to improve the above problems. Summary of the invention

[0004] In order to achieve the above-mentioned purpose and other advantages of the present invention, the first purpose of the present invention is to provide a cross-axis multi-camera calibration method in a large product guidance scenario, comprising the following steps:

[0005] Controlling the plurality of upper cameras to acquire a first film image;

[0006] associating a plurality of upper camera coordinates according to the first film image;

[0007] Calibrate multiple lower cameras using a second film;

[0008] Controlling one of the upper cameras to acquire a second film image;

[0009] Associating the upper camera and the plurality of lower cameras into the same servo module according to the second film image acquired by the upper camera;

[0010] The coordinate relationship of the multiple upper cameras is combined to obtain the coordinate relationship of the multiple upper cameras and the multiple lower cameras.

[0011] Furthermore, before the step of controlling the plurality of upper cameras to capture the first film image, the method further includes:

[0012] Placing the first film sheet at an assembly position;

[0013] Controlling the suction nozzle of the servo module to suck the first film sheet;

[0014] The cylinder is controlled to be lifted to ensure that the first film sheet is within the field of view of the plurality of upper cameras.

[0015] Furthermore, the step of controlling the plurality of upper cameras to capture the first film image includes:

[0016] Controlling the plurality of upper cameras to simultaneously or sequentially acquire the first film image.

[0017] Furthermore, the step of associating a plurality of upper camera coordinates according to the first film image comprises:

[0018] Extracting the coordinates of corner points on the first film in each first film image;

[0019] The extracted corner point coordinates are used as feature points, and the feature points captured by the multiple upper cameras are paired according to the feature points of each first film image;

[0020] Using paired feature points, calculate the extrinsic matrix between multiple upper cameras;

[0021] The external parameter matrix is ​​converted from world coordinates to camera coordinates to associate multiple upper camera coordinates.

[0022] Furthermore, before the step of calibrating the second films for the multiple lower cameras, the method further includes:

[0023] Controlling the suction nozzle to suck the second film;

[0024] The XY axis is controlled to move the second film into the field of view of the multiple lower cameras.

[0025] Furthermore, the step of calibrating the second film sheets for the multiple lower cameras includes:

[0026] Use one of the lower cameras as the master camera and the other lower cameras as slave cameras for multi-point calibration;

[0027] Each time a point is moved, the pixel coordinates and axis coordinates of the point are recorded until all points have moved and then rotation calibration is performed;

[0028] The rotation axis is controlled to rotate in a preset direction, and the image rotation angle and the rotation axis rotation angle are recorded each time the rotation is performed.

[0029] Furthermore, before the step of controlling one of the upper cameras to capture the second film image, the method further includes:

[0030] The servo module is moved to place the second film under the field of view of one of the upper cameras.

[0031] Furthermore, the step of associating the upper camera and the plurality of lower cameras into the same servo module according to the second film image captured by the upper camera includes:

[0032] According to the second film image acquired by the upper camera, an external parameter matrix of the upper camera and the plurality of lower cameras is calculated;

[0033] The coordinate system relationship between each lower camera and the upper camera is calculated according to the external parameter matrix of the upper camera and the plurality of lower cameras, so as to associate the upper camera and the plurality of lower cameras into the same servo module.

[0034] The second object of the present invention is to provide a cross-axis multi-camera calibration system in a large product guidance scenario, which implements the above method, including multiple upper cameras, multiple lower cameras, a first film, a second film, and a main controller, wherein the multiple upper cameras are installed above the assembly position; wherein,

[0035] The upper camera is used to capture the first film image and the second film image;

[0036] The lower camera is used to capture the second film image;

[0037] The main controller is used to associate the coordinates of multiple upper cameras according to the first film image, calibrate the lower camera according to the second film image, and calculate the coordinate relationship of the multiple upper cameras and the multiple lower cameras according to the second film image captured by the upper camera combined with the calibration relationship of the multiple lower cameras and the coordinate relationship of the multiple upper cameras.

[0038] The third object of the present invention is to provide an electronic device, comprising: a memory on which a program code is stored; a processor connected to the memory and implementing the above method when the program code is executed by the processor.

[0039] A fourth object of the present invention is to provide a computer-readable storage medium having program instructions stored thereon, wherein the program instructions implement the above method when executed by a processor.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The present invention realizes the calibration of multiple cameras by using two films of different specifications. A large-size film is used to associate multiple upper cameras that are far apart, and a small-size film is used to associate multiple lower cameras that are close together. Finally, multiple cameras are associated with the same module coordinate system. Compared with the traditional single camera separate calibration, the present invention has the advantages of convenient operation, short process time and high calibration accuracy, which meets the requirements of high product yield, few abnormal alarms and high production utilization rate in the factory production process.

[0042] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention in conjunction with the accompanying drawings. The specific implementation of the present invention is given in detail by the following embodiments and their accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0044] Figure 1 This is a flow chart of the cross-axis multi-camera calibration method in the large product guidance scenario of Example 1;

[0045] Figure 2 This is a flow chart of controlling the movement of the first film in Example 1;

[0046] Figure 3 The upper camera coordinate association flow chart of Example 1;

[0047] Figure 4 This is a flow chart of controlling the movement of the second film in Example 1;

[0048] Figure 5 This is a flow chart of the lower camera calibration in Example 1;

[0049] Figure 6 This is a flowchart of the association between the upper camera and the lower camera in Example 1;

[0050] Figure 7 This is a schematic diagram of a cross-axis multi-camera calibration system in a large product guidance scenario of Example 2;

[0051] Figure 8 This is a schematic diagram of first film image acquisition in Example 2;

[0052] Fig. 9 This is a schematic diagram of capturing the image of the second film in Example 2;

[0053] Fig.10 This is a schematic diagram of an electronic device according to Embodiment 3;

[0054] Fig.11 This is a schematic diagram of a computer-readable storage medium of Example 4.

[0055] In the figure: 1, upper camera; 11, first upper camera; 12, second upper camera; 2, lower camera; 21, first lower camera; 22, second lower camera; 3, nozzle; 4, first film; 5, second film. DETAILED DESCRIPTION

[0056] The present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. It should be noted that, under the premise of no conflict, the embodiments or technical features described below can be arbitrarily combined to form a new embodiment.

[0057] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making any creative work shall fall within the scope of protection of the present invention.

[0058] The figure numbers in this application are only used to distinguish the various steps in the scheme, and are not used to limit the execution order of the various steps. The specific execution order is subject to the description in the specification.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0060] Example 1

[0061] A multi-camera calibration method across axes in a large product guidance scenario, such as Figure 1 As shown, the following steps are included:

[0062] S1, controlling multiple upper cameras to capture a first film image;

[0063] For the convenience of description, two upper cameras are used as an example. Figure 7 , Figure 8 As shown, the two upper cameras 1 are respectively the first upper camera 11 and the second upper camera 12, and the first upper camera 11 and the second upper camera 12 are installed diagonally above the assembly position. Among them, the first film 4 can select a suitable size according to actual conditions. For example, this embodiment uses a 400mmX400mm film, which is only used as an example and should not be understood as a limitation on the size of the film. Put the 400mmX400mm film in the assembly position, vacuum suck the film, lift the cylinder, shoot the film with the camera, perform chessboard calibration, and associate the coordinates of the first upper camera 11 and the second upper camera 12.

[0064] In some embodiments, Figure 2 As shown, before the step of controlling the plurality of upper cameras to capture the first film image, the method further includes:

[0065] S100, placing the first film sheet at an assembly position;

[0066] S110, controlling the suction nozzle 3 of the servo module to suck the first film;

[0067] S120, control the cylinder to lift up to ensure that the first film is within the field of view of the plurality of upper cameras. Then the first upper camera 11 and the second upper camera 12 can be controlled to shoot the first film, perform chessboard calibration, and associate the coordinates of the first upper camera 11 and the second upper camera 12.

[0068] In some embodiments, the step of controlling the plurality of upper cameras to capture the first film image comprises:

[0069] Controlling the plurality of upper cameras to simultaneously or sequentially acquire the first film image.

[0070] S2, associating a plurality of upper camera coordinates according to the first film image;

[0071] In some embodiments, Figure 3 As shown, the step of associating a plurality of upper camera coordinates according to the first film image comprises:

[0072] S21, extracting the coordinates of corner points on the first film in each first film image;

[0073] S22, using the extracted corner point coordinates as feature points, and pairing the feature points captured by the multiple upper cameras according to the feature points of each first film image;

[0074] S23, using the paired feature points, calculating the extrinsic parameter matrix between the multiple upper cameras;

[0075] S24, converting the external parameter matrix from world coordinates to camera coordinates to associate multiple upper camera coordinates.

[0076] Taking two upper cameras as an example, calculate the external parameter matrix between the two upper cameras (including rotation matrix and translation vector), and then convert the world coordinates to camera coordinates P c =T -1 P W , P W is a point in the world coordinate system, P c is the corresponding point in the camera coordinate system, thereby associating the coordinates of the first upper camera 11 and the second upper camera 12

[0077] S3, calibrating the multiple lower cameras using the second film;

[0078] For the convenience of description, two lower cameras are used as an example. Figure 7 , Fig. 9 As shown, the two lower cameras 2 are respectively a first lower camera 21 and a second lower camera 22. The second film 5 can be selected in a suitable size according to actual conditions. For example, in this embodiment, a 200mmX50mm film is selected. This is only an example and should not be understood as a limitation on the film size.

[0079] In some embodiments, Figure 4 As shown, before the step of calibrating the second films for the multiple lower cameras, the method further includes:

[0080] S300, controlling the suction nozzle to suck the second film; for example, placing a 200mmX50mm film on the suction nozzle of the servo module, and opening the suction nozzle to vacuum suck the film.

[0081] S310, controlling the movement of the XY axis to move the second film sheet into the field of view of the plurality of lower cameras.

[0082] In some embodiments, Figure 5 As shown, the step of calibrating the second film sheets for the multiple lower cameras includes:

[0083] S31, use one of the lower cameras as the master camera and the other lower cameras as slave cameras to perform multi-point calibration; for example, perform 9-point calibration on the master and slave cameras, and if the field of view is larger, more points can be added, such as 12 points, 36 points, etc.

[0084] S32, each time a point moves, the pixel coordinates and axis coordinates of the point are recorded, and rotation calibration is performed after all points have moved. Taking the 9-point calibration of the master and slave cameras as an example, the movement trajectory is a 3*3 matrix, and each time a point moves, the pixel coordinates and axis (actual) coordinates of each point are recorded, and rotation calibration is performed after the 9-point movement is completed.

[0085] S33, control the rotation axis to rotate in a preset direction, and record the image rotation angle and the rotation axis rotation angle each time it rotates. For example, the rotation axis rotates in a clockwise (counterclockwise) direction for at least 3 times (not limited to 3 times, and can be set to 4 times, 5 times, etc. according to actual needs), and the rotation angle is fixed each time, and the image rotation angle and the (actual) rotation angle of the rotation axis are recorded and calibrated each time.

[0086] S4, controlling one of the upper cameras to capture a second film image;

[0087] In some embodiments, before the step of controlling one of the upper cameras to capture the second film image, the method further includes:

[0088] The servo module is moved to place the second film sheet under the field of view of one of the upper cameras. For example, the servo module is moved to place the 200mmX50mm film sheet under the field of view of the first upper camera 11, and then the first upper camera 11 is controlled to capture the 200mmX50mm film sheet image.

[0089] S5. Associating the upper camera and the plurality of lower cameras into the same servo module according to the second film image captured by the upper camera; for example, associating the first upper camera 11, the first lower camera 21 and the second lower camera 22 into the same servo module.

[0090] In some embodiments, Figure 6As shown, the step of associating the upper camera and the plurality of lower cameras into the same servo module according to the second film image captured by the upper camera includes:

[0091] S51, calculating the external parameter matrix of the upper camera and the plurality of lower cameras according to the second film image acquired by the upper camera;

[0092] S52, calculating the coordinate system relationship between each lower camera and the upper camera according to the external parameter matrix of the upper camera and the plurality of lower cameras, so as to associate the upper camera and the plurality of lower cameras into the same servo module.

[0093] S6. Combine the coordinate relationship of the multiple upper cameras to obtain the coordinate relationship of the multiple upper cameras and the multiple lower cameras. For example, after the first upper camera 11, the first lower camera 21, and the second lower camera 22 are associated with the same servo module, the coordinate relationship of the first upper camera 11 and the second upper camera 12 is combined to obtain the coordinate relationship of the first upper camera 11, the second upper camera 12, the first lower camera 21, and the second lower camera 22.

[0094] Taking two upper cameras and two lower cameras as an example, the external parameters T1, T3, T4 of the first upper camera 11, the first lower camera 21 and the second lower camera 22 are calculated, so as to obtain the coordinate system relationship between the first lower camera 21 and the first upper camera 11 Coordinate system relationship between the second lower camera 22 and the first upper camera 11 The first upper camera 11, the first lower camera 21 and the second lower camera 22 are associated with the same module, and the coordinate relationship between the first upper camera 11 and the second upper camera 12 obtained in the above steps is The coordinate relationship among the first upper camera 11 , the second upper camera 12 , the first lower camera 21 , and the second lower camera 22 can be obtained.

[0095] This embodiment realizes the calibration of multiple cameras by using two films of different specifications. A large-size film is used to associate multiple upper cameras that are far apart, and a small-size film is used to associate multiple lower cameras that are close together. Finally, multiple cameras are associated with the same module coordinate system. Compared with the traditional single camera calibration, it has the advantages of convenient operation, short process time and high calibration accuracy, and meets the requirements of high product yield, few abnormal alarms and high production utilization rate in the factory production process.

[0096] Example 2

[0097] Based on the same concept, this embodiment also provides a cross-axis multi-camera calibration system in a large product guidance scenario, which is used to implement the cross-axis multi-camera calibration method in the large product guidance scenario provided in Example 1. For a detailed description of the cross-axis multi-camera calibration method in the large product guidance scenario provided in Example 1, please refer to the corresponding description in Example 1 and will not be repeated here.

[0098] It is understandable that in order to achieve the above functions, the cross-axis multi-camera calibration system in the large product guidance scenario provided by this embodiment includes hardware structures and / or software modules corresponding to the execution of each function. In combination with the units and algorithm steps of the examples disclosed in this embodiment, this embodiment can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of this embodiment.

[0099] A multi-camera calibration system across axes in a large product guidance scenario, such as Figures 7 to 9 As shown, it includes a plurality of upper cameras 1, a plurality of lower cameras 2, a first film sheet 4, a second film sheet 5, and a main controller, wherein the plurality of upper cameras are installed above the assembly position; wherein,

[0100] The upper camera is used to capture the first film image and the second film image;

[0101] The lower camera is used to capture the second film image;

[0102] Figures 7 to 9 The calibration of two upper cameras and two lower cameras is shown. The two upper cameras 1 are respectively the first upper camera 11 and the second upper camera 12. The first upper camera 11 and the second upper camera 12 are installed diagonally above the assembly position. The two lower cameras 2 are respectively the first lower camera 21 and the second lower camera 22. The calibration of the first upper camera 11, the second upper camera 12, and the first upper camera 11 and the second upper camera 12 is achieved by using two different specifications of film sheets (i.e., the first film sheet 4 and the second film sheet 5).

[0103] The main controller is used to associate the coordinates of multiple upper cameras according to the first film image, calibrate the lower camera according to the second film image, and calculate the coordinate relationship of the multiple upper cameras and the multiple lower cameras according to the second film image captured by the upper camera combined with the calibration relationship of the multiple lower cameras and the coordinate relationship of the multiple upper cameras.

[0104] In some embodiments, the master controller may include the following process:

[0105] A first film image acquisition module, used for controlling the plurality of upper cameras to acquire a first film image;

[0106] An upper camera coordinate associating module, used to associate a plurality of upper camera coordinates according to the first film image;

[0107] A lower camera calibration module, used for calibrating a plurality of lower cameras using a second film;

[0108] A second film image acquisition module, used for controlling one of the upper cameras to acquire a second film image;

[0109] An upper and lower camera associating module, used for associating the upper camera and the plurality of lower cameras into the same servo module according to the second film image acquired by the upper camera;

[0110] The multi-camera association module is used to combine the coordinate relationship of multiple upper cameras to obtain the coordinate relationship of multiple upper cameras and multiple lower cameras.

[0111] In one embodiment, before the main controller controls the plurality of upper cameras to capture the first film image, the main controller further includes:

[0112] Placing the first film sheet at an assembly position;

[0113] Controlling the suction nozzle of the servo module to suck the first film sheet;

[0114] The cylinder is controlled to be lifted to ensure that the first film sheet is within the field of view of the plurality of upper cameras.

[0115] In one implementation, the upper camera coordinate association module controls the plurality of upper cameras to capture the first film image in the following manner:

[0116] Controlling the plurality of upper cameras to simultaneously or sequentially acquire the first film image.

[0117] In one implementation, the upper camera coordinate association module associates a plurality of upper camera coordinates according to the first film image in the following manner:

[0118] Extracting the coordinates of corner points on the first film in each first film image;

[0119] The extracted corner point coordinates are used as feature points, and the feature points captured by the multiple upper cameras are paired according to the feature points of each first film image;

[0120] Using paired feature points, calculate the extrinsic matrix between multiple upper cameras;

[0121] The external parameter matrix is ​​converted from world coordinates to camera coordinates to associate multiple upper camera coordinates.

[0122] In one embodiment, before the main controller calibrates the second films for the multiple lower cameras, it further includes:

[0123] Controlling the suction nozzle to suck the second film;

[0124] The XY axis is controlled to move the second film into the field of view of the multiple lower cameras.

[0125] In one implementation, the lower camera calibration module calibrates the second film sheets for the multiple lower cameras in the following manner:

[0126] Use one of the lower cameras as the master camera and the other lower cameras as slave cameras for multi-point calibration;

[0127] Each time a point is moved, the pixel coordinates and axis coordinates of the point are recorded until all points have moved and then rotation calibration is performed;

[0128] The rotation axis is controlled to rotate in a preset direction, and the image rotation angle and the rotation axis rotation angle are recorded each time the rotation is performed.

[0129] In one embodiment, before the main controller controls one of the upper cameras to capture the second film image, the main controller further includes:

[0130] The servo module is moved to place the second film under the field of view of one of the upper cameras.

[0131] In one embodiment, the upper and lower camera association module associates the upper camera and the plurality of lower cameras into the same servo module according to the second film image captured by the upper camera in the following manner:

[0132] According to the second film image acquired by the upper camera, an external parameter matrix of the upper camera and the plurality of lower cameras is calculated;

[0133] The coordinate system relationship between each lower camera and the upper camera is calculated according to the external parameter matrix of the upper camera and the plurality of lower cameras, so as to associate the upper camera and the plurality of lower cameras into the same servo module.

[0134] Regarding the main controller in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0135] It should be noted that the cross-axis multi-camera calibration system in the large product guidance scenario provided in this embodiment is similar to the principle and workflow of the cross-axis multi-camera calibration method in the large product guidance scenario provided in Example 1. The relevant parts can be referred to the above description and will not be repeated here.

[0136] This embodiment realizes the calibration of multiple cameras by using two films of different specifications. A large-size film is used to associate multiple upper cameras that are far apart, and a small-size film is used to associate multiple lower cameras that are close together. Finally, multiple cameras are associated with the same module coordinate system. Compared with the traditional single camera calibration, it has the advantages of convenient operation, short process time and high calibration accuracy, and meets the requirements of high product yield, few abnormal alarms and high production utilization rate in the factory production process.

[0137] Example 3

[0138] An electronic device, such as Fig.10 As shown, it includes: a memory on which program code is stored; a processor connected to the memory, and when the program code is executed by the processor, a method for calibrating multiple cameras across axes in a large product guidance scenario is implemented. For a detailed description of the method, reference can be made to the corresponding description in the above method embodiment, which will not be repeated here.

[0139] Example 4

[0140] A computer readable storage medium such as Fig.11 As shown, program instructions are stored thereon, and when the program instructions are executed by the processor, a method for calibrating multiple cameras across axes in a large product guidance scenario is implemented. For a detailed description of the method, reference may be made to the corresponding description in the above method embodiment, and no further description is given here.

[0141] The number of devices and processing scales described here are used to simplify the description of the present invention. Applications, modifications and variations of the present invention will be obvious to those skilled in the art.

[0142] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and implementation modes. They can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

[0143] The apparatus, electronic device, non-volatile computer storage medium and method provided in the embodiments of this specification correspond to each other, and therefore, the apparatus, electronic device and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, electronic device and non-volatile computer storage medium will not be repeated here.

[0144] Those skilled in the art also know that, in addition to implementing the controller in a purely computer-readable program code, the controller can be made to implement the same function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered as a hardware component, and the devices for implementing various functions included therein can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software units for implementing the method and structures within the hardware component.

[0145] The systems, devices or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above devices are described separately by functions in various units. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in the same or more software and / or hardware.

[0146] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0147] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0148] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0149] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0150] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0151] The specification may be described in the general context of computer-executable instructions executed by a computer, such as program units. Generally, program units include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program units may be located in local and remote computer storage media, including storage devices.

[0152] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0153] The above description is only an embodiment of this specification and is not intended to limit one or more embodiments of this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included in the scope of the claims of one or more embodiments of this specification.

Claims

1. A method for calibrating multiple cameras across axes in a large product guidance scenario, characterized in that: The following steps are involved: Controlling the plurality of upper cameras to acquire a first film image; associating a plurality of upper camera coordinates according to the first film image; Calibrate multiple lower cameras using a second film; Controlling one of the upper cameras to acquire a second film image; Associating the upper camera and the plurality of lower cameras into the same servo module according to the second film image acquired by the upper camera; The coordinate relationship of the multiple upper cameras is combined to obtain the coordinate relationship of the multiple upper cameras and the multiple lower cameras.

2. The method for calibrating multiple cameras across axes in a large product guidance scenario according to claim 1, characterized in that: Before the step of controlling the plurality of upper cameras to capture the first film image, the method further includes: Placing the first film sheet at an assembly position; Controlling the suction nozzle of the servo module to suck the first film sheet; The cylinder is controlled to be lifted to ensure that the first film sheet is within the field of view of the plurality of upper cameras.

3. The method for calibrating multiple cameras across axes in a large product guidance scenario according to claim 1, characterized in that: The step of controlling the plurality of upper cameras to capture the first film image comprises: Controlling the plurality of upper cameras to simultaneously or sequentially acquire the first film image.

4. The method for calibrating multiple cameras across axes in a large product guidance scenario as claimed in claim 3, characterized in that: The step of associating a plurality of upper camera coordinates according to the first film image comprises: Extracting the coordinates of corner points on the first film in each first film image; The extracted corner point coordinates are used as feature points, and the feature points captured by the multiple upper cameras are paired according to the feature points of each first film image; Using paired feature points, calculate the extrinsic matrix between multiple upper cameras; The external parameter matrix is ​​converted from world coordinates to camera coordinates to associate multiple upper camera coordinates.

5. The method for calibrating multiple cameras across axes in a large product guidance scenario as claimed in claim 2, characterized in that: Before the step of calibrating the second films for the multiple lower cameras, the method further includes: Controlling the suction nozzle to suck the second film; The XY axis is controlled to move the second film into the field of view of the multiple lower cameras.

6. The method for calibrating multiple cameras across axes in a large product guidance scenario as claimed in claim 4, characterized in that: The step of calibrating the second film sheets for the multiple lower cameras comprises: Use one of the lower cameras as the master camera and the other lower cameras as slave cameras for multi-point calibration; Each time a point is moved, the pixel coordinates and axis coordinates of the point are recorded until all points have moved and then rotation calibration is performed; The rotation axis is controlled to rotate in a preset direction, and the image rotation angle and the rotation axis rotation angle are recorded each time the rotation is performed.

7. The method for calibrating multiple cameras across axes in a large product guidance scenario as claimed in claim 5, characterized in that: Before the step of controlling one of the upper cameras to capture the second film image, the method further includes: The servo module is moved to place the second film under the field of view of one of the upper cameras.

8. The method for calibrating multiple cameras across axes in a large product guidance scenario as claimed in claim 6, characterized in that: The step of associating the upper camera and the plurality of lower cameras into the same servo module according to the second film image captured by the upper camera comprises: According to the second film image acquired by the upper camera, an external parameter matrix of the upper camera and the plurality of lower cameras is calculated; The coordinate system relationship between each lower camera and the upper camera is calculated according to the external parameter matrix of the upper camera and the plurality of lower cameras, so as to associate the upper camera and the plurality of lower cameras into the same servo module.

9. A cross-axis multi-camera calibration system in a large product guidance scene, implementing the method according to any one of claims 1 to 8, characterized in that: It includes a plurality of upper cameras, a plurality of lower cameras, a first film, a second film, and a main controller, wherein the plurality of upper cameras are installed above the assembly position; wherein, The upper camera is used to capture the first film image and the second film image; The lower camera is used to capture the second film image; The main controller is used to associate the coordinates of multiple upper cameras according to the first film image, calibrate the lower camera according to the second film image, and calculate the coordinate relationship of the multiple upper cameras and the multiple lower cameras according to the second film image captured by the upper camera combined with the calibration relationship of the multiple lower cameras and the coordinate relationship of the multiple upper cameras.

10. An electronic device, characterized in that: include: A memory having program code stored therein; A processor is connected to the memory, and when the program code is executed by the processor, the method according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium, characterized in that: Program instructions are stored thereon, and when the program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images