Visual alignment method, system and electronic device

Abstract

A visual centering method and its system and electronic device. The visual centering method comprises the steps of: obtaining target plate image data obtained by collecting target plate information of a device to be debugged via a reference module, wherein the reference module is installed on a tooling platform of the device to be debugged, and the target plate is within the field of view of the reference module; performing data processing on the target plate image data to obtain a deviation distance between the camera center of the reference module and the center of the target plate; and adjusting the relative position of the tooling platform and the center of the target plate in the device to be debugged based on the deviation distance until the deviation distance is less than or equal to a deviation threshold, so that the deviation distance between the camera center of the reference module and the center of the target plate can be calculated by means of vision plus software, so that the deviation value is visualized, which is helpful to improve the centering accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of module centering, and more specifically to a visual centering method and a system and electronic equipment thereof. Background Art

[0002] During the equipment assembly or module calibration equipment debugging, it is often necessary to meet the centering requirements or complete the preliminary centering operation. For example, during the module test, the module camera must be aligned with the center of the target board first, that is, the equipment must be debugged to align the center of the module camera with the center of the target board; for another example, the TOF module must first complete the alignment and debugging of the center of the receiving module (that is, the module camera) with the center of the light board at the light board test station to ensure that the center of the module camera is aligned with the center of the light board.

[0003] At present, in the light board testing position of the TOF module, a traditional method of centering is to open a small hole in the center of the light board, or first install a structural part in the center of the light board, and then open a small hole in the structural part; then, install a heavy hammer through the small hole, adjust the center of the module camera according to the drooping heavy hammer, and then complete the work of aligning the center of the module camera with the center of the heavy hammer by visual observation. However, on the one hand, since the deviation in this method of centering with the help of a heavy hammer cannot be digitized and can only be observed visually, this method of centering with the help of a heavy hammer cannot meet the accuracy requirements for scenes with high deviation requirements; on the other hand, this method of centering with the help of a heavy hammer is inconvenient to operate, especially the heavy hammer will shake with vibration to introduce large deviations. In addition, in places where space is small, this method of centering with the help of a heavy hammer takes a long time to disassemble and operate, resulting in low centering efficiency.

[0004] In order to overcome the deviation problem caused by the shaking of the heavy hammer with the vibration in the above method, another centering method is to process a circular hole at the installation position of the module camera, and insert a corrected cross laser in the circular hole to emit laser upward, and then adjust the position of the target plate or the module end fixture to achieve the purpose of centering. Although this centering method with the help of laser can solve the problems of the above centering method with the help of heavy hammer, such as inconvenient operation or shaking with vibration, it may still cause a large deviation in the centering accuracy due to installation errors. For example, the laser diffusion caused by the uneven installation surface of the laser or the uneven circular hole at the installation position makes the deviation of the centering accuracy very large. Summary of the invention

[0005] An advantage of the present invention is to provide a visual centering method and system and electronic equipment thereof, which can greatly improve the convenience and accuracy of centering operation.

[0006] Another advantage of the present invention is to provide a visual centering method and a system and electronic device thereof, wherein, in one embodiment of the present invention, the visual centering method can visualize the deviation value, which helps to improve the centering accuracy.

[0007] Another advantage of the present invention is to provide a visual centering method and a system and electronic device thereof, wherein, in one embodiment of the present invention, the visual centering method can greatly reduce the difficulty of the centering operation and can improve the efficiency of the centering operation.

[0008] Another advantage of the present invention is that it provides a visual centering method and its system and electronic device. In one embodiment of the present invention, the visual centering method can replace the traditional visual or structural part positioning method by a vision plus software method, which helps to improve the centering efficiency while improving the centering accuracy.

[0009] Another advantage of the present invention is to provide a visual centering method and a system and electronic device thereof, wherein, in one embodiment of the present invention, the visual centering method can meet the requirements of long-distance centering so as to take into account the centering requirements of various distances.

[0010] Another advantage of the present invention is that it provides a visual alignment method and its system and electronic device. In one embodiment of the present invention, the visual alignment method can be extended to some assembly environments where mechanical assembly requires alignment but is not easy to align using conventional mechanical methods, such as the assembly of shafts and holes or the docking of tracks.

[0011] Another advantage of the present invention is to provide a visual centering method and its system and electronic device, wherein, in one embodiment of the present invention, the visual centering method can complete the calculation of deviations, such as auxiliary laser ranging can calculate the alignment deviation between two tracks.

[0012] Another advantage of the present invention is to provide a visual alignment method and system and electronic device, wherein in order to achieve the above advantages, no expensive materials or complex structures are required in the present invention. Therefore, the present invention successfully and effectively provides a solution, not only providing a simple visual alignment method and system and electronic device, but also increasing the practicality and reliability of the visual alignment method and system and electronic device.

[0013] In order to achieve at least one of the above advantages or other advantages and invention purposes, the present invention provides a visual alignment method, comprising the steps of:

[0014] Acquire target plate image data obtained by collecting target plate information of a device to be debugged via a reference module, wherein the reference module is installed on a tooling platform of the device to be debugged, and the target plate is within the field of view of the reference module;

[0015] Processing the target plate image data to obtain a deviation distance between a camera center of the reference module and a center of the target plate; and

[0016] Based on the deviation distance, the relative position of the tooling platform and the center of the target plate in the device to be debugged is adjusted until the deviation distance is less than or equal to a deviation threshold.

[0017] According to an embodiment of the present application, the target plate is a target plate having a visual mark, and the visual mark is used to indicate the center of the target plate, so that the center position of the target plate can be fitted through the visual mark information in the target plate image data.

[0018] According to one embodiment of the present application, the visual mark of the target plate is selected from one of a point light source located at the center of the target plate, a regular pattern whose center coincides with the center of the target plate, a regular shape of the target plate, and multiple Make points whose geometric centers coincide with the center of the target plate.

[0019] According to one embodiment of the present application, the reference module is a calibrated TOF camera module or an RGB camera module.

[0020] According to an embodiment of the present application, the visual alignment method, before the step of acquiring target plate image data obtained by collecting target plate information of a device to be debugged via a reference module, wherein the reference module is installed on a tooling platform of the device to be debugged, and the target plate is within the field of view of the reference module, further comprises the steps of:

[0021] The parallelism between the installation plane of the tooling platform and the calibration plane of the target plate is adjusted so that the installation plane of the tooling platform is parallel to the calibration plane of the target plate.

[0022] According to an embodiment of the present application, the step of processing the target plate image data to obtain the deviation distance between the camera center of the reference module and the center of the target plate includes the steps of:

[0023] Extracting a central pixel point from the target plate image data as the camera center of the reference module;

[0024] Performing center fitting processing on the image of the visual mark of the target extracted from the target image data to obtain pixel coordinates of the image fitting center of the visual mark; and

[0025] The coordinate difference between the image fitting center of the visual mark and the central pixel point is calculated to obtain the distance between the camera center of the reference module and the center of the target plate.

[0026] According to an embodiment of the present application, the step of calculating the coordinate difference between the image fitting center of the visual mark and the central pixel point to obtain the distance between the camera center of the reference module and the center of the target plate includes the steps of:

[0027] According to the extracted central pixel point, simulating the camera center position of the reference module on a display interface;

[0028] According to the image fitting center of the visual mark, simulating the center position of the target plate on the display interface; and

[0029] The number of pixels between the center position of the camera of the reference module simulated on the display interface and the center position of the target plate is counted to obtain the deviation distance.

[0030] According to an embodiment of the present application, the step of adjusting the relative position between the tooling platform and the center of the target plate in the device to be debugged based on the deviation distance until the deviation distance is less than or equal to a deviation threshold comprises the steps of:

[0031] Determine whether the deviation distance is greater than the deviation threshold;

[0032] In response to the deviation distance being greater than the deviation threshold, start adjusting the position of the tooling platform of the device to be debugged to reduce the deviation distance; and

[0033] In response to the deviation distance being less than or equal to the deviation threshold, the position of the tooling platform of the device to be debugged is stopped to complete the alignment of the camera center of the reference module with the center of the target plate.

[0034] According to an embodiment of the present application, the step of adjusting the relative position between the tooling platform and the center of the target plate in the device to be debugged based on the deviation distance until the deviation distance is less than or equal to a deviation threshold comprises the steps of:

[0035] Determine whether the deviation distance is greater than the deviation threshold;

[0036] In response to the deviation distance being greater than the deviation threshold, starting to adjust the position of the target plate of the device to be debugged to reduce the deviation distance; and

[0037] In response to the deviation distance being less than or equal to the deviation threshold, the adjustment of the position of the target plate of the device to be debugged is stopped to complete the alignment work between the camera center of the reference module and the center of the target plate.

[0038] According to another aspect of the present application, the present application further provides a visual centering system for debugging a tooling platform or a standard plate of a device to be debugged, comprising:

[0039] a data acquisition module, used to acquire target plate image data obtained by collecting target plate information of the device to be debugged via a reference module, wherein the reference module is installed on the tooling platform of the device to be debugged, and the target plate is within the field of view of the reference module;

[0040] a data processing module, used for processing the target plate image data to obtain a deviation distance between the center of the camera of the reference module and the center of the target plate; and

[0041] A position adjustment module is used to adjust the relative position of the tooling platform and the center of the target plate in the device to be debugged based on the deviation distance until the deviation distance is less than or equal to a deviation threshold.

[0042] According to one embodiment of the present application, the data quantity module includes a center extraction module, a center fitting module and a difference calculation module that are communicatively connected to each other, wherein the center extraction module is used to extract the central pixel point from the target plate image data as the camera center of the reference module; wherein the center fitting module is used to obtain the pixel coordinates of the image fitting center of the visual mark by performing center fitting processing on the image of the visual mark of the target plate extracted from the target plate image data; wherein the difference calculation module is used to calculate the coordinate difference between the image fitting center of the visual mark and the central pixel point to obtain the distance between the camera center of the reference module and the center of the target plate.

[0043] According to one embodiment of the present application, the position adjustment module includes a deviation judgment module, a start adjustment module and a stop adjustment module that are communicatively connected to each other, wherein the deviation judgment module is used to judge whether the deviation distance is greater than the deviation threshold; wherein the start adjustment module is used to start adjusting the position of the tooling platform of the device to be debugged in response to the deviation distance being greater than the deviation threshold to reduce the deviation distance; wherein the stop adjustment module is used to stop adjusting the position of the tooling platform of the device to be debugged in response to the deviation distance being less than or equal to the deviation threshold to complete the alignment of the camera center of the reference module with the center of the target plate.

[0044] According to one embodiment of the present application, the position adjustment module includes a deviation judgment module, a start adjustment module and a stop adjustment module that are communicatively connected to each other, wherein the deviation judgment module is used to judge whether the deviation distance is greater than the deviation threshold; wherein the start adjustment module is used to start adjusting the position of the target plate of the device to be debugged in response to the deviation distance being greater than the deviation threshold to reduce the deviation distance; wherein the stop adjustment module is used to stop adjusting the position of the target plate of the device to be debugged in response to the deviation distance being less than or equal to the deviation threshold to complete the alignment of the camera center of the reference module with the center of the target plate.

[0045] According to another aspect of the present application, the present application further provides an electronic device, including:

[0046] at least one processor for executing instructions; and

[0047] A memory communicatively connected to the at least one processor, wherein the memory has at least one instruction, wherein the instruction is executed by the at least one processor to enable the at least one processor to perform all or part of the steps in a visual centering method, wherein the visual centering method comprises the steps of:

[0048] Acquire target plate image data obtained by collecting target plate information of a device to be debugged via a reference module, wherein the reference module is installed on a tooling platform of the device to be debugged, and the target plate is within the field of view of the reference module;

[0049] Processing the target plate image data to obtain a deviation distance between a camera center of the reference module and a center of the target plate; and

[0050] Based on the deviation distance, the relative position of the tooling platform and the center of the target plate in the device to be debugged is adjusted until the deviation distance is less than or equal to a deviation threshold.

[0051] According to another aspect of the present application, the present application further provides an electronic device, including:

[0052] A device to be debugged, wherein the device to be debugged comprises at least one tooling platform and a standard plate; and

[0053] A visual centering system, wherein the visual centering system is configured on the device to be debugged, and is used to debug the tooling platform or the target plate of the device to be debugged, and the visual centering system includes:

[0054] a data acquisition module, used to acquire target plate image data obtained by collecting target plate information of the device to be debugged via a reference module, wherein the reference module is installed on the tooling platform of the device to be debugged, and the target plate is within the field of view of the reference module;

[0055] a data processing module, used for processing the target plate image data to obtain a deviation distance between the center of the camera of the reference module and the center of the target plate; and

[0056] A position adjustment module is used to adjust the relative position of the tooling platform and the center of the target plate in the device to be debugged based on the deviation distance until the deviation distance is less than or equal to a deviation threshold.

[0057] Further objects and advantages of the present invention will be fully apparent from an understanding of the following description and the accompanying drawings.

[0058] These and other objects, features and advantages of the present invention will be more fully understood from the following detailed description, accompanying drawings and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 4 is a flow chart of a visual alignment method according to an embodiment of the present application.

[0060] Figure 2 and Figure 3 A flow chart of the data processing steps in the visual alignment method according to the above embodiment of the present application is shown.

[0061] Figure 4 An example of a position adjustment step in the visual centering method according to the above embodiment of the present application is shown.

[0062] Figure 5 Another example of the position adjustment step in the visual alignment method according to the above embodiment of the present application is shown.

[0063] Figure 6 An example of a target plate obtained by the visual alignment method according to the above embodiment of the present application is shown.

[0064] Figure 7 An example of a display interface in the visual alignment method according to the above embodiment of the present application is shown.

[0065] Figure 8 It is a block diagram of a visual alignment system according to an embodiment of the present application.

[0066] Fig. 9 An example of an electronic device according to the present application is shown.

[0067] Fig.10 is a three-dimensional schematic diagram of another electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0068] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

[0069] Those skilled in the art should understand that, in the disclosure of the present invention, the orientation or position relationship indicated by the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. are based on the orientation or position relationship shown in the drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the above terms should not be understood as limiting the present invention.

[0070] In the present invention, the term "one" in the claims and the specification should be understood as "one or more", that is, in one embodiment, the number of an element can be one, while in another embodiment, the number of the element can be multiple. Unless it is clearly indicated in the disclosure of the present invention that the number of the element is only one, the term "one" cannot be understood as unique or single, and the term "one" cannot be understood as a limitation on the quantity.

[0071] In the description of the present invention, it should be understood that "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through a medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0072] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0073] At present, in scenarios such as equipment assembly or module calibration equipment debugging, high requirements are usually placed on centering in order to achieve higher-precision assembly or calibration later. For example, before performing internal parameter calibration on the photosensitive receiving module of the TOF module (i.e., the camera in the TOF module), it is necessary to first debug the calibration equipment to align the center of the camera of the TOF module with the center of the target plate (i.e., centering work), so that it is possible to obtain a higher-precision calibration result when calibrating the internal parameters later; for example, when assembling the camera module, before the glue between the lens and the photosensitive component is cured, it is often necessary to test the main axis inclination of the lens, and in order to obtain a higher-precision inclination test result, it is also necessary to align the center of the camera module with the center of the test target plate. However, the existing heavy hammer method and laser method usually have the following problems: First, the centering deviation cannot be digitized and can only be observed visually, which cannot meet the requirements for application scenarios with high requirements; second, the operation is inconvenient, especially in small spaces, and the disassembly and assembly time will be long; third, the deviation of the centering accuracy will become very large due to installation errors. Therefore, in order to solve the above problems, the present application provides a visual centering method and its system and electronic equipment, which can greatly improve the convenience and accuracy of centering operation through visual positioning, and can visualize the deviation value, which helps to greatly improve the centering accuracy.

[0074] Schematic method

[0075] Reference to the attached drawings of the specification Figures 1 to 6 As shown in FIG. 1 , a visual alignment method according to an embodiment of the present invention is described. Specifically, Figure 1 As shown, the visual alignment method may include the steps of:

[0076] S100: Acquire target plate image data obtained by collecting target plate information of a device to be debugged via a reference module, wherein the reference module is installed on a tooling platform of the device to be debugged, and the target plate is within the field of view of the reference module;

[0077] S200: Processing the target plate image data to obtain a deviation distance between a camera center of the reference module and a center of the target plate; and

[0078] S300: Based on the deviation distance, adjust the relative position between the tooling platform and the target plate in the device to be debugged until the deviation distance is less than or equal to a deviation threshold.

[0079] It is worth noting that the reference module mentioned in this application can be, but is not limited to, implemented as a calibrated camera module such as a TOF camera module or an RGB camera module. The reference plate used is different for different types of camera modules and the items to be calibrated or tested later. For example, for the internal reference calibration of the TOF camera module, the reference plate used can be, but is not limited to, implemented as a light board reference plate; and for the lens tilt calibration of the RGB camera module, the reference plate used can be, but is not limited to, implemented as a point light source reference plate.

[0080] Preferably, the target may be implemented as a target having a visual mark, wherein the visual mark is used to indicate the center of the target, so as to fit the center position of the target through the visual mark information in the target image data.

[0081] More preferably, if Figure 6 As shown, the visual mark of the target plate can be a point light source located at the center of the target plate. Of course, in other examples of the present application, the visual mark of the target plate can also be a regular pattern such as a square, a circle or a cross set on the target plate, and the center of the regular pattern coincides with the center of the target plate, so as to fit the center of the regular pattern as the center of the target plate through the image of the regular pattern; or, the visual mark of the target plate can also be implemented as a regular shape of the target plate, such as a square target plate or a circular target plate, so as to fit the contour center as the center of the target plate through the contour image of the regular shape of the target plate; or, the visual mark of the target plate can also be a plurality of Make points set on the target plate, wherein the geometric centers of the plurality of Make points coincide with the center of the target plate, so as to fit the geometric centers of the plurality of Make points as the center of the target plate through the images of the plurality of Make points.

[0082] It is worth noting that the visual centering method of the present application also needs to adjust the parallelism between the installation plane of the tooling platform and the calibration plane of the target plate before the step S100, so that the installation plane of the tooling platform is as parallel to the calibration plane of the target plate as possible, so as to ensure that the visual mark of the target plate seen by the camera of the reference module will not deviate, which helps to improve the accuracy of the centering. For example, when the target plate is arranged horizontally so that the calibration plane of the target plate is parallel to the horizontal plane, it is only necessary to adjust the horizontality of the installation plane of the tooling platform to ensure that the visual mark of the target plate seen by the camera of the reference module will not deviate.

[0083] It can be understood that, since the visual centering method of the present application obtains the corresponding target plate image data through the reference module, so that the center of the target plate can be calculated by processing the target plate image data, and the camera center of the reference module corresponds to the image center of the target plate image data, the visual centering method of the present application can calculate the deviation distance between the camera center of the reference module and the center of the target plate by means of vision plus software, so that the deviation value is visualized, which helps to improve the centering accuracy. At the same time, compared with the traditional weight method and laser method, the visual centering method of the present application does not require the additional setting of a weight and laser, but only needs to install the reference module to the tooling platform to perform the centering operation, which greatly improves the convenience and accuracy of the centering operation.

[0084] It is worth mentioning that the reference module of the present application and the module to be subsequently calibrated or tested (i.e., the module to be calibrated or the module to be tested) belong to the same model of module, so that the camera center of the module to be calibrated or the module to be tested installed on the tooling platform coincides with the camera center of the reference module installed on the tooling platform. In this way, after the centering operation is completed, it is only necessary to install the module to be calibrated or the module to be tested on the tooling platform to replace the reference module, and subsequent calibration and testing work can be carried out, so as to eliminate the installation error between the module to be calibrated or the module to be tested and the reference module, which helps to efficiently continue the centering accuracy and realize subsequent batch calibration or testing.

[0085] According to the above embodiments of the present application, Figure 2 As shown, the step S200 of the visual centering method of the present application may include the steps of:

[0086] S210: extracting a central pixel point from the target plate image data as the camera center of the reference module;

[0087] S220: performing center fitting processing on the image of the visual mark of the target extracted from the target image data to obtain pixel coordinates of the image fitting center of the visual mark; and

[0088] S230: Calculate the coordinate difference between the image fitting center of the visual mark and the central pixel point to obtain the distance between the camera center of the reference module and the center of the target plate.

[0089] More specifically, if Figure 3 As shown, the step S230 of the visual centering method of the present application may include the steps of:

[0090] S231: simulating the camera center position of the reference module on a display interface according to the extracted central pixel point;

[0091] S232: simulating the center position of the target plate on the display interface according to the image fitting center of the visual mark; and

[0092] S233: Counting the number of pixels between the center position of the camera of the reference module simulated on the display interface and the center position of the target plate to obtain the deviation distance.

[0093] For example, after the reference module is installed and the drawing is started, the software simulates the center of the reference plate to be aligned on the display and uses a small green cross (such as Figure 7 The center cross in the center of the target plate is represented by the center of the target plate, where the center of the target plate can be obtained by fitting the center of a regular shape (such as a square or a circle, etc.) or by fitting the geometric center of multiple Make points; at the same time, the camera center software will simulate a red cross (such as Figure 7 The upper left cross in the image is used to represent the camera center of the reference module; and then the x / y coordinate difference between the red and green crosses can be read out through the number of pixels, so as to obtain the deviation distance between the camera center of the reference module and the center of the target plate.

[0094] According to the above embodiments of the present application, Figure 4 As shown, the step S300 of the visual centering method of the present application may include the steps of:

[0095] S310: Determine whether the deviation distance is greater than the deviation threshold;

[0096] S320: In response to the deviation distance being greater than the deviation threshold, start adjusting the position of the tooling platform of the device to be debugged to reduce the deviation distance; and

[0097] S330: In response to the deviation distance being less than or equal to the deviation threshold, stop adjusting the position of the tooling platform of the device to be debugged to complete the alignment of the camera center of the reference module with the center of the target plate.

[0098] It is worth noting that in other examples of this application, such as Figure 5 As shown, the step S300 of the visual centering method of the present application may also include the steps of:

[0099] S310': Determine whether the deviation distance is greater than the deviation threshold;

[0100] S320': in response to the deviation distance being greater than the deviation threshold, start adjusting the position of the target plate of the device to be debugged to reduce the deviation distance; and

[0101] S330': In response to the deviation distance being less than or equal to the deviation threshold, stop adjusting the position of the target plate of the device to be debugged to complete the centering work of the camera center of the reference module and the center of the target plate.

[0102] It is understandable that, whether by adjusting the position of the tooling platform of the device to be debugged or by adjusting the position of the target plate of the device to be debugged, the center of the camera of the reference module can be aligned with the center of the target plate, thereby completing the debugging of the device to be debugged. Afterwards, it is only necessary to install the module to be calibrated or the module to be tested on the tooling platform to replace the reference module, so as to ensure that the center of the camera of the module to be calibrated or the module to be tested is still aligned with the center of the target plate.

[0103] It is worth noting that the deviation threshold of the present application can be determined according to the requirements of the centering accuracy in the specific application scenario. For example, the deviation threshold can be, but is not limited to, implemented as ±1 mm.

[0104] In addition, for a device to be debugged with multiple workstations, the visual centering method of the present application can be used to perform centering debugging operations on multiple workstations respectively. Of course, if the multiple workstations of the device to be debugged work in a circular assembly line, then when adjusting the relative position between the tooling platform and the reference plate in the device to be debugged, only the position of the tooling platform of the device to be debugged can be adjusted, and the position of the reference plate of the device to be debugged cannot be adjusted. This is because the multiple tooling platforms are distributed in the multiple workstations, and the reference plate will only correspond to one of the workstations at a time. Therefore, only by adjusting the positions of the multiple tooling platforms in sequence can it be ensured that the center of the reference plate is always aligned with the camera center of the reference module on the workstation corresponding to the reference plate.

[0105] In addition, the visual alignment method of the present application can also be extended to mechanical axis and hole alignment, for example, a reference module or camera can be installed at a position concentric with the axis through mechanical positioning (that is, the module lens is concentric with the axis after the module or camera fixture is installed), and then the center of the hole is checked by opening the drawing to complete the alignment work through the visual alignment method of the present application.

[0106] Schematic system

[0107] Reference to the attached drawings of the specification Figure 8 As shown in FIG. 1 , a visual centering system according to an embodiment of the present invention is described. Specifically, as Figure 8 As shown, the visual centering system 1 is used to debug a tooling platform or a target plate of a device to be debugged, and the visual centering system 1 includes: a data acquisition module 10, which is communicatively connected to each other, and is used to obtain target plate image data obtained by collecting target plate information of a device to be debugged via a reference module, wherein the reference module is installed on the tooling platform of the device to be debugged, and the target plate is within the field of view of the reference module; a data processing module 20, which is used to process the target plate image data to obtain a deviation distance between the camera center of the reference module and the center of the target plate; and a position adjustment module 30, which is used to adjust the relative position of the tooling platform and the target plate center in the device to be debugged based on the deviation distance, until the deviation distance is less than or equal to a deviation threshold.

[0108] It is worth noting that Figure 8 As shown, the data quantity module 20 includes a center extraction module 21, a center fitting module 22 and a difference calculation module 23 which are communicatively connected to each other, wherein the center extraction module 21 is used to extract the center pixel point from the target plate image data as the camera center of the reference module; wherein the center fitting module 22 is used to obtain the pixel coordinates of the image fitting center of the visual mark by performing center fitting processing on the image of the visual mark of the target plate extracted from the target plate image data; wherein the difference calculation module 23 is used to calculate the coordinate difference between the image fitting center of the visual mark and the center pixel point, so as to obtain the distance between the camera center of the reference module and the center of the target plate.

[0109] In an example of the present application, the difference calculation module 23 is further used to: simulate the camera center position of the reference module on a display interface based on the extracted central pixel point; simulate the center position of the target plate on the display interface based on the image fitting center of the visual mark; and count the number of pixels between the camera center position of the reference module simulated on the display interface and the center position of the target plate to obtain the deviation distance.

[0110] In the above embodiments of the present application, Figure 8 As shown, the position adjustment module 30 may include a deviation judgment module 31, a start adjustment module 32 and a stop adjustment module 33 which are communicatively connected to each other, wherein the deviation judgment module 31 is used to judge whether the deviation distance is greater than the deviation threshold; wherein the start adjustment module 32 is used to start adjusting the position of the tooling platform of the device to be debugged in response to the deviation distance being greater than the deviation threshold, so as to reduce the deviation distance; wherein the stop adjustment module 33 is used to stop adjusting the position of the tooling platform of the device to be debugged in response to the deviation distance being less than or equal to the deviation threshold, so as to complete the alignment work between the camera center of the reference module and the center of the target plate.

[0111] In other examples of this application, Figure 8 As shown, the position adjustment module 30 may also include a deviation judgment module 31, a start adjustment module 32 and a stop adjustment module 33 which are communicatively connected to each other, wherein the deviation judgment module 31 is used to judge whether the deviation distance is greater than the deviation threshold; wherein the start adjustment module 32 is used to start adjusting the position of the target plate of the device to be debugged in response to the deviation distance being greater than the deviation threshold, so as to reduce the deviation distance; wherein the stop adjustment module 33 is used to stop adjusting the position of the target plate of the device to be debugged in response to the deviation distance being less than or equal to the deviation threshold, so as to complete the alignment work between the camera center of the reference module and the center of the target plate.

[0112] Schematic electronic equipment

[0113] Below, reference Fig. 9 An electronic device according to an embodiment of the present invention is described below. Fig. 9 As shown, the electronic device 90 includes one or more processors 91 and a memory 92 .

[0114] The processor 91 may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 90 to perform desired functions. In other words, the processor 91 includes one or more physical devices configured to execute instructions. For example, the processor 91 may be configured to execute instructions as part of the following: one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical structures. Such instructions may be implemented to perform tasks, implement data types, convert the state of one or more components, implement technical effects, or otherwise obtain desired results.

[0115] The processor 91 may include one or more processors configured to execute software instructions. In addition or in lieu thereof, the processor 91 may include one or more hardware or firmware logic machines configured to execute hardware or firmware instructions. The processor of the processor 91 may be single-core or multi-core, and the instructions executed thereon may be configured for serial, parallel and / or distributed processing. The various components of the processor 91 may optionally be distributed on two or more separate devices, which may be remotely located and / or configured to perform collaborative processing. Various aspects of the processor 91 may be virtualized and executed by a remotely accessible networked computing device configured in a cloud computing configuration.

[0116] The memory 92 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 91 may execute the program instructions to implement some or all of the steps in the above-mentioned schematic method of the present invention described above, and / or other desired functions.

[0117] In other words, the memory 92 includes one or more physical devices configured to store machine-readable instructions that can be executed by the processor 91 to implement the methods and processes described herein. When implementing these methods and processes, the state of the memory 92 can be changed (for example, to store different data). The memory 92 may include removable and / or built-in devices. The memory 92 may include optical storage (for example, CD, DVD, HD-DVD, Blu-ray disc, etc.), semiconductor storage (for example, RAM, EPROM, EEPROM, etc.) and / or magnetic storage (for example, hard disk drive, floppy disk drive, tape drive, MRAM, etc.), etc. The memory 92 may include volatile, non-volatile, dynamic, static, read / write, read-only, random access, sequential access, location addressable, file addressable and / or content addressable devices.

[0118] It is understood that the memory 92 includes one or more physical devices. However, various aspects of the instructions described herein may alternatively be propagated through a communication medium (e.g., electromagnetic signals, optical signals, etc.) that is not held by a physical device for a limited period of time. Various aspects of the processor 91 and the memory 92 may be integrated together into one or more hardware logic components. These hardware logic components may include, for example, field programmable gate arrays (FPGAs), program and application-specific integrated circuits (PASIC / ASIC), program and application-specific standard products (PSSP / ASSP), systems on chip (SOCs), and complex programmable logic devices (CPLDs).

[0119] In one example, if Fig. 9 As shown, the electronic device 90 may also include an input device 93 and an output device 94, which are interconnected by a bus system and / or other forms of connection mechanisms (not shown). For example, the input device 93 may be, for example, a camera module for collecting image data or video data, etc. As another example, the input device 93 may include or dock with one or more user input devices such as a keyboard, a mouse, a touch screen, or a game controller. In some embodiments, the input device 93 may include or dock with a selected natural user input (NUI) component. Such component parts may be integrated or peripheral, and the transduction and / or processing of input actions may be processed on or off the board. Example NUI components may include microphones for language and / or speech recognition; infrared, color, stereoscopic display and / or depth cameras for machine vision and / or gesture recognition; head trackers, eye trackers, accelerometers and / or gyroscopes for motion detection and / or intent recognition; and electric field sensing components for evaluating brain activity and / or body movement; and / or any other suitable sensors.

[0120] The output device 94 can output various information to the outside, including classification results, etc. The output device 94 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0121] Of course, the electronic device 90 can also further include the communication device, wherein the communication device can be configured to couple the electronic device 90 to one or more other computer devices. The communication device can include wired and / or wireless communication devices compatible with one or more different communication protocols. As a non-limiting example, the communication subsystem can be configured to communicate via a wireless telephone network or a wired or wireless local area network or a wide area network. In certain embodiments, the communication device can allow the electronic device 90 to send a message to other devices and / or receive a message from other devices via a network such as the Internet.

[0122] It will be appreciated that the configurations and / or methods described herein are exemplary in nature, and these specific embodiments or examples should not be considered restrictive, as many variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. Thus, the various actions shown and / or described may be performed in the order shown and / or described, in other orders, in parallel, or omitted. Likewise, the order of the above process may be changed.

[0123] Of course, to simplify, Fig. 9 Only some of the components related to the present invention in the electronic device 90 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 90 may also include any other appropriate components.

[0124] According to another aspect of the present application, the present application further provides an electronic device. Specifically, Fig.10 As shown, the electronic device includes a device to be debugged 80 and the above-mentioned visual centering system 1, wherein the device to be debugged 80 includes at least one tooling platform 81 and a target plate 82, and the visual centering system 1 is configured on the device to be debugged 80, and is used to debug the tooling platform 81 or the target plate 82 of the device to be debugged 80, so that the center of the camera of the reference module 70 installed on the tooling platform 81 is aligned with the center of the target plate 82.

[0125] For example, Fig.10 As shown, the multiple workstations of the equipment to be debugged 80 work in a circular assembly line, that is, the equipment to be debugged 80 can further include a turntable 83, wherein the four tooling platforms 81 are evenly arranged on the turntable 83, so that under the rotation of the turntable 83, the four tooling platforms 81 will correspond to the reference plate 82 in sequence, so that the positions of the tooling platforms 81 can be adjusted separately through the visual alignment system 1 of the present application, so that the center of the reference plate 82 is always aligned with the camera center of the reference module 70 on the tooling platform 81 corresponding to the reference plate 82.

[0126] It should also be noted that in the device, apparatus and method of the present invention, each component or each step can be decomposed and / or reassembled, and such decomposition and / or reassembly should be regarded as an equivalent solution of the present invention.

[0127] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present invention. Therefore, the present invention is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0128] It should be understood by those skilled in the art that the embodiments of the present invention described above and shown in the accompanying drawings are only examples and do not limit the present invention. The purpose of the present invention has been fully and effectively achieved. The functional and structural principles of the present invention have been demonstrated and explained in the embodiments, and the embodiments of the present invention may be deformed or modified in any way without departing from the principles.

Claims

1. A visual alignment method, characterized in that: Includes steps: Acquire target plate image data obtained by collecting target plate information of a device to be debugged via a reference module, wherein the reference module is installed on a tooling platform of the device to be debugged, and the target plate is within the field of view of the reference module; Processing the target plate image data to obtain a deviation distance between a camera center of the reference module and a center of the target plate; and Based on the deviation distance, adjusting the relative position of the tooling platform in the device to be debugged and the center of the target plate until the deviation distance is less than or equal to a deviation threshold, so as to complete the alignment of the camera center of the reference module and the center of the target plate; The target plate is a target plate with a visual mark, and the visual mark is used to indicate the center of the target plate, so as to fit the center position of the target plate through the visual mark information in the target plate image data; the visual mark of the target plate is one selected from a point light source located at the center of the target plate, a regular pattern whose center coincides with the center of the target plate, a regular shape of the target plate, and a plurality of Make points whose geometric centers coincide with the center of the target plate; The step of processing the target plate image data to obtain the deviation distance between the camera center of the reference module and the center of the target plate comprises the following steps: Extracting a central pixel point from the target plate image data as the camera center of the reference module; By performing center fitting processing on the image of the visual mark of the target plate extracted from the target plate image data, the pixel coordinates of the image fitting center of the visual mark are obtained; as well as The coordinate difference between the image fitting center of the visual mark and the central pixel point is calculated to obtain the distance between the camera center of the reference module and the center of the target plate.

2. The visual alignment method according to claim 1, wherein: The reference module is a calibrated TOF camera module or RGB camera module.

3. The visual alignment method according to claim 1, further comprising the step of: before the step of acquiring target image data obtained by collecting target information of a device to be debugged via a reference module, wherein the reference module is installed on a tooling platform of the device to be debugged and the target is within the field of view of the reference module: The parallelism between the installation plane of the tooling platform and the calibration plane of the target plate is adjusted so that the installation plane of the tooling platform is parallel to the calibration plane of the target plate.

4. The visual alignment method according to any one of claims 1 to 3, wherein: The step of calculating the coordinate difference between the image fitting center of the visual mark and the central pixel point to obtain the distance between the camera center of the reference module and the center of the target plate comprises the steps of: According to the extracted central pixel point, simulating the camera center position of the reference module on a display interface; According to the image fitting center of the visual mark, simulating the center position of the mark plate on the display interface; as well as The number of pixels between the center position of the camera of the reference module simulated on the display interface and the center position of the target plate is counted to obtain the deviation distance.

5. The visual alignment method according to any one of claims 1 to 3, wherein: The step of adjusting the relative position between the tooling platform and the center of the target plate in the device to be debugged based on the deviation distance until the deviation distance is less than or equal to a deviation threshold comprises the steps of: Determine whether the deviation distance is greater than the deviation threshold; In response to the deviation distance being greater than the deviation threshold, start adjusting the position of the tooling platform of the device to be debugged to reduce the deviation distance; as well as In response to the deviation distance being less than or equal to the deviation threshold, the position of the tooling platform of the device to be debugged is stopped to complete the alignment of the camera center of the reference module with the center of the target plate.

6. The visual alignment method according to any one of claims 1 to 3, wherein: The step of adjusting the relative position between the tooling platform and the center of the target plate in the device to be debugged based on the deviation distance until the deviation distance is less than or equal to a deviation threshold comprises the steps of: Determine whether the deviation distance is greater than the deviation threshold; In response to the deviation distance being greater than the deviation threshold, start adjusting the position of the target plate of the device to be debugged to reduce the deviation distance; as well as In response to the deviation distance being less than or equal to the deviation threshold, the adjustment of the position of the target plate of the device to be debugged is stopped to complete the alignment work between the camera center of the reference module and the center of the target plate.

7. A visual centering system, used for debugging the tooling platform or standard plate of the equipment to be debugged, characterized in that: comprising communicatively connected to each other: a data acquisition module, used to acquire target plate image data obtained by collecting target plate information of the device to be debugged via a reference module, wherein the reference module is installed on the tooling platform of the device to be debugged, and the target plate is within the field of view of the reference module; a data processing module, used for processing the target plate image data to obtain a deviation distance between the center of the camera of the reference module and the center of the target plate; and a position adjustment module, used for adjusting the relative position of the tooling platform and the center of the target plate in the device to be debugged based on the deviation distance, until the deviation distance is less than or equal to a deviation threshold, so as to complete the alignment of the camera center of the reference module and the center of the target plate; The target plate is a target plate with a visual mark, and the visual mark is used to indicate the center of the target plate, so as to fit the center position of the target plate through the visual mark information in the target plate image data; the visual mark of the target plate is one selected from a point light source located at the center of the target plate, a regular pattern whose center coincides with the center of the target plate, a regular shape of the target plate, and a plurality of Make points whose geometric centers coincide with the center of the target plate; The data processing module comprises a center extraction module, a center fitting module and a difference calculation module which are communicatively connected to each other, wherein the center extraction module is used to extract a center pixel point from the target plate image data as the camera center of the reference module; wherein the center fitting module is used to obtain the pixel coordinates of the image fitting center of the visual mark by performing center fitting processing on the image of the visual mark of the target plate extracted from the target plate image data; The difference calculation module is used to calculate the coordinate difference between the image fitting center of the visual mark and the central pixel point to obtain the distance between the camera center of the reference module and the center of the target plate.

8. The visual alignment system according to claim 7, wherein: The position adjustment module includes a deviation judgment module, a start adjustment module and a stop adjustment module which are communicatively connected to each other, wherein the deviation judgment module is used to judge whether the deviation distance is greater than the deviation threshold; wherein the start adjustment module is used to start adjusting the position of the tooling platform of the device to be debugged in response to the deviation distance being greater than the deviation threshold, so as to reduce the deviation distance; wherein the stop adjustment module is used to stop adjusting the position of the tooling platform of the device to be debugged in response to the deviation distance being less than or equal to the deviation threshold, so as to complete the alignment work between the camera center of the reference module and the center of the target plate.

9. The visual alignment system according to claim 7, wherein: The position adjustment module includes a deviation judgment module, a start adjustment module and a stop adjustment module which are communicatively connected to each other, wherein the deviation judgment module is used to judge whether the deviation distance is greater than the deviation threshold; wherein the start adjustment module is used to start adjusting the position of the target plate of the device to be debugged in response to the deviation distance being greater than the deviation threshold, so as to reduce the deviation distance; wherein the stop adjustment module is used to stop adjusting the position of the target plate of the device to be debugged in response to the deviation distance being less than or equal to the deviation threshold, so as to complete the alignment work between the camera center of the reference module and the center of the target plate.

10. An electronic device, characterized in that: include: at least one processor for executing instructions; and A memory communicatively connected to the at least one processor, wherein the memory has at least one instruction, wherein the instruction is executed by the at least one processor to enable the at least one processor to perform all or part of the steps in a visual centering method, wherein the visual centering method comprises the steps of: Acquire target plate image data obtained by collecting target plate information of a device to be debugged via a reference module, wherein the reference module is installed on a tooling platform of the device to be debugged, and the target plate is within the field of view of the reference module; Processing the target plate image data to obtain a deviation distance between a camera center of the reference module and a center of the target plate; and Based on the deviation distance, adjusting the relative position of the tooling platform in the device to be debugged and the center of the target plate until the deviation distance is less than or equal to a deviation threshold, so as to complete the alignment of the camera center of the reference module and the center of the target plate; The target plate is a target plate with a visual mark, and the visual mark is used to indicate the center of the target plate, so as to fit the center position of the target plate through the visual mark information in the target plate image data; the visual mark of the target plate is one selected from a point light source located at the center of the target plate, a regular pattern whose center coincides with the center of the target plate, a regular shape of the target plate, and a plurality of Make points whose geometric centers coincide with the center of the target plate; The step of processing the target plate image data to obtain the deviation distance between the camera center of the reference module and the center of the target plate comprises the following steps: Extracting a central pixel point from the target plate image data as the camera center of the reference module; Performing center fitting processing on the image of the visual mark of the target extracted from the target image data to obtain pixel coordinates of the image fitting center of the visual mark; and The coordinate difference between the image fitting center of the visual mark and the central pixel point is calculated to obtain the distance between the camera center of the reference module and the center of the target plate.

11. An electronic device, characterized in that: include: A device to be debugged, wherein the device to be debugged comprises at least one tooling platform and a standard plate; and A visual centering system, wherein the visual centering system is configured on the device to be debugged, and is used to debug the tooling platform or the target plate of the device to be debugged, and the visual centering system includes: a data acquisition module, used to acquire target plate image data obtained by collecting target plate information of the device to be debugged via a reference module, wherein the reference module is installed on the tooling platform of the device to be debugged, and the target plate is within the field of view of the reference module; a data processing module, used for processing the target plate image data to obtain a deviation distance between the center of the camera of the reference module and the center of the target plate; and a position adjustment module, used for adjusting the relative position of the tooling platform and the center of the target plate in the device to be debugged based on the deviation distance, until the deviation distance is less than or equal to a deviation threshold, so as to complete the alignment of the camera center of the reference module and the center of the target plate; The target plate is a target plate with a visual mark, and the visual mark is used to indicate the center of the target plate, so as to fit the center position of the target plate through the visual mark information in the target plate image data; the visual mark of the target plate is one selected from a point light source located at the center of the target plate, a regular pattern whose center coincides with the center of the target plate, a regular shape of the target plate, and a plurality of Make points whose geometric centers coincide with the center of the target plate; The data processing module comprises a center extraction module, a center fitting module and a difference calculation module which are communicatively connected to each other, wherein the center extraction module is used to extract a center pixel point from the target plate image data as the camera center of the reference module; wherein the center fitting module is used to obtain the pixel coordinates of the image fitting center of the visual mark by performing center fitting processing on the image of the visual mark of the target plate extracted from the target plate image data; The difference calculation module is used to calculate the coordinate difference between the image fitting center of the visual mark and the central pixel point to obtain the distance between the camera center of the reference module and the center of the target plate.

Images