Cross-view multi-camera calibration positioning method and system

Abstract

The embodiment of the invention provides a cross-view multi-camera calibration positioning method and system, and the method comprises the steps: obtaining the coordinate deviation value of a first camera and a second camera, and enabling the position of the first camera to be parallel to the position of the second camera; unifying the coordinates of the first camera and the second camera in the same coordinate system according to the coordinate deviation value when the first camera and the second camera detect the target point, and obtaining a first point coordinate set of a first angular point and a second point coordinate set of a second angular point of the target workpiece detected by the first camera, and the second camera detects a third point coordinate set of a third triangular point of the target workpiece and a fourth point coordinate set of a fourth angular point of the target workpiece. And generating a coordinate set of four edges of the target workpiece according to the first point coordinate set, the second point coordinate set, the third point coordinate set and the fourth point coordinate set. And according to the coordinate set of the four edges, obtaining position information of a target position in the target workpiece. Therefore, the positioning accuracy of the workpiece is improved.

Description

Technical Field

[0001] The present application relates to the field of automation equipment technology, and in particular to a positioning method and system for cross-field multi-camera calibration. Background Art

[0002] With the continuous development of the economy, the assembly accuracy requirements for workpieces in the manufacturing industry are getting higher and higher. Taking the display industry as an example, the size of the display screens currently produced is getting larger and larger, and the local area of ​​the product photographed by a single camera cannot correctly reflect the overall position and posture of the product. For example, if only the image of the display screen within the field of view of a single camera is used for positioning, when a certain corner is deformed or external interference occurs, it will cause the grasping position of a single camera to deviate, thereby causing a deviation in the overall positioning, and ultimately resulting in poor positioning. However, if four cameras are used to shoot the four corners of the display screen (or two cameras are moved and shot twice) to obtain the overall position and posture of the display screen, there is still a problem that the joint calibration method of multiple cameras is relatively complicated and difficult to implement.

[0003] Therefore, how to improve the positioning accuracy of the workpiece is an urgent problem to be solved. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a positioning method and system for cross-field multi-camera calibration to improve the positioning accuracy of the workpiece.

[0005] In conjunction with the first aspect of the present application, a positioning method for cross-field multi-camera calibration is provided, the method comprising: Acquire a coordinate deviation between a first camera and a second camera, wherein a position of the first camera is parallel to a position of the second camera; Unifying the coordinates of the target point detected by the first camera and the second camera into the same coordinate system according to the coordinate deviation; Acquire a first point coordinate set of a first corner point and a second point coordinate set of a second corner point of a target workpiece detected by the first camera, a third point coordinate set of a third corner point and a fourth point coordinate set of a fourth corner point of the target workpiece detected by the second camera, and the target workpiece is a rectangular workpiece; Generate coordinate sets of four sides of the target workpiece according to the first point coordinate set, the second point coordinate set, the third point coordinate set, and the fourth point coordinate set; According to the coordinate set of the four edges, the position information of the target position inside the target workpiece is obtained.

[0006] In a possible implementation of the first aspect, generating coordinate sets of four edges of the target workpiece according to the first point coordinate set, the second point coordinate set, the third point coordinate set, and the fourth point coordinate set includes: Generate a coordinate set of the first side by fitting according to the coordinates of the points belonging to the first side of the target workpiece in the first point coordinate set and the third point coordinate set; Generate a coordinate set of the second side by fitting according to the coordinates of the points belonging to the second side of the target workpiece in the second point coordinate set and the fourth point coordinate set; Generate a coordinate set of the third side by fitting according to the coordinates of the points belonging to the third side of the target workpiece in the first point coordinate set and the second point coordinate set; A coordinate set of the fourth side is generated by fitting according to the coordinates of the points in the third point coordinate set and the fourth point coordinate set that belong to the fourth side of the target workpiece.

[0007] The coordinate sets of the four sides of the target workpiece are generated according to the coordinate set of the first side, the coordinate set of the second side, the coordinate set of the third side, and the coordinate set of the fourth side.

[0008] In a possible implementation manner of the first aspect, the target position is a central position inside the target workpiece, and acquiring position information of the target position inside the target workpiece according to the coordinate set of the four edges includes: According to the coordinate sets of the four sides of the target workpiece, obtaining the coordinate points of the first corner point, the second corner point, the third corner point, and the fourth corner point of the target workpiece; Obtaining coordinates of midpoints of the four sides according to the coordinates of the first corner point, the second corner point, the third corner point, and the fourth corner point; Acquiring candidate position information of the center position of the target workpiece according to the midpoint coordinates; According to the angle of the line connecting the midpoint coordinates of two parallel sides in the midpoint coordinates, the candidate position information is corrected to obtain the position information of the center position of the target workpiece.

[0009] In a possible implementation manner of the first aspect, acquiring candidate position information of the center position of the target workpiece according to the midpoint coordinates includes: According to the first midpoint coordinates of the first side and the second midpoint coordinates of the second side, obtaining a first connecting line between the first midpoint coordinates and the second midpoint coordinates; According to the third midpoint coordinates of the third side and the fourth midpoint coordinates of the fourth side, obtaining a second connecting line between the third midpoint coordinates and the fourth midpoint coordinates; Acquire candidate position information of the center position according to the first connection line and the second connection line; The step of correcting the candidate position information according to the angle of the line connecting the midpoint coordinates of two parallel sides in the midpoint coordinates to obtain the position information of the center position of the target workpiece includes: According to the third midpoint coordinates and the fourth midpoint coordinates, obtaining a connection angle of the second connection line; Determine an angle deviation according to an angle difference between the connecting angles of the first connecting line and the second connecting line; According to the angle deviation, the candidate position information is corrected to obtain the position information of the center position of the target workpiece.

[0010] In a possible implementation of the first aspect, the fitting and generating the coordinate set of the first edge according to the point coordinates of the first point coordinate set and the third point coordinate set belonging to the first edge of the target workpiece includes: According to the mutual distance relationship between the point coordinates of the first edge, detecting and removing abnormal point coordinates whose mutual distance relationship is greater than or equal to a preset distance threshold, and generating a set of point coordinates to be fitted; Determining a reliability weight of the coordinates of the point in the set of coordinates of the point to be fitted according to the position of the coordinates of the point in the set of coordinates of the point to be fitted in the field of view of the first camera or the second camera; Obtaining a preliminary fitting result of the first edge based on a weighted least square method according to the coordinate set of the points to be fitted and the reliability weight; Acquire an ordered point set of points belonging to the first edge in the preliminary fitting result, and a preset segmentation rule, wherein the preset segmentation rule is used to segment the ordered point set into a plurality of sub-point sets; According to the point coordinate sets included in each of the sub-point sets and the reliability weights corresponding to the point coordinate sets included in each of the sub-point sets, a weighted least square method is used to perform fitting to obtain fitting results of each sub-segment corresponding to each of the sub-point sets; According to the difference measure between the fitting results of the adjacent sub-segments and the preset difference threshold, the sub-segments whose difference measure is less than or equal to the preset difference threshold are merged and fitted again to generate the fitting result of the first edge; A coordinate set of the first edge is generated according to the fitting result of the first edge.

[0011] In a possible implementation manner of the first aspect, obtaining a coordinate deviation between the first camera and the second camera includes: Acquire a first coordinate of a first reference point photographed by the first camera at a reference position, and a second coordinate of a second reference point photographed by the second camera at the reference position, wherein the distance between the first reference point and the second reference point is a preset distance; Acquire a third coordinate of the first reference point photographed by the first camera after the first camera is rotated forward, and a fourth coordinate of the second reference point photographed by the second camera after the second camera is rotated forward; Acquire a fifth coordinate of the first reference point taken by the first camera after the first camera is inverted, and a sixth coordinate of the second reference point taken by the second camera after the second camera is inverted; The coordinate deviation between the first camera and the second camera is calculated according to the preset distance, the first coordinate, the second coordinate, the third coordinate, the fourth coordinate, the fifth coordinate, and the sixth coordinate.

[0012] In a possible implementation of the first aspect, the first camera and the second camera are cameras that have been distortion-corrected by a chessboard placed thereunder, and the first camera and the second camera are cameras that have completed multi-point calibration based on the first reference point and the second reference point.

[0013] In combination with the second aspect of the present application, a cross-field multi-camera calibrated positioning system is provided, the cross-field multi-camera calibrated positioning system comprising a machine-readable storage medium and a processor, the machine-readable storage medium storing machine-executable instructions, and when the processor executes the machine-executable instructions, the cross-field multi-camera calibrated positioning system implements the aforementioned cross-field multi-camera calibrated positioning method.

[0014] In conjunction with the third aspect of the present application, a computer-readable storage medium is provided, in which computer-executable instructions are stored. When the computer-executable instructions are executed, the aforementioned cross-field multi-camera calibration positioning method is implemented.

[0015] In conjunction with the fourth aspect of the present application, a computer program product is provided, and when the computer program is executed by a processor, the computer program implements the aforementioned cross-field multi-camera calibration positioning method.

[0016] In combination with any of the above aspects, by obtaining the coordinate deviation of the first camera and the second camera, the position of the first camera is parallel to the position of the second camera, and the coordinates of the first camera and the second camera when detecting the target point are unified in the same coordinate system according to the coordinate deviation. The first camera detects the first corner point of the target workpiece, the second corner point of the second point coordinate set, the second camera detects the third corner point of the target workpiece, and the fourth corner point of the target workpiece. The target workpiece is a rectangular workpiece. According to the first point coordinate set, the second point coordinate set, the third point coordinate set, and the fourth point coordinate set, the coordinate sets of the four sides of the target workpiece are generated, and according to the coordinate sets of the four sides, the position information of the target position inside the target workpiece is obtained. Thereby, the positioning accuracy of the workpiece is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained by combining these drawings without paying creative work.

[0018] Figure 1 A flowchart of a positioning method for cross-field multi-camera calibration provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of a first camera and a second camera provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. According to the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0021] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0022] Figure 1 The flowchart of the cross-view multi-camera calibration positioning method provided by the embodiment of the present application is shown. It should be understood that in other embodiments, the order of some steps of the cross-view multi-camera calibration positioning method of this embodiment can be shared with each other according to actual needs, or some steps can be omitted or maintained. The details of the cross-view multi-camera calibration positioning method include: Step S110, obtaining the coordinate deviation of the first camera and the second camera.

[0023] The position of the first camera is parallel to the position of the second camera.

[0024] In this embodiment, Figure 2 This is a schematic diagram of the structure of a first camera and a second camera provided in an embodiment of the present application. Figure 2 As shown in the figure, the device is divided into two parts. The upper part is the camera positioning module, which includes the first camera and the second camera. The first camera and the second camera take images of the four corners of the target workpiece for processing. The lower part is the adjustment platform, which performs camera alignment correction through the compensation value calculated by the camera positioning module.

[0025] The coordinate deviation is the deviation between the coordinates of the first camera and the second camera. If there is a large coordinate deviation, the coordinate data of different corners of the target workpiece photographed will have a large deviation, which is equivalent to the deviation of the coordinate system used by the first camera and the second camera. Therefore, it is necessary to obtain the coordinate deviation to correct the coordinate data obtained when the camera is shooting, so as to improve the accuracy of subsequent processing.

[0026] Optionally, the coordinate deviation of the first camera and the second camera can be directly obtained from a database. For example, the coordinate deviation of the first camera and the second camera is known in advance and stored in the database. When executing this method, the corresponding coordinate deviation is obtained from the database according to the first camera and the second camera.

[0027] Optionally, the coordinate deviation between the first camera and the second camera may also be temporarily obtained through calibration when executing the method, for example, by calibration with reference points and derivation of coordinate data obtained by multiple camera shots.

[0028] Step S120: unify the coordinates of the target point detected by the first camera and the second camera into the same coordinate system according to the coordinate deviation.

[0029] In this embodiment, after obtaining the coordinate deviation, when the first camera and the second camera detect the target point, the coordinate information of the target point detected by the first camera is obtained, and the coordinates of the first camera are adjusted according to the coordinate deviation. For example, if the coordinate deviation is [dx, dy], and the coordinates of the target point detected by the first camera are [x bn ,y bn ], then the adjusted coordinates are [x bn +dx,y bn +dy].

[0030] Alternatively, after obtaining the coordinate information of the target point detected by the second camera, similar adjustment operations can be performed according to the coordinate deviation. In this way, the server unifies the coordinates of the target point detected by the first camera and the second camera into the same coordinate system, ensuring that subsequent detection and positioning operations of the target workpiece can be performed in the unified coordinate system.

[0031] Step S130, obtaining a first point coordinate set of a first corner point and a second point coordinate set of a second corner point detected by the first camera, and a third point coordinate set of a third corner point and a fourth point coordinate set of a fourth corner point detected by the second camera.

[0032] Wherein, the target workpiece is a rectangular workpiece.

[0033] In this embodiment, an instruction to detect the corner point of the target workpiece may be sent to the first camera and the second camera. After receiving the instruction, the first camera starts to detect the first corner point of the target workpiece. The first camera acquires image data of the first corner point through its imaging system, and processes the image data (for example, identifying corner point features and other operations, which are based on image processing algorithms pre-stored in the first camera), thereby acquiring coordinate information of the first corner point obtained after processing, and forming a first point coordinate set based on the coordinate information.

[0034] Next, the first camera continues to detect the second corner point of the target workpiece. Similarly, the coordinate information of the second corner point is obtained according to the detection result to generate a second point coordinate set.

[0035] While the first camera detects the first corner point of the target workpiece, the second camera detects the third corner point of the target workpiece to obtain coordinate information of the detected third corner point, and generates a third point coordinate set based on the coordinate information. Correspondingly, while the first camera detects the second corner point of the target workpiece, the second camera detects the fourth corner point of the target workpiece to obtain coordinate information of the fourth corner point, and generates a fourth point coordinate set based on the coordinate information.

[0036] Step S140 , generating coordinate sets of four sides of the target workpiece according to the first point coordinate set, the second point coordinate set, the third point coordinate set, and the fourth point coordinate set.

[0037] In this embodiment, a fitting straight line of the corresponding edge may be generated by fitting according to the point coordinates of the same edge of the target workpiece in different point coordinate sets, and a coordinate set of the corresponding edge may be obtained according to the fitting straight line.

[0038] For example, if the first point coordinate set and the third point coordinate set include point coordinates on edge 1 of the target workpiece, fitting can be performed based on these point coordinates on edge 1 to generate a fitting straight line corresponding to edge 1, thereby obtaining the coordinate set on the fitting straight line as the coordinate set corresponding to edge 1 of the target workpiece.

[0039] Step S150: acquiring position information of a target position inside the target workpiece according to the coordinate set of the four edges.

[0040] According to the coordinate set of the four sides of the target workpiece, the coordinates of the first corner point, the second corner point, the third corner point and the fourth corner point of the target workpiece are calculated according to the pre-stored corner point coordinate calculation algorithm. For example, for a rectangular workpiece, the coordinates of the first corner point may be the coordinates of the intersection of the first side and the third side, and the coordinates of the intersection are obtained by solving the equations of the two sides (obtaining the equations of the sides according to the coordinate set of the sides).

[0041] According to the coordinate points of the first corner point, the second corner point, the third corner point and the fourth corner point, the coordinate range corresponding to the internal area of ​​the target workpiece is obtained. When the position information of the target position inside the target workpiece needs to be obtained, it can be calculated according to the coordinate range and the coordinate points of the four corner points. The target position can be, for example, any position inside the target workpiece, such as the center position, or any other position.

[0042] Based on the above steps, by obtaining the coordinate deviation of the first camera and the second camera, the position of the first camera is parallel to the position of the second camera, and the coordinates of the first camera and the second camera when detecting the target point are unified in the same coordinate system according to the coordinate deviation. The first camera detects the first corner point of the target workpiece, the second corner point of the second point coordinate set, and the second camera detects the third corner point of the target workpiece. The target workpiece is a rectangular workpiece. According to the first point coordinate set, the second point coordinate set, the third point coordinate set, and the fourth point coordinate set, the coordinate sets of the four sides of the target workpiece are generated, and according to the coordinate sets of the four sides, the position information of the target position inside the target workpiece is obtained. Thereby, the positioning accuracy of the workpiece is improved.

[0043] In a possible implementation, generating the coordinate sets of four edges of the target workpiece according to the first point coordinate set, the second point coordinate set, the third point coordinate set, and the fourth point coordinate set includes: Generate a coordinate set of the first side by fitting according to the coordinates of the points belonging to the first side of the target workpiece in the first point coordinate set and the third point coordinate set; Generate a coordinate set of the second side by fitting according to the coordinates of the points belonging to the second side of the target workpiece in the second point coordinate set and the fourth point coordinate set; Generate a coordinate set of the third side by fitting according to the coordinates of the points belonging to the third side of the target workpiece in the first point coordinate set and the second point coordinate set; A coordinate set of the fourth side is generated by fitting according to the coordinates of the points in the third point coordinate set and the fourth point coordinate set that belong to the fourth side of the target workpiece.

[0044] The coordinate sets of the four sides of the target workpiece are generated according to the coordinate set of the first side, the coordinate set of the second side, the coordinate set of the third side, and the coordinate set of the fourth side.

[0045] In this embodiment, it is assumed that a detection task of a liquid crystal display screen (target workpiece) is being processed. When the first point coordinate set of the first corner point of the target workpiece detected by the first camera and the third point coordinate set of the third corner point of the target workpiece detected by the second camera have been obtained, the point coordinates belonging to the first side of the target workpiece can be screened out from the first point coordinate set and the third point coordinate set. Among them, these coordinate points are obtained by the camera through imaging the target workpiece and undergoing image recognition and coordinate conversion. For example, if the first side is the upper edge of the display screen, the coordinates of these points can be determined based on the positional relationship of the coordinates (such as points with larger ordinates and abscissas within a certain range).

[0046] By fitting the coordinates of the points belonging to the same edge, for example, by fitting using the least square method, a fitting straight line corresponding to the edge can be obtained. According to the fitting straight line, the coordinate set of the first edge located on the fitting straight line can be determined.

[0047] Similarly, the coordinate set of the second side, the coordinate set of the third side, and the coordinate set of the fourth side can all be generated by the above-mentioned method of generating the coordinate set of the first side, which will not be repeated here.

[0048] In a possible implementation manner, the target position is a central position inside the target workpiece, and acquiring position information of the target position inside the target workpiece according to the coordinate set of the four edges includes: According to the coordinate sets of the four sides of the target workpiece, obtaining the coordinate points of the first corner point, the second corner point, the third corner point, and the fourth corner point of the target workpiece; Obtaining coordinates of midpoints of the four sides according to the coordinates of the first corner point, the second corner point, the third corner point, and the fourth corner point; Acquiring candidate position information of the center position of the target workpiece according to the midpoint coordinates; According to the angle of the line connecting the midpoint coordinates of two parallel sides in the midpoint coordinates, the candidate position information is corrected to obtain the position information of the center position of the target workpiece.

[0049] In this embodiment, the coordinate set of the four sides of the target workpiece includes the discrete coordinate points of the four sides of the target workpiece in a unified coordinate system. For the first corner point, it is determined by analyzing the coordinate set of the first side (upper edge) and the third side (lower edge). Find the point with the smallest horizontal coordinate in the coordinate set of the first edge, and the point with the smallest horizontal coordinate in the coordinate set of the third edge, and then calculate the intersection of the straight line where these two points are located (i.e., the left edge of the rectangle) and the first side and the third side. This intersection is the coordinate point of the first corner point. The coordinate points of the second corner point, the third corner point, and the fourth corner point of the target workpiece can be obtained in the same way. According to the coordinates of the first corner point, the second corner point, the third corner point, and the fourth corner point, the coordinates of the midpoints of the four sides can be obtained. For example, according to the coordinates of the first corner point and the second corner point, the coordinates of the midpoint of the line connecting the first corner point and the second corner point can be obtained. Similarly, the coordinates of the midpoints of the line connecting the other three corner points can be obtained.

[0050] After obtaining the coordinate points of the midpoints of the four sides, the candidate position information of the center position of the target workpiece can be obtained by connecting the midpoints of two pairs of parallel sides.

[0051] Since the positions of the aforementioned coordinate points are obtained through fitting, there may be cumulative errors. Therefore, the candidate position information can be further corrected by taking the angle of the line connecting the midpoint coordinates of two parallel edges in the midpoint coordinates as a reference to obtain the position information of the center position of the target workpiece.

[0052] In a possible implementation manner, acquiring candidate position information of the center position of the target workpiece according to the midpoint coordinates includes: According to the first midpoint coordinates of the first side and the second midpoint coordinates of the second side, obtaining a first connecting line between the first midpoint coordinates and the second midpoint coordinates; According to the third midpoint coordinates of the third side and the fourth midpoint coordinates of the fourth side, obtaining a second connecting line between the third midpoint coordinates and the fourth midpoint coordinates; Acquire candidate position information of the center position according to the first connection line and the second connection line; The step of correcting the candidate position information according to the angle of the line connecting the midpoint coordinates of two parallel sides in the midpoint coordinates to obtain the position information of the center position of the target workpiece includes: According to the third midpoint coordinates and the fourth midpoint coordinates, obtaining a connection angle of the second connection line; Determine an angle deviation according to an angle difference between the connecting angles of the first connecting line and the second connecting line; According to the angle deviation, the candidate position information is corrected to obtain the position information of the center position of the target workpiece.

[0053] In this embodiment, according to the first midpoint coordinates of the first side and the second midpoint coordinates of the second side, a first connecting line between the first midpoint coordinates and the second midpoint coordinates is obtained. According to the third midpoint coordinates of the third side and the fourth midpoint coordinates of the fourth side, a second connecting line between the third midpoint coordinates and the fourth midpoint coordinates is obtained. According to the first connecting line and the second connecting line, candidate position information of the center position is obtained.

[0054] The connection angle of the second connecting line is obtained according to the third midpoint coordinate and the fourth midpoint coordinate. The angle deviation is determined according to the angle difference between the connection angles of the first connecting line and the second connecting line. According to the angle deviation, the candidate position information is corrected to obtain the position information of the center position of the target workpiece.

[0055] Specifically, according to the coordinates of the third midpoint and the fourth midpoint, determine the direction vector pointing from the third midpoint to the fourth midpoint. Through the relative relationship between the direction vector and the coordinate axis (such as the positive direction of the axis), determine the quadrant and approximate direction of the vector, so as to obtain the approximate angle range between the second connecting line and the positive direction of the axis, and then further determine the specific connecting line angle. Use a similar method to obtain the connecting line angle of the first connecting line. Compare the angles of the two connecting lines, and subtract the small angle from the large angle to obtain the angle difference, that is, the angle deviation. Determine a certain point of the target workpiece (such as the geometric center of the four sides) as the rotation center. According to the positive or negative of the angle deviation, rotate the candidate position point around the rotation center in the corresponding direction (positive is clockwise, negative is counterclockwise) to obtain the center position information after correction.

[0056] In a possible implementation, the step of fitting and generating the coordinate set of the first side according to the coordinates of the points in the first point coordinate set and the third point coordinate set belonging to the first side of the target workpiece includes: According to the mutual distance relationship between the point coordinates of the first edge, detecting and removing abnormal point coordinates whose mutual distance relationship is greater than or equal to a preset distance threshold, and generating a set of point coordinates to be fitted; Determining a reliability weight of the coordinates of the point in the set of coordinates of the point to be fitted according to the position of the coordinates of the point in the set of coordinates of the point to be fitted in the field of view of the first camera or the second camera; Obtaining a preliminary fitting result of the first edge based on a weighted least square method according to the coordinate set of the points to be fitted and the reliability weight; Acquire an ordered point set of points belonging to the first edge in the preliminary fitting result, and a preset segmentation rule, wherein the preset segmentation rule is used to segment the ordered point set into a plurality of sub-point sets; According to the point coordinate sets included in each of the sub-point sets and the reliability weights corresponding to the point coordinate sets included in each of the sub-point sets, a weighted least square method is used to perform fitting to obtain fitting results of each sub-segment corresponding to each of the sub-point sets; According to the difference measure between the fitting results of the adjacent sub-segments and the preset difference threshold, the sub-segments whose difference measure is less than or equal to the preset difference threshold are merged and fitted again to generate the fitting result of the first edge; A coordinate set of the first edge is generated according to the fitting result of the first edge.

[0057] In this embodiment, firstly, a first point coordinate set obtained by detecting the target workpiece by the first camera and a third point coordinate set obtained by detecting the same target workpiece by the second camera are obtained. Then, the point coordinates belonging to the first side of the target workpiece are screened out from these two sets. By starting to check the mutual distance relationship between these point coordinates, for example, the distance between each point and other points is calculated in turn. According to a pre-stored preset distance threshold (this threshold is determined based on factors such as the size and shape of the target workpiece and the accuracy of the camera), for each point, if the distance between it and most of the other points is greater than or equal to the preset distance threshold, the point is determined to be an abnormal point. These abnormal points are removed from the point coordinate set, and the remaining point coordinates constitute the point coordinate set to be fitted.

[0058] For each point coordinate in the set of point coordinates to be fitted, analyze its position in the field of view of the first camera or the second camera. If a point coordinate is located in the central area of ​​the camera's field of view, then a higher reliability weight can be assigned to this point coordinate, because the imaging in the central area of ​​the field of view is relatively more accurate and less affected by factors such as lens distortion. On the contrary, if a point coordinate is located in the edge area of ​​the camera's field of view, it can be assigned a lower reliability weight, because there may be more imaging errors at the edge of the field of view. In this way, the corresponding reliability weight can be determined for each point coordinate in the set of point coordinates to be fitted.

[0059] The weighted least squares method is used to perform the fitting operation by taking the coordinate set of the points to be fitted and the corresponding reliability weights as input. The weighted least squares method will pay more attention to the role of the coordinates of the points with high reliability (larger weight) in the fitting process according to the reliability weight of each point coordinate. After a series of calculations (these calculations are based on the mathematical principle of the weighted least squares method and are implemented by the corresponding algorithm, which will not be described here), the preliminary fitting result of the first side is obtained. This preliminary fitting result is a preliminary approximate description of the shape of the first side, such as a curve or straight line represented by a certain mathematical expression (depending on the actual shape of the first side of the target workpiece and the distribution of the point coordinates).

[0060] From the preliminary fitting results obtained above, extract the points belonging to the first edge, and then arrange them according to a natural order on the first edge (such as the order along the direction of the edge) to form an ordered point set. According to the pre-stored preset segmentation rule, the ordered point set is divided into several sub-point sets. The preset segmentation rule may be to segment according to a fixed number of points, or to segment according to a certain distribution feature of the point coordinates.

[0061] For each sub-point set, the point coordinate set and the corresponding reliability weight contained therein are obtained. Then, the weighted least squares method is used again to perform fitting operations on each sub-point set. Since each sub-point set has its own point coordinates and weights, the fitting results obtained for each sub-point set are different, and these fitting results correspond to different sub-segments of the first edge.

[0062] Calculate the difference measure between the fitting results of adjacent sub-segments. This difference measure is an indicator used to measure the similarity of the fitting results of adjacent sub-segments. For example, it can be calculated based on factors such as the slope difference and intercept difference of the fitting curve or line. According to the pre-stored preset difference threshold, if the difference measure between the fitting results of two adjacent sub-segments is less than or equal to the preset difference threshold, the two sub-segments are considered to be similar and can be merged. All sub-segments that meet the conditions are merged, and then the weighted least squares method is used to fit the merged point coordinate set (these point coordinates come from the merged sub-segments) and the corresponding reliability weights again. After such merging and re-fitting operations, the fitting result of the first side is finally obtained. This fitting result describes the shape and position of the first side of the target workpiece more accurately than the preliminary fitting result.

[0063] According to the fitting result of the first side, a series of points are selected on the shape of the first side according to certain accuracy requirements. The selection method of these points should be able to accurately represent the shape of the first side, for example, the points can be selected at equal intervals. The coordinates of these selected points are combined to generate the coordinate set of the first side. This coordinate set can fully describe the position and shape information of the first side of the target workpiece in the coordinate system.

[0064] In a possible implementation manner, obtaining the coordinate deviation between the first camera and the second camera includes: Acquire a first coordinate of a first reference point photographed by the first camera at a reference position, and a second coordinate of a second reference point photographed by the second camera at the reference position, wherein the distance between the first reference point and the second reference point is a preset distance; Acquire a third coordinate of the first reference point photographed by the first camera after the first camera is rotated forward, and a fourth coordinate of the second reference point photographed by the second camera after the second camera is rotated forward; Acquire a fifth coordinate of the first reference point taken by the first camera after the first camera is inverted, and a sixth coordinate of the second reference point taken by the second camera after the second camera is inverted; The coordinate deviation between the first camera and the second camera is calculated according to the preset distance, the first coordinate, the second coordinate, the third coordinate, the fourth coordinate, the fifth coordinate, and the sixth coordinate.

[0065] In this embodiment, the first reference point and the second reference point are used to calibrate the first camera and the second camera to determine the coordinate deviation between the two, and then the coordinate data of the target workpiece captured and collected by the first camera and / or the second camera are processed according to the coordinate deviation, so that the coordinate data collected by the two correspond to each other in the same coordinate system, thereby improving the accuracy of the coordinate data of the target workpiece.

[0066] First, a first coordinate of a first reference point photographed by the first camera at a reference position is obtained, and a second coordinate of a second reference point photographed by the second camera at the reference position is obtained, and the distance between the first reference point and the second reference point is a preset distance.

[0067] Then, the first camera and the second camera are rotated forward to obtain the third coordinates of the first reference point photographed by the first camera after the first camera is rotated forward, and the fourth coordinates of the second reference point photographed by the second camera after the second camera is rotated forward.

[0068] Next, the first camera and the second camera are reversed again to obtain the fifth coordinate of the first reference point photographed by the first camera after the first camera is reversed, and the sixth coordinate of the second reference point photographed by the second camera after the second camera is reversed.

[0069] According to the above six coordinates, preset distances, and assumed coordinate deviation parameters, three corresponding sets of two-variable quadratic equations are generated, and the equations are solved to obtain the coordinate deviation.

[0070] For example, assuming that the first camera photographs the first reference point at the reference position, and the second camera photographs the second reference point at the reference position, the two cameras can obtain the coordinate points [x a0 ,y a0 ] and [x b0 ,y b0 ], then rotate the first camera and the second camera forward once, and take pictures to get the coordinate point [x a1 ,y a1 ] and [x b1 ,y b1 ], and then reverse the first camera and the second camera to obtain the coordinate point [x a2 ,y a2 ] and [x b2 ,y b2 ], assuming that the preset distance between the first reference point and the second reference point is d, and the coordinate deviation of the second camera relative to the first camera is unknown and is set to [dx, dy], then three sets of quadratic equations can be obtained: (x b0 +dx-x a0 ) 2 +(y b0 +dy-ya0 ) 2 =d 2 (1) (x b1 +dx-x a1 ) 2 +(y b1 +dy-y a1 ) 2 =d 2 (2) (x b2 +dx-x a2 ) 2 +(y b2 +dy-y a2 ) 2 =d 2 (3) From this, we can solve the coordinate deviation of the second camera relative to the first camera [dx, dy], and the coordinates of the detection point in the second camera are [x bn +dx,y bn +dy]. At this point, the points detected by the first camera and the second camera are unified into the same coordinate system.

[0071] In a possible implementation, the first camera and the second camera are cameras that have been subjected to distortion correction by a chessboard placed thereunder, and the first camera and the second camera are cameras that have completed multi-point calibration based on the first reference point and the second reference point. For example, the first camera and the second camera may be calibrated at nine points by the first reference point and the second reference point, respectively. It should be understood that the specific manner of summarizing how to perform distortion correction by a chessboard and how to perform multi-point calibration of a camera by using reference points in this implementation can refer to the prior art, and this application will not elaborate on this.

[0072] In the above embodiments, a positioning system for cross-field multi-camera calibration for executing the above method embodiments has at least one processor, a control module (chip set) coupled to at least one of the (at least one) processors, a memory coupled to the control module, a non-volatile memory (NVM) / storage device coupled to the control module, at least one input / output device coupled to the control module, and a network interface coupled to the control module.

[0073] The processor may include at least one single-core or multi-core processor, and the processor may include any combination of general-purpose processors or dedicated processors (such as graphics processors, application processors, baseband processors, etc.). For some alternative implementations, the cross-view multi-camera calibration positioning system can be used as a cross-view multi-camera calibration positioning system device such as the gateway described in the embodiments of the present application.

[0074] For some alternative embodiments, a positioning system for cross-field multi-camera calibration may include at least one computer-readable medium (e.g., a memory or NVM / storage device) having instructions and at least one processor integrated with the at least one computer-readable medium and configured to execute the instructions to implement a module to perform the actions described in the present disclosure.

[0075] For one embodiment, the control module may include any suitable interface controller to provide any suitable interface to at least one of the processor(s) and / or any suitable device or component in communication with the control module.

[0076] The control module may include a memory controller module to provide an interface to the memory. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0077] The memory may be used, for example, to load and store data and / or instructions for a positioning system for cross-view multi-camera calibration. For one embodiment, the memory may include any suitable volatile memory, such as a suitable DRAM.

[0078] For one embodiment, the control module may include at least one input / output controller to provide an interface to the NVM / storage device and (at least one) input / output device.

[0079] For example, the NVM / storage device may be used to store data and / or instructions. The NVM / storage device may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable (at least one) non-volatile storage device (e.g., at least one hard disk drive (HDD), at least one compact disk (CD) drive, and / or at least one digital versatile disk (DVD) drive).

[0080] The NVM / storage device may include storage resources that are physically part of the device on which the positioning system for cross-view multi-camera calibration is installed, or it may be accessible to the device without being part of the device. For example, the NVM / storage device may be accessed via (at least one) input / output device based on a network.

[0081] (At least one) input / output device may provide an interface for the cross-view multi-camera calibrated positioning system to communicate with any other appropriate device, and the input / output device may include a communication component, a phonetic component, a sensor component, etc. A network interface may provide an interface for the cross-view multi-camera calibrated positioning system to communicate based on at least one network, and the cross-view multi-camera calibrated positioning system may wirelessly communicate with at least one component of a wireless network based on any standard and / or protocol of at least one wireless network standard and / or protocol, for example, access a wireless network based on a communication standard.

[0082] For one embodiment, at least one of the (at least one) processor may be loaded together with the logic of at least one controller of the control module (e.g., a memory controller module). For one embodiment, at least one of the (at least one) processor may be loaded together with the logic of at least one controller of the control module to form a system level load. For one embodiment, at least one of the (at least one) processor may be integrated on the same die with the logic of at least one controller of the control module. For one embodiment, at least one of the (at least one) processor may be integrated on the same die with the logic of at least one controller of the control module to form a system on chip (SoC).

[0083] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0084] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the positioning method for cross-field multi-camera calibration described in the aforementioned embodiment.

[0085] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps in the positioning method for cross-field multi-camera calibration described in the aforementioned embodiment.

[0086] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0087] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other medium that can be used to be computer-readable with or store data.

[0088] Finally, it should be noted that what is disclosed above is only a preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A positioning method for cross-field multi-camera calibration, characterized in that: include: Acquire a coordinate deviation between a first camera and a second camera, wherein a position of the first camera is parallel to a position of the second camera; Unifying the coordinates of the target point detected by the first camera and the second camera into the same coordinate system according to the coordinate deviation; Acquire a first point coordinate set of a first corner point and a second point coordinate set of a second corner point of a target workpiece detected by the first camera, a third point coordinate set of a third corner point and a fourth point coordinate set of a fourth corner point of the target workpiece detected by the second camera, and the target workpiece is a rectangular workpiece; Generate coordinate sets of four sides of the target workpiece according to the first point coordinate set, the second point coordinate set, the third point coordinate set, and the fourth point coordinate set; According to the coordinate set of the four edges, the position information of the target position inside the target workpiece is obtained.

2. The positioning method for cross-view multi-camera calibration according to claim 1, characterized in that: The generating of the coordinate sets of the four sides of the target workpiece according to the first point coordinate set, the second point coordinate set, the third point coordinate set, and the fourth point coordinate set includes: Generate a coordinate set of the first side by fitting according to the coordinates of the points belonging to the first side of the target workpiece in the first point coordinate set and the third point coordinate set; Generate a coordinate set of the second side by fitting according to the coordinates of the points belonging to the second side of the target workpiece in the second point coordinate set and the fourth point coordinate set; Generate a coordinate set of the third side by fitting according to the coordinates of the points belonging to the third side of the target workpiece in the first point coordinate set and the second point coordinate set; Generate a coordinate set of the fourth side by fitting according to the coordinates of the points belonging to the fourth side of the target workpiece in the third point coordinate set and the fourth point coordinate set; The coordinate sets of the four sides of the target workpiece are generated according to the coordinate set of the first side, the coordinate set of the second side, the coordinate set of the third side, and the coordinate set of the fourth side.

3. The positioning method for cross-view multi-camera calibration according to claim 2, characterized in that: The target position is a central position inside the target workpiece, and obtaining position information of the target position inside the target workpiece according to the coordinate set of the four edges includes: According to the coordinate sets of the four sides of the target workpiece, obtaining the coordinate points of the first corner point, the second corner point, the third corner point, and the fourth corner point of the target workpiece; Obtaining coordinates of midpoints of the four sides according to the coordinates of the first corner point, the second corner point, the third corner point, and the fourth corner point; Acquiring candidate position information of the center position of the target workpiece according to the midpoint coordinates; According to the angle of the line connecting the midpoint coordinates of two parallel sides in the midpoint coordinates, the candidate position information is corrected to obtain the position information of the center position of the target workpiece.

4. The positioning method for cross-view multi-camera calibration according to claim 3 is characterized in that: The step of acquiring candidate position information of the center position of the target workpiece according to the midpoint coordinates includes: According to the first midpoint coordinates of the first side and the second midpoint coordinates of the second side, obtaining a first connecting line between the first midpoint coordinates and the second midpoint coordinates; According to the third midpoint coordinates of the third side and the fourth midpoint coordinates of the fourth side, obtaining a second connecting line between the third midpoint coordinates and the fourth midpoint coordinates; Acquire candidate position information of the center position according to the first connection line and the second connection line; The step of correcting the candidate position information according to the angle of the line connecting the midpoint coordinates of two parallel sides in the midpoint coordinates to obtain the position information of the center position of the target workpiece includes: According to the third midpoint coordinates and the fourth midpoint coordinates, obtaining a connection angle of the second connection line; Determine an angle deviation according to an angle difference between the connecting angles of the first connecting line and the second connecting line; According to the angle deviation, the candidate position information is corrected to obtain the position information of the center position of the target workpiece.

5. The positioning method for cross-view multi-camera calibration according to claim 2, characterized in that: The step of fitting and generating the coordinate set of the first side according to the coordinates of the points in the first point coordinate set and the third point coordinate set belonging to the first side of the target workpiece comprises: According to the mutual distance relationship between the point coordinates of the first edge, detecting and removing abnormal point coordinates whose mutual distance relationship is greater than or equal to a preset distance threshold, and generating a set of point coordinates to be fitted; Determining a reliability weight of the coordinates of the point in the set of coordinates of the point to be fitted according to the position of the coordinates of the point in the set of coordinates of the point to be fitted in the field of view of the first camera or the second camera; Obtaining a preliminary fitting result of the first edge based on a weighted least square method according to the coordinate set of the points to be fitted and the reliability weight; Acquire an ordered point set of points belonging to the first edge in the preliminary fitting result, and a preset segmentation rule, wherein the preset segmentation rule is used to segment the ordered point set into a plurality of sub-point sets; According to the point coordinate sets included in each of the sub-point sets and the reliability weights corresponding to the point coordinate sets included in each of the sub-point sets, a weighted least square method is used to perform fitting to obtain fitting results of each sub-segment corresponding to each of the sub-point sets; According to the difference measure between the fitting results of the adjacent sub-segments and the preset difference threshold, the sub-segments whose difference measure is less than or equal to the preset difference threshold are merged and fitted again to generate the fitting result of the first edge; A coordinate set of the first edge is generated according to the fitting result of the first edge.

6. The positioning method for cross-view multi-camera calibration according to any one of claims 1 to 5, characterized in that: The obtaining of the coordinate deviation between the first camera and the second camera includes: Acquire a first coordinate of a first reference point photographed by the first camera at a reference position, and a second coordinate of a second reference point photographed by the second camera at the reference position, wherein the distance between the first reference point and the second reference point is a preset distance; Acquire a third coordinate of the first reference point photographed by the first camera after the first camera is rotated forward, and a fourth coordinate of the second reference point photographed by the second camera after the second camera is rotated forward; Acquire a fifth coordinate of the first reference point taken by the first camera after the first camera is inverted, and a sixth coordinate of the second reference point taken by the second camera after the second camera is inverted; The coordinate deviation between the first camera and the second camera is calculated according to the preset distance, the first coordinate, the second coordinate, the third coordinate, the fourth coordinate, the fifth coordinate, and the sixth coordinate.

7. The positioning method for cross-view multi-camera calibration according to claim 6, characterized in that: The first camera and the second camera are cameras that have been subjected to distortion correction by a chessboard placed thereunder, and the first camera and the second camera are cameras that have completed multi-point calibration based on the first reference point and the second reference point.

8. A cross-view multi-camera calibration positioning system, characterized in that: The invention comprises a processor and a computer-readable storage medium, wherein the computer-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by a computer, the positioning method for cross-field multi-camera calibration according to any one of claims 1 to 7 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by a computer, the positioning method for cross-field multi-camera calibration described in any one of claims 1 to 7 is implemented.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.

Images