Visual positioning method and device, computer equipment and storage medium

Abstract

The invention relates to a visual positioning method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring corresponding processing coordinates of a reference calibration object on a processing surface in a processing coordinate system, and a distribution distance between an auxiliary calibration object and the reference calibration object; the machining coordinate system is parallel to the machining surface, and the original point is on the key machine; determining a pixel distance corresponding to the distribution distance and a pixel coordinate corresponding to the reference calibration object based on a calibration image acquired by an image acquisition unit of which the optical axis is vertical to the processing surface; determining a coordinate conversion coefficient according to the pixel distance and the distribution distance; performing weighting processing on the pixel coordinates corresponding to the reference calibration object according to the coordinate conversion coefficient to obtain a weighting result; determining a coordinate offset based on the processing coordinate corresponding to the reference calibration object and the weighting result; and processing coordinates corresponding to the key are determined according to the coordinate conversion coefficient, the coordinate offset and a key image acquired by the image acquisition unit. By adopting the method, the positioning efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of mechanical processing technology, and in particular to a visual positioning method, device, computer equipment and storage medium. Background Art

[0002] As the types of keys in life continue to increase, the processing requirements for keys are becoming increasingly diversified and refined, and key machines are becoming more and more important. Key machines can process a variety of keys through advanced CNC technology and precise mechanical structures.

[0003] In the conventional technology, the key machine locates the key by manually inputting the position of the key. Obviously, this method is quite limited and cannot avoid the problem of low positioning efficiency. Summary of the invention

[0004] Based on this, it is necessary to provide a visual positioning method, device, computer equipment and storage medium that can improve positioning efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides a visual positioning method, comprising:

[0006] Obtaining a processing coordinate corresponding to a reference calibration object placed on a processing surface in a processing coordinate system; the origin of the processing coordinate system is located on the key machine, and the processing coordinate system is parallel to the processing surface;

[0007] Obtaining the distribution distance between the auxiliary calibration object and the reference calibration object placed on the processing surface;

[0008] Acquire a calibration image acquired by the image acquisition unit of the key machine of the auxiliary calibration object and the reference calibration object placed on the processing surface; the optical axis of the image acquisition unit is perpendicular to the processing surface;

[0009] Determine a pixel distance corresponding to the distribution distance based on the calibration image;

[0010] Determine a coordinate conversion coefficient according to the distribution distance and the pixel distance;

[0011] Determine pixel coordinates corresponding to the reference calibration object in a pixel coordinate system based on the calibration image;

[0012] Performing weighted processing on the pixel coordinates corresponding to the reference calibration object according to the coordinate conversion coefficient to obtain a weighted result;

[0013] Determining a coordinate offset based on the processing coordinates corresponding to the reference calibration object and the weighted result;

[0014] Acquiring a key image captured by the image acquisition unit from the key placed on the processing surface;

[0015] Coordinate transformation is performed on the pixel coordinates corresponding to the key in the key image according to the coordinate transformation coefficient and the coordinate offset to obtain the processing coordinates corresponding to the key.

[0016] In a second aspect, the present application also provides a visual positioning device, comprising:

[0017] A detection module, used for acquiring a processing coordinate corresponding to a reference calibration object placed on a processing surface in a processing coordinate system; the origin of the processing coordinate system is located on the key machine, and the processing coordinate system is parallel to the processing surface;

[0018] An acquisition module, used for acquiring the distribution distance between the auxiliary calibration objects and the reference calibration objects placed on the processing surface;

[0019] An acquisition module, used for acquiring a calibration image acquired by an image acquisition unit of the key machine from an auxiliary calibration object and a reference calibration object placed on the processing surface; the optical axis of the image acquisition unit is perpendicular to the processing surface;

[0020] A calibration module, used to determine the pixel distance corresponding to the distribution distance based on the calibration image; determine the coordinate conversion coefficient according to the distribution distance and the pixel distance; determine the pixel coordinates corresponding to the reference calibration object in the pixel coordinate system based on the calibration image; perform weighted processing on the pixel coordinates corresponding to the reference calibration object according to the coordinate conversion coefficient to obtain a weighted result; determine the coordinate offset based on the processing coordinates corresponding to the reference calibration object and the weighted result;

[0021] The acquisition module is further used to obtain the key image captured by the image acquisition unit from the key placed on the processing surface;

[0022] The positioning module is used to perform coordinate transformation on the pixel coordinates corresponding to the key in the key image according to the coordinate transformation coefficient and the coordinate offset to obtain the processing coordinates corresponding to the key.

[0023] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above method when executing the computer program.

[0024] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps in the above method when executed by a processor.

[0025] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the steps in the above method when executed by a processor.

[0026] In the above-mentioned visual positioning method, device, computer equipment, storage medium and computer program product, the origin of the processing coordinate system is located on the key machine, and the processing coordinate system is parallel to the processing surface. Moreover, because the optical axis of the image acquisition unit of the key machine is perpendicular to the processing surface of the key machine, the pixel coordinate system of the image acquired by the image acquisition unit is also parallel to the processing surface, so the pixel coordinate system and the processing coordinate system are parallel to each other, so that only a simple two-dimensional calibration is required to realize the coordinate conversion from the pixel coordinate system to the processing coordinate system in the processing surface, which greatly reduces the computational complexity and improves the calibration efficiency. By obtaining the processing coordinates corresponding to the reference calibration object placed on the processing surface in the processing coordinate system; obtaining the distribution distance between the auxiliary calibration object and the reference calibration object placed on the processing surface; obtaining the calibration image acquired by the image acquisition unit of the key machine of the auxiliary calibration object and the reference calibration object placed on the processing surface; determining the pixel distance corresponding to the distribution distance based on the calibration image; determining the coordinate conversion coefficient according to the distribution distance and the pixel distance; determining the pixel coordinates corresponding to the reference calibration object in the pixel coordinate system based on the calibration image; weighting the pixel coordinates corresponding to the reference calibration object according to the coordinate conversion coefficient to obtain a weighted result; determining the coordinate offset based on the processing coordinates corresponding to the reference calibration object and the weighted result, a two-dimensional calibration from the pixel coordinate system to the processing coordinate system is realized, and the coordinate conversion coefficient and the coordinate offset can convert the pixel coordinates in the pixel coordinate system into the processing coordinates in the processing coordinate system. Furthermore, the key machine obtains a key image captured by the image acquisition unit of the key on the key machine; obtains a key image captured by the image acquisition unit of the key placed on the processing surface; and performs coordinate conversion on the pixel coordinates corresponding to the key in the key image according to the coordinate conversion coefficient and the coordinate offset to obtain the processing coordinates corresponding to the key. The key can be positioned on the key machine without manually inputting the position of the key, thereby greatly improving the positioning efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A flowchart of a visual positioning method provided in an embodiment of the present application.

[0028] Figure 2 A schematic diagram of a probe detection initialization mark provided in an embodiment of the present application.

[0029] Figure 3 A schematic diagram of the origin coordinates in a processing coordinate system, the processing coordinates corresponding to the calibration plate, and the processing coordinates corresponding to the reference calibration object provided in an embodiment of the present application.

[0030] Figure 4 A schematic diagram of a calibration plate provided in an embodiment of the present application.

[0031] Figure 5A schematic diagram of the origin coordinates of a pixel coordinate system and the pixel coordinates corresponding to a reference calibration object provided in an embodiment of the present application.

[0032] Figure 6 A schematic diagram of a two-dimensional calibration process provided in an embodiment of the present application.

[0033] Figure 7 A flowchart of a circle center update algorithm provided in an embodiment of the present application.

[0034] Figure 8a A schematic diagram of pixel coordinates of a feature point provided in an embodiment of the present application.

[0035] Figure 8b A schematic diagram of a probe probing a tooth-shaped structure provided in an embodiment of the present application.

[0036] Fig. 9 This is a structural block diagram of a visual positioning device provided in an embodiment of the present application.

[0037] Fig.10 This is a diagram of the internal structure of a key machine provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0039] In some embodiments, the method provided in the present application can be applied to at least one of a numerical control device or a computer device. The numerical control device may include a key machine. The computer device may include at least one of a terminal or a server.

[0040] In an exemplary embodiment, Figure 1 As shown, a flow chart of a visual positioning method is provided, which is described by taking the method applied to a key machine as an example, and includes the following steps 102 to 120.

[0041] Step 102, obtaining the processing coordinates corresponding to the reference calibration object placed on the processing surface in the processing coordinate system; the origin of the processing coordinate system is located on the key machine, and the processing coordinate system is parallel to the processing surface.

[0042] Among them, the processing surface refers to the surface of the key machine that processes the key. The processing surface can be located on the fixture of the key machine. The processing coordinate system is a two-dimensional coordinate system on the key machine, and the origin of the processing coordinate system is fixed relative to the position of the key machine. The processing coordinates corresponding to any object in the processing coordinate system are used to characterize the position of the object orthogonally projected into the processing coordinate system. It can be understood that the key machine is usually equipped with a special position detector, such as a probe, etc., to realize the detection of workpieces such as keys. The key machine can first realize two-dimensional positioning through the processing coordinates in the processing coordinate system, and then control the position detector to detect in the third dimension to realize positioning in three-dimensional space. No manual operation is required, which can greatly improve the positioning efficiency. In addition, after the key machine realizes two-dimensional positioning through the processing coordinates in the processing coordinate system, the user can also manually operate the key machine to realize positioning in the third dimension. Compared with the method of manually inputting all positions, it can also improve the positioning efficiency.

[0043] In some embodiments, the key machine may obtain processing coordinates corresponding to the reference calibration object placed on the input processing surface in the processing coordinate system.

[0044] In some embodiments, the key machine may control the position detector to detect the reference calibration object placed on the processing surface to obtain the processing coordinates corresponding to the reference calibration object in the processing coordinate system.

[0045] In some embodiments, the reference calibration object is located on the calibration plate. The key machine can control the position detector to detect the calibration plate to obtain the processing coordinates corresponding to the calibration plate in the processing coordinate system. The processing coordinates corresponding to the calibration plate are offset according to the relative position information of the reference calibration object on the calibration plate to obtain the processing coordinates corresponding to the reference calibration object.

[0046] In some embodiments, there are at least three calibration objects on the calibration plate, one of which is used as a reference calibration object, and at least two calibration objects other than the reference calibration object are used as auxiliary calibration objects, and the centers of the at least two auxiliary calibration objects are not on the same straight line as the center of the reference calibration object.

[0047] In some embodiments, the position detector may be, but is not limited to, a probe.

[0048] Step 104 , obtaining the distribution distance between the auxiliary calibration objects and the reference calibration objects placed on the processing surface.

[0049] Exemplarily, the key cutting machine may obtain the distribution distance between the auxiliary calibration object and the reference calibration object placed on the input processing surface.

[0050] In some embodiments, the distribution distance between the auxiliary calibration object and the reference calibration object can be automatically identified and measured by the image size measuring instrument. The distribution distance between the auxiliary calibration object and the reference calibration object can also be manually measured.

[0051] In some embodiments, the key machine can control the probe to detect the distribution distance between the auxiliary calibration object and the reference calibration object placed on the processing surface.

[0052] In some embodiments, the auxiliary calibration object and the reference calibration object are calibration objects on the calibration plate. The key machine can obtain the distribution distance between the auxiliary calibration object and the reference calibration object on the calibration plate input.

[0053] In some embodiments, the distribution distance may include a first distribution distance and a second distribution distance in a first machine coordinate axis direction and a second machine coordinate axis direction of the machine coordinate system, respectively.

[0054] Step 106, obtaining a calibration image acquired by an image acquisition unit of the key machine of the auxiliary calibration object and the reference calibration object placed on the processing surface; the optical axis of the image acquisition unit is perpendicular to the processing surface.

[0055] The calibration image includes image regions to which the auxiliary calibration object and the reference calibration object are projected respectively.

[0056] Exemplarily, the auxiliary calibration object and the reference calibration object are both located on the calibration plate. The key cutting machine can obtain a calibration image captured by the image acquisition unit of the calibration plate placed on the processing surface.

[0057] In some embodiments, when auxiliary calibration objects and reference calibration objects are placed on the processing surface, the key machine can control the image acquisition unit to perform image acquisition in response to the image acquisition operation to obtain a calibration image.

[0058] In some embodiments, the image acquisition unit may be, but is not limited to, a camera or a scanner.

[0059] Step 108: Determine the pixel distance corresponding to the distribution distance based on the calibration image.

[0060] The pixel distance is used to characterize the distance between the image area where the auxiliary calibration object is projected and the image area where the reference calibration object is projected. It can be understood that after the auxiliary calibration object and the reference calibration object are projected onto the image plane, the distribution distance between the two is expressed as a pixel distance on the image plane.

[0061] For example, the key machine can perform image recognition based on the calibration image to determine the image area where the auxiliary calibration object is projected and the image area where the reference calibration object is projected. According to the spacing between the image area where the auxiliary calibration object is projected and the image area where the reference calibration object is projected, the pixel distance corresponding to the distribution distance is determined.

[0062] In some embodiments, the reference calibration objects and auxiliary calibration objects have specific markings to facilitate identification and measurement in the image. The reference calibration objects and auxiliary calibration objects can be two-dimensional regular patterns, such as a checkerboard pattern or a circular pattern. The reference calibration objects and auxiliary calibration objects can also be three-dimensional regular objects, such as a three-dimensional calibration block or a calibration sphere.

[0063] In some embodiments, the reference calibration object and the auxiliary calibration object have a regular first shape and a regular second shape, respectively. The first shape and the second shape may be the same or different. The key machine may perform area detection of the first shape and the second shape based on the calibration image, and determine the image area that conforms to the first shape and the image area that conforms to the second shape. The image area to which the reference calibration object is projected is determined based on the image area that conforms to the first shape. The image area to which the auxiliary calibration object is projected is determined based on the image area that conforms to the second shape.

[0064] In some embodiments, the key machine can determine the area center of the image area where the auxiliary calibration object is projected to obtain the area center corresponding to the auxiliary calibration object. Determine the area center of the image area where the reference calibration object is projected to obtain the area center corresponding to the reference calibration object. Determine the pixel distance corresponding to the distribution distance based on the distance between the area center corresponding to the auxiliary calibration object and the area center corresponding to the reference calibration object.

[0065] In some embodiments, the user may manually mark the pixel distance corresponding to the distribution distance in the calibration image. The key machine may obtain the pixel distance corresponding to the distribution distance marked in the calibration image.

[0066] In some embodiments, the pixel distance may include a first pixel distance in a first pixel coordinate axis direction and a second pixel distance in a second pixel coordinate axis direction in the pixel coordinate system.

[0067] In some embodiments, the first pixel coordinate axis direction may be parallel to the first process coordinate axis direction. The second pixel coordinate axis direction may be parallel to the second process coordinate axis direction.

[0068] Step 110, determining a coordinate conversion coefficient according to the distribution distance and the pixel distance.

[0069] Exemplarily, the distribution distance may include a first distribution distance and a second distribution distance. The pixel distance may include a first pixel distance and a second pixel distance. The coordinate conversion coefficient may include a first conversion coefficient corresponding to the first pixel coordinate axis direction and a second conversion coefficient corresponding to the second pixel coordinate axis direction. The key machine may determine the first conversion coefficient according to the ratio of the first distribution distance to the first pixel distance. The second conversion coefficient may be determined according to the ratio of the second distribution distance to the second pixel distance.

[0070] In some embodiments, there are at least two auxiliary calibration objects. The key machine can determine the distribution distance corresponding to each auxiliary calibration object and the pixel distance corresponding to the distribution distance. The distribution distance corresponding to the auxiliary calibration object refers to the distribution distance between the auxiliary calibration object and the reference calibration object. The distribution distance includes a first distribution distance and a second distribution distance. The pixel distance includes a first pixel distance and a second pixel distance. For the first distribution distance and the second distribution distance corresponding to each auxiliary calibration object, the ratio of the first distribution distance to the corresponding first pixel distance is determined as the first reference coefficient. The ratio of the second distribution distance to the corresponding second pixel distance is determined as the second reference coefficient. At least one of the outliers or the extreme values ​​is removed from each first reference coefficient to obtain the remaining first reference coefficient. The first conversion coefficient is determined based on the remaining first reference coefficient. At least one of the outliers or the extreme values ​​is removed from each second reference coefficient to obtain the remaining second reference coefficient. The second conversion coefficient is determined based on the remaining second reference coefficient.

[0071] In some embodiments, on the processing surface of the key cutting machine, the centers of at least two auxiliary calibration objects and the center of the reference calibration object are not in a straight line.

[0072] In some embodiments, the maximum value may include at least one of a maximum value or a minimum value. The method of removing outliers may be, but is not limited to, a quartile method or a clustering algorithm. It should be noted that the order of removing outliers and removing the maximum value is not specifically limited in this embodiment.

[0073] In some embodiments, the key machine may select a first conversion coefficient from the remaining first reference coefficients. The key machine may also perform mean calculation on the remaining first reference coefficients to obtain the first conversion coefficient. The key machine may select a second conversion coefficient from the remaining second reference coefficients. The key machine may also perform mean calculation on the remaining second reference coefficients to obtain the second conversion coefficient.

[0074] Step 112: Determine the pixel coordinates corresponding to the reference calibration object in the pixel coordinate system based on the calibration image.

[0075] Exemplarily, the key machine may perform image recognition based on the calibration image to determine the image region where the reference calibration object is projected. Based on the image region where the reference calibration object is projected, the pixel coordinates corresponding to the reference calibration object in the pixel coordinate system are determined.

[0076] In some embodiments, the key machine can determine the area center of the image area where the auxiliary calibration object is projected to obtain the area center corresponding to the auxiliary calibration object, and determine the pixel coordinates of the area center corresponding to the auxiliary calibration object as the pixel coordinates corresponding to the reference calibration object in the pixel coordinate system.

[0077] In some embodiments, the origin of the pixel coordinate system is located at the image position corresponding to the initialization mark. The key machine can obtain the image position corresponding to the initialization mark, and use the image position as the origin of the pixel coordinate system to initialize the pixel coordinate system.

[0078] In some embodiments, the key machine can use the image position as the cropping starting point, and crop the calibration image along the two coordinate axis directions of the initial pixel coordinate system of the calibration image to obtain a cropped image. It can be understood that the origin of the pixel coordinate system of the image is at the "upper left corner" of the image, and in the cropped image obtained after the calibration image is cropped, the image position is located at the "upper left corner", so the origin of the target pixel coordinate system of the cropped image is located at the above-mentioned image position.

[0079] In some embodiments, the key machine may determine the pixel coordinates corresponding to the reference calibration object in the target pixel coordinate system.

[0080] In some embodiments, the user may manually mark the pixel coordinates corresponding to the reference calibration object in the calibration image. The key machine may obtain the pixel coordinates corresponding to the reference calibration object marked in the calibration image.

[0081] Step 114 , weighting the pixel coordinates corresponding to the reference calibration object according to the coordinate conversion coefficient to obtain a weighted result.

[0082] Exemplarily, the coordinate conversion coefficient may include a first conversion coefficient corresponding to the first pixel coordinate axis direction, and a second conversion coefficient corresponding to the second pixel coordinate axis direction. The pixel coordinate may include a first pixel coordinate value in the first pixel coordinate axis direction, and a second pixel coordinate value in the second pixel coordinate axis direction. The weighted result may include a first weighted coordinate value and a second weighted coordinate value. The key machine may calculate the product of the first conversion coefficient and the first pixel coordinate value corresponding to the reference calibration object to obtain a first weighted coordinate value. The product of the second conversion coefficient and the second pixel coordinate value corresponding to the reference calibration object is calculated to obtain a second weighted coordinate value.

[0083] Step 116 , determining the coordinate offset based on the processing coordinates corresponding to the reference calibration object and the weighted result.

[0084] Exemplarily, the weighted result may include a first weighted coordinate value and a second weighted coordinate value. The processing coordinate may include a first processing coordinate value in the direction of the first processing coordinate axis and a second processing coordinate value in the direction of the second processing coordinate axis. The coordinate offset may include a first offset and a second offset. The key machine may calculate the difference between the first processing coordinate value corresponding to the reference calibration object and the first weighted coordinate value to obtain a first offset. The difference between the second processing coordinate value corresponding to the reference calibration object and the second weighted coordinate value is calculated to obtain a second offset.

[0085] In some embodiments, the first conversion coefficient and the first offset are used to convert the first pixel coordinate value into the first processing coordinate value. Specifically, Px = px × x_rate + x_bias. Wherein, Px represents the first processing coordinate value. px represents the first pixel coordinate value. x_rate represents the first conversion coefficient. x_bias represents the first offset. The second conversion coefficient and the second offset are used to convert the second pixel coordinate value into the second processing coordinate value. Specifically, Py = py × y_rate + y_bias. Wherein, Py represents the second processing coordinate value. py represents the second pixel coordinate value. y_rate represents the second conversion coefficient. y_bias represents the second offset.

[0086] Step 118 , obtaining a key image captured by the image acquisition unit of the key placed on the processing surface.

[0087] Exemplarily, when a key is placed on the processing surface, the key machine may control the image acquisition unit to perform image acquisition in response to an image acquisition operation to obtain a key image.

[0088] In some embodiments, the processing surface may be located on the fixture. The key machine may obtain a key image captured by the image acquisition unit of the key clamped on the fixture.

[0089] Step 120 , coordinate transformation is performed on the pixel coordinates corresponding to the key in the key image according to the coordinate transformation coefficient and the coordinate offset to obtain the processing coordinates corresponding to the key.

[0090] Exemplarily, the coordinate conversion coefficient may include a first conversion coefficient corresponding to the first pixel coordinate axis direction, and a second conversion coefficient corresponding to the second pixel coordinate axis direction. The coordinate offset may include a first offset and a second offset. The pixel coordinate may include a first pixel coordinate value and a second pixel coordinate value. The processing coordinate may include a first processing coordinate value and a second processing coordinate value. The key machine may calculate the product of the first conversion coefficient and the first pixel coordinate value corresponding to the key, and then superimpose the first offset to obtain the first processing coordinate value corresponding to the key. After calculating the product of the second conversion coefficient and the second pixel coordinate value corresponding to the key, and then superimpose the second offset to obtain the second processing coordinate value corresponding to the key.

[0091] In some embodiments, the processing coordinates corresponding to the key may include the processing coordinates corresponding to the tooth structure of the key in the processing coordinate system. The key machine can identify the tooth area from the key image. The tooth area refers to the image area where the tooth structure of the key is projected. The pixel coordinates of the feature point are extracted from the tooth area. The pixel coordinates of the feature point are transformed according to the coordinate conversion coefficient and the coordinate offset to obtain the processing coordinates corresponding to the key. It should be noted that this embodiment does not specifically limit the method for identifying the tooth area and the method for extracting the feature points.

[0092] In the above-mentioned visual positioning method, the origin of the processing coordinate system is located on the key machine, and the processing coordinate system is parallel to the processing surface. Moreover, because the optical axis of the image acquisition unit of the key machine is perpendicular to the processing surface of the key machine, the pixel coordinate system of the image acquired by the image acquisition unit is also parallel to the processing surface, so the pixel coordinate system and the processing coordinate system are parallel to each other, so only a simple two-dimensional calibration is needed to realize the coordinate conversion from the pixel coordinate system to the processing coordinate system in the processing surface, which greatly reduces the computational complexity and improves the calibration efficiency. By obtaining the processing coordinates corresponding to the reference calibration object placed on the processing surface in the processing coordinate system; obtaining the distribution distance between the auxiliary calibration object and the reference calibration object placed on the processing surface; obtaining the calibration image acquired by the image acquisition unit of the key machine of the auxiliary calibration object and the reference calibration object placed on the processing surface; determining the pixel distance corresponding to the distribution distance based on the calibration image; determining the coordinate conversion coefficient according to the distribution distance and the pixel distance; determining the pixel coordinates corresponding to the reference calibration object in the pixel coordinate system based on the calibration image; weighting the pixel coordinates corresponding to the reference calibration object according to the coordinate conversion coefficient to obtain a weighted result; determining the coordinate offset based on the processing coordinates corresponding to the reference calibration object and the weighted result, a two-dimensional calibration from the pixel coordinate system to the processing coordinate system is realized, and the coordinate conversion coefficient and the coordinate offset can convert the pixel coordinates in the pixel coordinate system into the processing coordinates in the processing coordinate system. Furthermore, the key machine obtains a key image captured by the image acquisition unit of the key on the key machine; obtains a key image captured by the image acquisition unit of the key placed on the processing surface; and performs coordinate conversion on the pixel coordinates corresponding to the key in the key image according to the coordinate conversion coefficient and the coordinate offset to obtain the processing coordinates corresponding to the key. The key can be positioned on the key machine without manually inputting the position of the key, thereby greatly improving the positioning efficiency.

[0093] In some embodiments, obtaining the processing coordinates corresponding to the reference calibration object placed on the processing surface in the processing coordinate system includes: controlling the probe of the key machine to detect the initialization mark on the key machine; using the spatial position of the detected initialization mark as the origin position of the processing coordinate system to initialize the processing coordinate system parallel to the processing surface; controlling the probe to detect the boundary of the calibration plate placed on the processing surface to obtain the processing coordinates corresponding to the calibration plate in the processing coordinate system; obtaining the boundary spacing between the reference calibration object on the calibration plate and the boundary of the calibration plate; and determining the processing coordinates corresponding to the reference calibration object in the processing coordinate system based on the boundary spacing and the processing coordinates corresponding to the calibration plate.

[0094] In some embodiments, the position of the initialization mark is the position where the probe is initialized. Figure 2The diagram shows a probe detecting an initialization mark. The initialization mark can be, but is not limited to, a boss on a fixture. The fixture is in a matching relationship with the key machine and is assembled at a fixed position on the key machine. The boss is located on an immovable part of the fixture, so the position of the boss relative to the key machine is fixed. Figure 2 In the example, when clamping the key, the right part of the fixture can move to the right, while the left part is fixed, and the boss is located on the left part. The key machine can control the probe to detect the boss on the fixture to obtain the spatial position of the target corner point of the boss. The target corner point can refer to the upper right corner position of the boss. The spatial position of the target corner point is used as the origin position of the machining coordinate system, and a machining coordinate system parallel to the machining surface is established.

[0095] In some embodiments, the key machine may obtain a boundary distance between an input reference calibration object on the calibration plate and a boundary of the calibration plate.

[0096] In some embodiments, the key machine may perform an offset process on the processing coordinates corresponding to the calibration plate according to the boundary spacing to obtain the processing coordinates corresponding to the reference calibration object.

[0097] In some embodiments, the boundary spacing may be automatically identified and measured by an image size measuring instrument, or may be manually measured.

[0098] In some embodiments, Figure 3 As shown, a schematic diagram of the origin coordinates in the processing coordinate system, the processing coordinates corresponding to the calibration plate, and the processing coordinates corresponding to the reference calibration object is provided. The origin coordinate P0 in the processing coordinate system is located at the upper right corner of the boss. The processing coordinate P1 corresponding to the calibration plate is located at the intersection of two mutually perpendicular boundaries on the calibration plate. The reference calibration object is a circular pattern on the calibration plate that is closest to the origin coordinates, and the processing coordinates corresponding to the reference calibration object are located at the center of the circular pattern.

[0099] In some embodiments, the spatial position detected by the probe is essentially a three-dimensional coordinate in a three-dimensional coordinate system, and the directions of two machining coordinate axes in the machining coordinate system are respectively parallel to the directions of two probe coordinate axes in the three-dimensional coordinate system that are parallel to the machining surface.

[0100] In this embodiment, the initialization mark on the key machine is detected by controlling the probe of the key machine; the spatial position of the detected initialization mark is used as the origin position of the processing coordinate system, and the processing coordinate system parallel to the processing surface is initialized, thereby realizing the initialization of the processing coordinate system. Furthermore, the probe is controlled to detect the boundary of the calibration plate placed on the processing surface to obtain the processing coordinates corresponding to the calibration plate in the processing coordinate system; the boundary of the calibration plate is easier to detect than the reference calibration object on the calibration plate, which can improve the efficiency of detection. The boundary spacing between the reference calibration object on the calibration plate and the boundary of the calibration plate is obtained. The boundary spacing is a structural parameter of the calibration plate itself and can be directly obtained. Therefore, based on the boundary spacing and the processing coordinates corresponding to the calibration plate, the processing coordinates corresponding to the reference calibration object in the processing coordinate system can be accurately determined.

[0101] In some embodiments, the boundary of the calibration plate includes a first boundary and a second boundary that are perpendicular to each other; the probe is controlled to detect the boundary of the calibration plate placed on the processing surface to obtain the processing coordinates corresponding to the calibration plate in the processing coordinate system, including: when the calibration plate is placed on the processing surface and the first boundary and the second boundary are respectively perpendicular to the two coordinate axes in the processing coordinate system, the probe is controlled to detect the first vertical distance and the second vertical distance between the first boundary and the second boundary and the initialization mark respectively; based on the first vertical distance and the second vertical distance, the processing coordinates corresponding to the calibration plate in the processing coordinate system are determined.

[0102] Exemplarily, in order to ensure the efficiency of calibration, the position of the calibration plate when it is placed on the machining surface needs to satisfy the condition that the first boundary and the second boundary are respectively perpendicular to the two coordinate axes in the machining coordinate system. When the calibration plate is placed on the machining surface, and the first boundary and the second boundary are respectively perpendicular to the first machining coordinate axis and the second machining coordinate axis in the machining coordinate system, the control probe detects the first vertical distance between the first boundary and the spatial position of the initialization mark, and the second vertical distance between the second boundary and the spatial position of the initialization mark. The first vertical distance is used as the first machining coordinate value corresponding to the calibration plate, and the second vertical distance is used as the second machining coordinate value corresponding to the calibration plate to obtain the machining coordinate corresponding to the calibration plate.

[0103] In some embodiments, the boundary spacing may include a first boundary spacing and a second boundary spacing. The key machine may obtain a first boundary spacing between the center of the reference calibration object and the first boundary, and a second boundary spacing between the center of the reference calibration object and the second boundary. It is understood that if the reference calibration object is a circular pattern, then the center of the reference calibration object is the center of the circular pattern.

[0104] In some embodiments, the key machine can sum the first boundary spacing and the first processing coordinate value corresponding to the calibration plate to obtain the first processing coordinate value corresponding to the reference calibration object, and sum the second boundary spacing and the second processing coordinate value corresponding to the calibration plate to obtain the second processing coordinate value corresponding to the reference calibration object, thereby obtaining the processing coordinate corresponding to the reference calibration object.

[0105] In some embodiments, the structure of the calibration plate meets the requirements of the key structure. For example, the thickness of the calibration plate adopts the average thickness of the sample key, and the length and width of the calibration plate cover the size of the sample key.

[0106] In some embodiments, Figure 4 As shown, a schematic diagram of a calibration plate is provided. The calibration plate is in a shape similar to a key. The calibration plate includes a plurality of circular patterns and a first boundary and a second boundary perpendicular to each other. The reference calibration object is the first circular pattern in the upper left corner. In addition to the reference calibration object, the remaining circular patterns are auxiliary calibration objects.

[0107] In some embodiments, in the calibration image, the image area where the reference calibration object is projected is in the shape of a circle. The key machine can perform circle detection on the calibration image to obtain an initial center of the circular area. Based on the initial center, the pixel coordinates corresponding to the reference calibration object in the pixel coordinate system are determined.

[0108] In some embodiments, the circle detection method may be, but is not limited to, detecting the circle using Hough transform after Gaussian filtering.

[0109] In some embodiments, the key machine can determine the initial circle center corresponding to the reference calibration object from each initial circle center, and use the pixel coordinates of the corresponding initial circle center as the pixel coordinates corresponding to the reference calibration object.

[0110] In some embodiments, in the calibration image, the shape of the image area to which the auxiliary calibration object is projected is also circular. The key machine can determine the initial circle center corresponding to the auxiliary calibration object from each initial circle center. According to the distance between the initial circle center corresponding to the auxiliary calibration object and the initial circle center corresponding to the reference calibration object, the pixel distance corresponding to the distribution distance is determined.

[0111] In some embodiments, the key machine can perform edge detection on the calibration image to obtain a set of edge pixels for each image area. The range of the circular ring is determined based on the range parameter and the initial center and initial radius of the circular area. The target edge pixels located within the circular ring range are determined from the set of edge pixels. The initial center of the circular area is updated according to the target edge pixels to obtain the target center. The target centers corresponding to the auxiliary calibration object and the reference calibration object are determined from each target center. The pixel distance corresponding to the distribution distance is determined based on the distance between the target center corresponding to the auxiliary calibration object and the target center corresponding to the reference calibration object.

[0112] In this embodiment, the boundaries of the calibration plate include a first boundary and a second boundary that are perpendicular to each other; when the calibration plate is placed on the processing surface and the first boundary and the second boundary are respectively perpendicular to the two coordinate axes in the processing coordinate system, the control probe detects the first vertical distance and the second vertical distance between the first boundary and the second boundary and the initialization mark respectively; the first vertical distance and the second vertical distance reflect the distance of the calibration plate relative to the initialization mark in the two coordinate axis directions in the processing coordinate system, and the spatial position of the initialization mark is the origin of the processing coordinate system. Therefore, based on the first vertical distance and the second vertical distance, the processing coordinates corresponding to the calibration plate in the processing coordinate system can be accurately determined, thereby ensuring the accuracy of the calibration.

[0113] In some embodiments, the pixel coordinates corresponding to the reference calibration object in the pixel coordinate system are determined based on the calibration image, including: obtaining the image position for the initialization identification mark; using the image position as the cropping starting point, and cropping the calibration image along the two coordinate axis directions of the initial pixel coordinate system of the calibration image to obtain a cropped image; the origin of the target pixel coordinate system of the cropped image is located at the image position; and determining the pixel coordinates corresponding to the reference calibration object in the target pixel coordinate system based on the cropped image.

[0114] Exemplarily, the key machine can obtain the image position for the initialization identification mark. It can be understood that the parameters of the image acquisition unit are fixed, and the image positions in each image acquired are one-to-one corresponding. Therefore, the image position for the initialization identification mark can be marked in any image acquired by the image acquisition unit, and it is not necessarily marked in the calibration image. The key machine can use the image position as the cropping starting point, and crop the calibration image according to the cropping size parameters along the first pixel coordinate axis direction and the second pixel coordinate axis direction of the initial pixel coordinate system of the calibration image to obtain a cropped image. Image recognition is performed on the cropped image to obtain the pixel coordinates corresponding to the reference calibration object in the target pixel coordinate system.

[0115] In some embodiments, in the calibration image, the image area to which the reference calibration object is projected is in the shape of a circle. The key machine can perform circle detection on the cropped image to obtain an initial center of the circular area. Based on the initial center, the pixel coordinates corresponding to the reference calibration object in the pixel coordinate system are determined.

[0116] In some embodiments, the key machine can determine the initial circle center corresponding to the reference calibration object from each initial circle center, and use the pixel coordinates of the corresponding initial circle center as the pixel coordinates corresponding to the reference calibration object.

[0117] In some embodiments, in the calibration image, the shape of the image area to which the auxiliary calibration object is projected is also circular. The key machine can determine the initial circle center corresponding to the auxiliary calibration object from each initial circle center. According to the distance between the initial circle center corresponding to the auxiliary calibration object and the initial circle center corresponding to the reference calibration object, the pixel distance corresponding to the distribution distance is determined.

[0118] In this embodiment, the image position corresponding to the initialization mark is obtained; the image position is used as the clipping starting point, and the calibration image is clipped along the two coordinate axis directions of the initial pixel coordinate system of the calibration image to obtain a clipped image; the pixel coordinates corresponding to the reference calibration object in the target pixel coordinate system are determined based on the clipped image. The clipped image is smaller in size than the calibration image, which can reduce the amount of image processing calculations. In addition, the origin of the target pixel coordinate system of the clipped image is located at the image position corresponding to the initialization mark, which is consistent with the origin of the processing coordinate system, and can ensure the accuracy of the two-dimensional calibration.

[0119] In some embodiments, in the calibration image, the shape of the image area to which the reference calibration object is projected is a circle; the pixel coordinates corresponding to the reference calibration object in the pixel coordinate system are determined based on the cropped image, including: performing circle detection on the cropped image to obtain an initial center and initial radius of the circular area; performing edge detection on the cropped image to obtain a set of edge pixels of each image area; determining a circular ring range based on a range parameter and the initial center and initial radius of the circular area; determining target edge pixels within the circular ring range from the edge pixel set; updating the initial center of the circular area according to the target edge pixels to obtain the target center; and determining the pixel coordinates corresponding to the reference calibration object in the pixel coordinate system based on the target center.

[0120] Exemplarily, the key machine can perform Gaussian filtering on the cropped image and then use the Hough transform to detect the circle to obtain the pixel coordinates and initial radius of the initial center of each circle center area. It can be understood that because the Hough circle detection is sensitive to noise, the cropped image is first Gaussian filtered. The cropped image is enhanced to obtain an enhanced image. Specifically, the key machine can enhance the cropped image by contrast limited adaptive histogram equalization (CLAHE). After bilateral filtering the enhanced image, edge detection is performed to obtain a set of edge pixels in each image area. The edge detection method can be but is not limited to canny edge detection. Calculate the difference between the initial radius and the range parameter to obtain the radius starting value, sum the initial radius and the range parameter to obtain the radius ending value. Determine the range of the ring formed by an inner circle with the initial center as the center and a radius equal to the radius starting value, and an outer circle with the initial center as the center and a radius equal to the radius ending value. Search for the target edge pixel within the ring range from the edge pixel set. If the number of target edge pixels is less than a preset number, the initial center of the circular area is determined as the target center of the circular area. If the number of target edge pixels is not less than a preset number, the center of the circle where each target edge pixel is located is determined as the target center of the circular area. The key machine can determine the target center corresponding to the reference calibration object from each target center. The pixel coordinates of the target center corresponding to the reference calibration object are determined as the pixel coordinates corresponding to the reference calibration object. It can be understood that the target center corresponding to the reference calibration object is the target center of the center area to which the reference calibration object is projected.

[0121] In some embodiments, the key machine can use the Open Source Computer Vision Library (OpenCV) to find checkerboard corners, circular grid extraction algorithms, or sub-pixel corner detection algorithms to identify the target center of the circular area from the cropped image.

[0122] In this embodiment, in the calibration image, the shape of the image area to which the reference calibration object is projected is a circle; by performing circle detection on the cropped image, the initial center and initial radius of the circular area are obtained; edge detection is performed on the cropped image to obtain a set of edge pixels of each image area; the range of the circular ring is determined based on the range parameter and the initial center and initial radius of the circular area; the target edge pixels located within the circular ring range are determined from the edge pixel set; the initial center of the circular area is updated according to the target edge pixels to obtain the target center; the pixel coordinates corresponding to the reference calibration object in the pixel coordinate system are determined based on the target center, so that the pixel coordinates corresponding to the reference calibration object can be automatically identified without manual marking, thereby improving the efficiency of two-dimensional calibration.

[0123] In some embodiments, in the calibration image, the shape of the image area to which the auxiliary calibration object is projected is also circular; the pixel coordinates corresponding to the reference calibration object in the target pixel coordinate system are determined based on the target center, including: determining the target center corresponding to the reference calibration object and the target center corresponding to the auxiliary calibration object from each target center; determining the pixel coordinates of the target center corresponding to the reference calibration object as the pixel coordinates corresponding to the reference calibration object in the pixel coordinate system; determining the pixel distance corresponding to the distribution distance based on the calibration image, including: determining the pixel distance corresponding to the distribution distance according to the distance between the target center corresponding to the auxiliary calibration object and the target center corresponding to the reference calibration object.

[0124] The target circle center corresponding to the auxiliary calibration object is the target circle center of the circular area to which the auxiliary calibration object is projected.

[0125] For example, Figure 5 As shown, the origin coordinates of the pixel coordinate system and the pixel coordinates corresponding to the reference calibration object are provided. The pixel coordinates p2 corresponding to the reference calibration object are closest to the origin coordinates p0 of the pixel coordinate system, and the key machine can determine the target circle center closest to the origin coordinates of the pixel coordinate system as the target circle center corresponding to the reference calibration object. The relative position relationship between the auxiliary calibration object and the reference calibration object placed on the processing surface is consistent with the relative position relationship between the target circle centers in the cropped image. The key machine can determine the auxiliary calibration object corresponding to each target circle center based on the relative position relationship between the target circle centers. For example, for the target circle center that is adjacent to and in the same row as the target circle center corresponding to the reference calibration object, the corresponding auxiliary calibration object is the auxiliary calibration object that is adjacent to and in the same row as the reference calibration object. The pixel coordinates of the target circle center corresponding to the reference calibration object are determined to be the pixel coordinates corresponding to the reference calibration object. The distance between the target circle center corresponding to the auxiliary calibration object and the target circle center corresponding to the reference calibration object is determined to be the pixel distance corresponding to the distribution distance.

[0126] In some embodiments, the pixel coordinates include a first pixel coordinate value and a second pixel coordinate value. The pixel distance includes a first pixel distance and a second pixel distance. The key machine can calculate the difference between the first pixel coordinate value of the target circle center corresponding to the auxiliary calibration object and the first pixel coordinate value of the target circle center corresponding to the reference calibration object to obtain the first pixel distance. The difference between the second pixel coordinate value of the target circle center corresponding to the auxiliary calibration object and the second pixel coordinate value of the target circle center corresponding to the reference calibration object is calculated to obtain the second pixel distance.

[0127] In this embodiment, the target center corresponding to the reference calibration object and the target center corresponding to the auxiliary calibration object are determined from each target center; the pixel coordinates of the target center corresponding to the reference calibration object are determined as the pixel coordinates corresponding to the reference calibration object in the pixel coordinate system; the pixel distance corresponding to the distribution distance is determined according to the distance between the target center corresponding to the auxiliary calibration object and the target center corresponding to the reference calibration object. The pixel coordinates and pixel distance corresponding to the reference calibration object can be automatically identified without manual marking, thereby improving the efficiency of two-dimensional calibration.

[0128] In some embodiments, the method further includes: controlling a probe of the key machine to move to a processing coordinate corresponding to the key, and move along a vertical direction of the processing surface, and detecting a spatial position corresponding to the key by the probe.

[0129] Exemplarily, the processing coordinates corresponding to the key may include processing coordinates corresponding to the tooth structure of the key in the processing coordinate system. The probe is controlled to move to the processing coordinate corresponding to the tooth structure and move along the vertical direction of the processing surface, and the spatial position corresponding to the tooth structure is detected by the probe.

[0130] In this embodiment, the probe of the key machine is controlled to move to the processing coordinate corresponding to the key, and move along the vertical direction of the processing surface. The spatial position corresponding to the key is detected by the probe. The vertical direction of the processing surface is the third direction perpendicular to the two-dimensional coordinate system of the processing coordinate system. The key is detected by the probe in the third direction perpendicular to the processing coordinate system. The detected spatial position corresponding to the key can be used to more accurately locate the key compared to the processing coordinate corresponding to the key, thereby improving the positioning accuracy.

[0131] In some embodiments, Figure 6 As shown, a schematic diagram of the process of two-dimensional calibration is provided. When the calibration plate is clamped by the fixture and the first boundary and the second boundary are respectively perpendicular to the two coordinate axes in the processing coordinate system, the key machine can control the image acquisition unit to acquire the image and obtain the calibration image. The key machine can use the image position marked for the initialization mark as the starting point for cropping, and crop the calibration image along the two coordinate axis directions of the initial pixel coordinate system of the calibration image to obtain a cropped image. After Gaussian filtering the cropped image, the Hough transform detection circle is used to obtain the pixel coordinates and initial radius of the initial center of each circle center area. Image enhancement is performed on the cropped image to obtain an enhanced image. After bilateral filtering the enhanced image, edge detection is performed to obtain a set of edge pixels in each image area.

[0132] Based on the initial circle center and edge pixel set, the target circle center is obtained using the circle center update algorithm. Specifically, Figure 7As shown, a flow chart of the circle center update algorithm is provided. The difference between the initial radius and the range parameter is calculated to obtain the radius starting value, and the initial radius and the range parameter are summed to obtain the radius ending value. Determine the ring range formed by an inner circle with the initial circle center as the circle center and a radius equal to the radius starting value, and an outer circle with the initial circle center as the circle center and a radius equal to the radius ending value. Search for target edge pixels within the ring range from the edge pixel set. If the number of target edge pixels is not less than the preset number, the target circle center is determined based on the target edge pixels, otherwise, the initial circle center is used as the target circle center.

[0133] The key machine can determine the target center corresponding to the reference calibration object and the target center corresponding to the auxiliary calibration object from each target center according to the relative position relationship of each target center. The pixel coordinates of the target center corresponding to the reference calibration object are determined as the pixel coordinates corresponding to the reference calibration object in the pixel coordinate system. The difference between the first pixel coordinate value of the target center corresponding to the auxiliary calibration object and the first pixel coordinate value of the target center corresponding to the reference calibration object is calculated to obtain the first pixel distance. The difference between the second pixel coordinate value of the target center corresponding to the auxiliary calibration object and the second pixel coordinate value of the target center corresponding to the reference calibration object is calculated to obtain the second pixel distance. The first distribution distance and the second distribution distance between the auxiliary calibration object and the reference calibration object are obtained. For each first distribution distance and second distribution distance corresponding to the auxiliary calibration object, the ratio of the first distribution distance to the corresponding first pixel distance is determined as the first reference coefficient. The ratio of the second distribution distance to the corresponding second pixel distance is determined as the second reference coefficient. At least one of the outliers or the maximum value is removed from each first reference coefficient to obtain the remaining first reference coefficient. The remaining first reference coefficients are averaged to obtain the first conversion coefficient. At least one of the outliers and the maximum values ​​is removed from each second reference coefficient to obtain the remaining second reference coefficients. The remaining second reference coefficients are averaged to obtain the second conversion coefficients.

[0134] Although the origin of the machining coordinate system is obtained by detecting the initialization mark, and the origin of the pixel coordinate system is the image position marked by the initialization mark, since the image position of the mark may not be accurate, the pixel coordinates corresponding to the origin of the machining coordinate system may not be the image position of the mark, resulting in an offset between the origin of the machining coordinate system and the origin of the pixel coordinate system. Figure 3 and Figure 5 As shown, Figure 3 The origin coordinates P0 of the machining coordinate system in Figure 5The origin coordinate p0 of the pixel coordinate system in the real scene corresponds to different spatial positions, and there is an offset. The above offset can be eliminated by the coordinate offset. Therefore, two-dimensional calibration not only needs to calculate the coordinate conversion coefficient, but also needs to calculate the coordinate offset. The key machine can calculate the product of the first conversion coefficient and the first pixel coordinate value corresponding to the reference calibration object to obtain the first weighted coordinate value. Calculate the product of the second conversion coefficient and the second pixel coordinate value corresponding to the reference calibration object to obtain the second weighted coordinate value. Calculate the difference between the first processing coordinate value corresponding to the reference calibration object and the first weighted coordinate value to obtain the first offset. Calculate the difference between the second processing coordinate value corresponding to the reference calibration object and the second weighted coordinate value to obtain the second offset.

[0135] Furthermore, the key machine can convert the first pixel coordinate value into the first processing coordinate value based on the first conversion coefficient and the first offset, specifically, Px = px × x_rate + x_bias. Wherein, Px represents the first processing coordinate value. px represents the first pixel coordinate value. x_rate represents the first conversion coefficient. x_bias represents the first offset. The second pixel coordinate value is converted into the second processing coordinate value based on the second conversion coefficient and the second offset. Specifically, Py = py × y_rate + y_bias. Wherein, Py represents the second processing coordinate value. py represents the second pixel coordinate value. y_rate represents the second conversion coefficient. y_bias represents the second offset.

[0136] The key machine can identify the tooth-shaped area from the key image. The tooth-shaped area refers to the image area where the tooth structure of the key is projected. The pixel coordinates of the feature points are extracted from the tooth-shaped area. Figure 8a As shown, a schematic diagram of the pixel coordinates of feature points is provided. Figure 8a The tooth-shaped area where the feature point is located is the image area where the tooth-shaped structure of the inner milling groove type is projected, and the feature point is located in the tooth-shaped area close to one end of the key handle.

[0137] The key machine can perform coordinate transformation on the pixel coordinates of the feature point based on the first transformation coefficient and the first offset, and the second transformation coefficient and the second offset, to obtain the processing coordinates corresponding to the tooth structure of the key. Figure 8b As shown, a schematic diagram of the probe detecting the tooth structure is provided. The key machine can control the probe to move to the processing coordinate corresponding to the tooth structure, and move along the vertical direction of the processing surface, and detect the spatial position corresponding to the tooth structure through the probe.

[0138] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0139] Based on the same inventive concept, the embodiment of the present application also provides a visual positioning device for implementing the visual positioning method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more visual positioning device embodiments provided below can refer to the limitations of the visual positioning method above, and will not be repeated here.

[0140] In an exemplary embodiment, Fig. 9 As shown, a visual positioning device 900 is provided, including: a detection module 902 , an acquisition module 904 , a collection module 906 , a calibration module 908 and a positioning module 910 .

[0141] The detection module 902 is used to obtain the processing coordinates corresponding to the reference calibration object placed on the processing surface in the processing coordinate system; the origin of the processing coordinate system is located on the key machine, and the processing coordinate system is parallel to the processing surface.

[0142] The acquisition module 904 is used to acquire the distribution distance between the auxiliary calibration objects and the reference calibration objects placed on the processing surface.

[0143] The acquisition module 906 is used to obtain the calibration image acquired by the image acquisition unit of the key machine from the auxiliary calibration object and the reference calibration object placed on the processing surface; the optical axis of the image acquisition unit is perpendicular to the processing surface.

[0144] The calibration module 908 is used to determine the pixel distance corresponding to the distribution distance based on the calibration image; determine the coordinate conversion coefficient according to the distribution distance and the pixel distance; determine the pixel coordinates corresponding to the reference calibration object in the pixel coordinate system based on the calibration image; perform weighted processing on the pixel coordinates corresponding to the reference calibration object according to the coordinate conversion coefficient to obtain a weighted result; and determine the coordinate offset based on the processing coordinates corresponding to the reference calibration object and the weighted result.

[0145] The acquisition module 906 is further used to obtain a key image captured by the image acquisition unit of the key placed on the processing surface.

[0146] The positioning module 910 is used to perform coordinate transformation on the pixel coordinates corresponding to the key in the key image according to the coordinate transformation coefficient and the coordinate offset to obtain the processing coordinates corresponding to the key.

[0147] In some embodiments, the detection module 902 is used to control the probe of the key machine to detect the initialization mark on the key machine; use the spatial position of the detected initialization mark as the origin position of the processing coordinate system to initialize the processing coordinate system parallel to the processing surface; control the probe to detect the boundary of the calibration plate placed on the processing surface to obtain the processing coordinates corresponding to the calibration plate in the processing coordinate system; obtain the boundary spacing between the reference calibration object on the calibration plate and the boundary of the calibration plate; determine the processing coordinates corresponding to the reference calibration object in the processing coordinate system based on the boundary spacing and the processing coordinates corresponding to the calibration plate.

[0148] In some embodiments, the boundary of the calibration plate includes a first boundary and a second boundary that are perpendicular to each other; the detection module 902 is used to control the probe to detect the first vertical distance and the second vertical distance between the first boundary and the second boundary and the initialization mark respectively when the calibration plate is placed on the processing surface and the first boundary and the second boundary are respectively perpendicular to the two coordinate axes in the processing coordinate system; based on the first vertical distance and the second vertical distance, determine the processing coordinates corresponding to the calibration plate in the processing coordinate system.

[0149] In some embodiments, the calibration module 908 is used to obtain the image position for the initialization identification mark; use the image position as the cropping starting point, and crop the calibration image along the two coordinate axis directions of the initial pixel coordinate system of the calibration image to obtain a cropped image; the origin of the target pixel coordinate system of the cropped image is located at the image position; and determine the pixel coordinates corresponding to the reference calibration object in the target pixel coordinate system based on the cropped image.

[0150] In some embodiments, in the calibration image, the shape of the image area to which the reference calibration object is projected is a circle; the calibration module 908 is used to perform circle detection on the cropped image to obtain an initial center and initial radius of the circular area; perform edge detection on the cropped image to obtain a set of edge pixels for each image area; determine a circular ring range based on a range parameter and an initial center and initial radius of the circular area; determine target edge pixels within the circular ring range from the edge pixel set; update the initial center of the circular area according to the target edge pixels to obtain a target center; and determine the pixel coordinates corresponding to the reference calibration object in the pixel coordinate system based on the target center.

[0151] In some embodiments, in the calibration image, the shape of the image area to which the auxiliary calibration object is projected is also circular; the calibration module 908 is used to determine the target center corresponding to the reference calibration object and the target center corresponding to the auxiliary calibration object from each target center; determine the pixel coordinates of the target center corresponding to the reference calibration object as the pixel coordinates corresponding to the reference calibration object in the pixel coordinate system; determine the pixel distance corresponding to the distribution distance based on the distance between the target center corresponding to the auxiliary calibration object and the target center corresponding to the reference calibration object.

[0152] In some embodiments, the positioning module 910 is used to control the probe of the key machine to move to the processing coordinates corresponding to the key and move along the vertical direction of the processing surface to detect the spatial position corresponding to the key through the probe.

[0153] In some embodiments, the acquisition module 904 is used to acquire the distribution distance between the auxiliary calibration object and the reference calibration object automatically identified and measured by the image size measuring instrument.

[0154] In some embodiments, the acquisition module 906 is used to control the image acquisition unit to perform image acquisition in response to an image acquisition operation to obtain a calibration image when auxiliary calibration objects and reference calibration objects are placed on the processing surface; and to control the image acquisition unit to perform image acquisition in response to an image acquisition operation to obtain a key image when a key is placed on the processing surface.

[0155] Each module in the above-mentioned visual positioning device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0156] In an exemplary embodiment, a key machine is provided, and its internal structure diagram can be shown as follows: Fig.10As shown. The key machine includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the key machine is used to provide computing and control capabilities. The memory of the key machine includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the key machine is used to exchange information between the processor and an external device. The communication interface of the key machine is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a visual positioning method is implemented. The display unit of the key machine is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the key machine can be a touch layer covering the display screen, or a button, trackball or touchpad set on the key machine housing, or an external keyboard, touchpad or mouse.

[0157] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component placement.

[0158] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0159] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0160] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0161] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0162] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A visual positioning method, characterized in that: The method comprises: Obtaining a processing coordinate corresponding to a reference calibration object placed on a processing surface in a processing coordinate system; the origin of the processing coordinate system is located on the key machine, and the processing coordinate system is parallel to the processing surface; Obtaining a distribution distance between the auxiliary calibration object and the reference calibration object placed on the processing surface; Acquire a calibration image acquired by the image acquisition unit of the key machine of the auxiliary calibration object and the reference calibration object placed on the processing surface; the optical axis of the image acquisition unit is perpendicular to the processing surface; Determine a pixel distance corresponding to the distribution distance based on the calibration image; Determine a coordinate conversion coefficient according to the distribution distance and the pixel distance; Determine pixel coordinates corresponding to the reference calibration object in a pixel coordinate system based on the calibration image; Performing weighted processing on the pixel coordinates corresponding to the reference calibration object according to the coordinate conversion coefficient to obtain a weighted result; Determining a coordinate offset based on the processing coordinates corresponding to the reference calibration object and the weighted result; Acquiring a key image captured by the image acquisition unit from the key placed on the processing surface; Coordinate transformation is performed on the pixel coordinates corresponding to the key in the key image according to the coordinate transformation coefficient and the coordinate offset to obtain the processing coordinates corresponding to the key.

2. The method according to claim 1, characterized in that The step of obtaining the processing coordinates corresponding to the reference calibration object placed on the processing surface in the processing coordinate system includes: Controlling the probe of the key machine to detect the initialization mark on the key machine; Using the detected spatial position of the initialization mark as the origin position of the processing coordinate system, initializing the processing coordinate system parallel to the processing surface; Controlling the probe to detect the boundary of the calibration plate placed on the processing surface to obtain the processing coordinates corresponding to the calibration plate in the processing coordinate system; Acquire a boundary distance between a reference calibration object on the calibration plate and a boundary of the calibration plate; The processing coordinates corresponding to the reference calibration object in the processing coordinate system are determined based on the boundary distance and the processing coordinates corresponding to the calibration plate.

3. The method according to claim 2, characterized in that The boundary of the calibration plate includes a first boundary and a second boundary that are perpendicular to each other; and the controlling the probe to detect the boundary of the calibration plate placed on the processing surface to obtain the processing coordinates corresponding to the calibration plate in the processing coordinate system includes: When the calibration plate is placed on the processing surface and the first boundary and the second boundary are respectively perpendicular to two coordinate axes in the processing coordinate system, controlling the probe to detect a first vertical distance and a second vertical distance between the first boundary and the second boundary and the initialization mark respectively; The processing coordinates corresponding to the calibration plate in the processing coordinate system are determined based on the first vertical distance and the second vertical distance.

4. The method according to claim 1, characterized in that: The step of determining the pixel coordinates corresponding to the reference calibration object in a pixel coordinate system based on the calibration image includes: Get the image position marked for the initialization mark; Taking the image position as a clipping starting point, clipping the calibration image along two coordinate axis directions of the initial pixel coordinate system of the calibration image to obtain a clipped image; the origin of the target pixel coordinate system of the clipped image is located at the image position; The pixel coordinates corresponding to the reference calibration object in the target pixel coordinate system are determined based on the cropped image.

5. The method according to claim 4, characterized in that In the calibration image, the shape of the image region to which the reference calibration object is projected is circular; and determining the pixel coordinates corresponding to the reference calibration object in the target pixel coordinate system based on the cropped image includes: Performing circle detection on the cropped image to obtain an initial center and initial radius of the circular area; Performing edge detection on the cropped image to obtain a set of edge pixels in each image area; Determine the range of the circular ring based on the range parameter and the initial center and initial radius of the circular area; Determine the target edge pixel located within the range of the circular ring from the edge pixel set; Update the initial center of the circular area according to the target edge pixel to obtain the target center; The pixel coordinates corresponding to the reference calibration object in the target pixel coordinate system are determined based on the target circle center.

6. The method according to claim 5, characterized in that In the calibration image, the shape of the image region to which the auxiliary calibration object is projected is also circular; and determining the pixel coordinates corresponding to the reference calibration object in the target pixel coordinate system based on the target circle center includes: Determining the target circle center corresponding to the reference calibration object and the target circle center corresponding to the auxiliary calibration object from among the target circle centers; Determine the pixel coordinates of the target circle center corresponding to the reference calibration object as the pixel coordinates corresponding to the reference calibration object in the pixel coordinate system; The determining the pixel distance corresponding to the distribution distance based on the calibration image includes: The pixel distance corresponding to the distribution distance is determined according to the distance between the target circle center corresponding to the auxiliary calibration object and the target circle center corresponding to the reference calibration object.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: The probe of the key machine is controlled to move to the processing coordinate corresponding to the key and move along the vertical direction of the processing surface, and the spatial position corresponding to the key is detected by the probe.

8. A visual positioning device, characterized in that: The device comprises: A detection module, used for acquiring a processing coordinate corresponding to a reference calibration object placed on a processing surface in a processing coordinate system; the origin of the processing coordinate system is located on the key machine, and the processing coordinate system is parallel to the processing surface; An acquisition module, used for acquiring the distribution distance between the auxiliary calibration objects and the reference calibration objects placed on the processing surface; An acquisition module, used for acquiring a calibration image acquired by an image acquisition unit of the key machine from an auxiliary calibration object and a reference calibration object placed on the processing surface; the optical axis of the image acquisition unit is perpendicular to the processing surface; A calibration module, used to determine the pixel distance corresponding to the distribution distance based on the calibration image; determine the coordinate conversion coefficient according to the distribution distance and the pixel distance; determine the pixel coordinates corresponding to the reference calibration object in the pixel coordinate system based on the calibration image; perform weighted processing on the pixel coordinates corresponding to the reference calibration object according to the coordinate conversion coefficient to obtain a weighted result; determine the coordinate offset based on the processing coordinates corresponding to the reference calibration object and the weighted result; The acquisition module is further used to obtain the key image captured by the image acquisition unit from the key placed on the processing surface; The positioning module is used to perform coordinate transformation on the pixel coordinates corresponding to the key in the key image according to the coordinate transformation coefficient and the coordinate offset to obtain the processing coordinates corresponding to the key.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images