Projection positioning method, system and electronic device

Abstract

The application provides a projection positioning method, system and electronic equipment, wherein the method comprises the following steps: firstly, collecting first initial coordinate information of a projected object and second initial coordinate information of a projection device, and calibrating a tracker and the projected object, and the projection device and the tracker based on the first initial coordinate information and the second initial coordinate information, to obtain first and second calibration parameters; in the working process, collecting first real-time coordinate information of the projected object and second real-time coordinate information of the projection device, combining the calibration parameters to obtain real-time projection parameters of the projection device, and then updating to-be-projected information and projecting the to-be-projected information to the projected object. The application can automatically correct the projection content according to the spatial pose change of a workpiece or a device, ensure that the to-be-projected information is always accurately matched with a target area in a dynamic environment, effectively solve the projection misalignment problem caused by object movement in the traditional technology, and greatly improve the precision and efficiency of machining guidance.

Description

Technical Field

[0001] This application relates to the field of projection positioning technology, and more specifically, to a projection positioning method, system, and electronic device. Background Technology

[0002] Projection positioning technology is rapidly gaining popularity in industrial and interactive scenarios such as aerospace, automotive manufacturing, and interactive projection. Current core applications cover high-precision operations such as composite material layup, assembly hole positioning, automotive body-in-white welding / painting, and large component assembly.

[0003] The current mainstream technical solutions are mainly divided into two categories: one is the laser projection solution, which usually uses binocular vision in conjunction with a laser galvanometer; the other is the structured light projection solution, which is usually used with a depth camera to achieve image or structured light projection onto the target surface.

[0004] However, existing projection positioning technologies suffer from a sharp drop in edge accuracy when projecting over a large area due to lens distortion and increased angle. Secondly, multiple calibrations at different stations are required for large workpieces, resulting in large cumulative errors and low efficiency. Thirdly, the calibration process is complex and highly dependent on cooperative targets, making deployment cumbersome and time-consuming. Furthermore, traditional visual tracking has high latency in dynamic scenes and is susceptible to interference from complex textures and lighting. Finally, relying on a single large-area projection device is costly and has a high deployment threshold. Summary of the Invention

[0005] The purpose of this application is to address the shortcomings of the prior art by providing a projection positioning method, system, and electronic device to solve the problems of unstable projection accuracy and low efficiency in the prior art.

[0006] To achieve the above objectives, the technical solution adopted in this application is as follows: In a first aspect, this application provides a projection positioning method applied to a processing device in a projection positioning system. The projection positioning system includes: the processing device, a tracker, a movable ball cage frame, and a projection device disposed on the ball cage frame. The method includes: When the object being projected is stationary, the first initial coordinate information corresponding to the object being projected and the second initial coordinate information of the frame marker points on the ball cage frame are collected. The tracker and the object being projected are calibrated to obtain the first calibration parameters. The projection device and the tracker are calibrated to obtain the second calibration parameters. The first calibration parameters indicate the coordinate system transformation relationship between the tracker and the object being projected, and the second calibration parameters indicate the coordinate system transformation relationship between the tracker and the projection device. During the projection process using the projection device, the first real-time coordinate information corresponding to the projected object is collected in real time, and the second real-time coordinate information of the frame marker point is collected in real time through the tracker. Based on the first initial coordinate information and the first real-time coordinate information, the first calibration parameter is updated to obtain the third calibration parameter, and based on the second initial coordinate information and the second real-time coordinate information, the second calibration parameter is updated to obtain the fourth calibration parameter; Based on the third calibration parameter and the fourth calibration parameter, the information to be projected in the projection device is projected onto the object being projected.

[0007] Optionally, updating the first calibration parameters based on the first initial coordinate information and the first real-time coordinate information to obtain the third calibration parameters includes: Spatial matching and pose calculation are performed on the first initial coordinate information and the first real-time coordinate information to obtain the dynamic transformation matrix; The third calibration parameter is obtained by performing matrix operations on the dynamic transformation matrix and the first calibration parameter.

[0008] Optionally, projecting the information to be projected from the projection device onto the object to be projected according to the third calibration parameter and the fourth calibration parameter includes: The projection information is generated based on the third calibration parameter, the fourth calibration parameter, and the initial projection information; The information to be projected is sent to the projection device so that the information to be projected is projected onto the object being projected.

[0009] Optionally, generating the projection information based on the third calibration parameter, the fourth calibration parameter, and the initial projection information includes: Based on the third calibration parameter and the fourth calibration parameter, determine the coordinate transformation matrix between the projection device and the projected object; The initial projection information is transformed based on the coordinate transformation matrix and the intrinsic parameters of the projection device to obtain the information to be projected.

[0010] Optionally, the projection device includes: a surface projector and a line projector; The step of projecting the information to be projected from the projection device onto the object being projected includes: Control the area projector to project an area array image onto the object being projected, and / or control the line projector to project a laser line onto the object being projected; The area array image includes a color difference model image for characterizing the processing area or surface state, and a region frame image for characterizing the boundary of the processing area. The laser line includes line segments, curved segments, and processing trajectory lines for indicating the processing path, as well as contour lines for indicating the processing contour.

[0011] Optionally, the projection positioning system further includes: a scanner; The acquisition of the first initial coordinate information corresponding to the projected object includes: The scanner scans the feature data of the projected object, and the first initial coordinate information is determined based on the feature data; The calibration of the tracker and the projected object to obtain the first calibration parameters includes: The first calibration parameter is obtained by calibrating based on the feature data and the model data of the projected object.

[0012] Optionally, the object being projected is provided with a plurality of preset marker points; The acquisition of the first initial coordinate information corresponding to the projected object includes: The tracker collects the coordinates of the multiple preset marker points on the projected object to obtain the first initial coordinate information. The calibration of the tracker and the projected object to obtain the first calibration parameters includes: Spatial matching and pose calculation are performed on the coordinates of each preset marker point to obtain the first calibration parameters.

[0013] Optionally, calibrating the projection device and the tracker to obtain second calibration parameters includes: Based on the second initial coordinate information and the intrinsic parameters of the projection device, the coordinate system transformation relationship of the projection device in the coordinate system of the ball cage frame is calculated and used as the second calibration parameter.

[0014] In a second aspect, this application provides a projection positioning system, which includes: the processing device, a tracker, a movable ball cage frame, and a projection device disposed on the ball cage frame, wherein the processing device is used to perform the steps of the projection positioning method as described in the first aspect.

[0015] Thirdly, embodiments of this application also provide a processing device, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the processing device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of a projection positioning method as described in any one of the first aspects.

[0016] The beneficial effects of this application are as follows: By real-time acquisition of the coordinate changes of the projected object and updating the first calibration parameters, displacement errors caused by force or vibration during workpiece processing are eliminated; by real-time acquisition of the coordinate changes of the marker points of the ball cage frame and updating the second calibration parameters, the projection system is able to move flexibly with the frame, eliminating the cumulative errors caused by frequent station changes in large-space operations. Projection based on the real-time updated third and fourth calibration parameters enables the projection information to be accurately and without delay attached to the surface of the projected object even under complex working conditions where both the workpiece and equipment are moving, significantly improving the flexibility and positioning accuracy of the assembly and processing of large and complex components.

[0017] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This paper shows a schematic diagram of the architecture of a projection positioning system provided in an embodiment of the present application; Figure 2 A flowchart of a projection positioning method provided in an embodiment of this application is shown; Figure 3 This document illustrates a flowchart of an embodiment of obtaining a third calibration parameter. Figure 4 A flowchart illustrating a projection method provided in an embodiment of this application is shown; Figure 5 This document illustrates a flowchart of a method for generating projection information according to an embodiment of this application. Figure 6 A schematic diagram of a projection pattern provided in an embodiment of this application is shown; Figure 7 A schematic diagram illustrating a projection effect provided in an embodiment of this application is shown; Figure 8 A schematic diagram of the structure of a processing device provided in an embodiment of this application is shown. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0021] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0022] In existing projection positioning technologies, the projection device and the projected object are usually relatively stationary, making it difficult to cope with situations where the object moves or deforms in actual working conditions. Once the workpiece moves or deforms in actual working conditions, the system cannot automatically correct the deviation, which can easily lead to projection errors.

[0023] Secondly, when using handheld projection devices, the device's pose changes in real time, and existing technologies lack a dynamic real-time calculation mechanism for the relative pose of the projection device and the object, which cannot guarantee projection accuracy.

[0024] In addition, traditional single projection methods are difficult to meet the combined display requirements of large-area array information and high-precision line contours, and lack a unified global coordinate reference, making it difficult to maintain high-precision projection positioning effect in large-space operations.

[0025] Based on this, this application proposes a projection positioning method, which is applied to... Figure 1 In the projection positioning system shown, such as Figure 1 As shown, the system includes a processing device, a tracker, and a movable ball cage frame. The projection device is housed within the ball cage frame. The processing device establishes network communication with both the tracker and the projection device for data transmission. The processing device can be an electronic device with hardware terminal capabilities for data computation and logic control. The tracker can be... Figure 1 The spatial position tracking device shown can capture the three-dimensional coordinates of specific landmarks in space in real time. The surface or interior of the ball cage frame is equipped with frame landmarks, and a projection device is installed on the frame. The projection device can be a surface projector or a line projector.

[0026] Taking the processing of large and complex workpieces as an example, when it is necessary to project the workpiece for machining position positioning, it can be done by... Figure 1 The system shown achieves projective positioning of the workpiece. (Refer to...) Figure 1 The scenario includes a workpiece as the projection object. A tracker can be positioned anywhere in the projection space. An operator can move a ball cage frame within the tracker's tracking range, projecting the information to be projected from the processing equipment onto the object through a projection device within the ball cage frame. In normal mode, the workpiece remains stationary while the ball cage frame moves. In this case, no markers need to be placed on the workpiece. The tracker calculates the new pose of the projection device after the frame moves by tracking the real-time coordinate flow of the projection device within the ball cage frame. If the workpiece may move, markers can be pre-set on the workpiece. The tracker can then dynamically track the workpiece by tracking these markers. The processing equipment can calculate the pose change of the workpiece relative to the tracker and the new pose of the projection device after the frame moves, based on the real-time coordinate flow fed back by the tracker. Finally, the processing equipment fuses the projection information with these two real-time updated parameters, projecting a high-precision area array image or laser line onto the dynamically changing workpiece surface through the projection device, achieving an adaptive dynamic tracking projection effect.

[0027] Figure 2 A flowchart of a projection positioning method provided in an embodiment of this application is shown, as follows: Figure 2 As shown, the execution entity of this method can be Figure 1 The processing equipment of the projection positioning system shown, such as Figure 2 As shown, the method includes: S201. When the object being projected is in a static state, collect the first initial coordinate information corresponding to the object being projected and the second initial coordinate information of the frame markers on the ball cage frame, calibrate the tracker and the object being projected to obtain the first calibration parameters, and calibrate the projection device and the tracker to obtain the second calibration parameters.

[0028] The first calibration parameter indicates the coordinate system transformation relationship between the tracker and the projected object, while the second calibration parameter indicates the coordinate system transformation relationship between the tracker and the projection device.

[0029] The object being projected refers to a physical entity that needs to be guided by projection, such as aircraft skin or automobile body-in-white.

[0030] Initial coordinate information refers to the three-dimensional spatial coordinates of the marker points on the workpiece or frame in the tracker coordinate system, acquired by the tracker at the initial calibration time of the system. The first calibration parameter and the second calibration parameter can be coordinate transformation matrices.

[0031] As an optional implementation, multiple workpiece markers can be pre-set on the projected object. When the projected object is stationary, the tracker can simultaneously acquire the workpiece markers on the projected object and the frame markers on the ball cage frame. For the first calibration parameter, the workpiece feature data can be obtained by scanning and aligning it with the digital model, or the workpiece markers can be directly calculated using the tracker to obtain the first calibration parameter. For the second calibration parameter, the processing device can use the known initial coordinates of the frame markers, the intrinsic parameters of the projection device, and the frame marker coordinates fed back by the tracker to calculate the precise pose of the projection device in the ball cage frame coordinate system, thus obtaining the second calibration parameter.

[0032] S202. During the projection process using a projection device, the first real-time coordinate information corresponding to the projected object is collected in real time, and the second real-time coordinate information of the frame marker points is collected in real time through a tracker.

[0033] The first real-time coordinate information refers to the three-dimensional coordinate data of the marker points of the projected object, continuously acquired by the tracker at a high frame rate during the dynamic operation. The second real-time coordinate information refers to the three-dimensional coordinate data of the frame marker points on the ball cage frame, continuously acquired by the tracker at a high frame rate during the dynamic operation.

[0034] In actual operation, the projected object may undergo slight movement due to processing forces, or the ball cage frame may be moved to a new workstation. Therefore, a tracker can continuously track the marker points of the projected object and the frame marker points of the ball cage frame, and synchronously transmit the acquired real-time coordinate stream to the processing equipment.

[0035] S203. Based on the first initial coordinate information and the first real-time coordinate information, update the first calibration parameter to obtain the third calibration parameter, and based on the second initial coordinate information and the second real-time coordinate information, update the second calibration parameter to obtain the fourth calibration parameter.

[0036] The third calibration parameter indicates the latest coordinate system transformation between the updated tracker and the projected object. The fourth calibration parameter indicates the latest coordinate system transformation between the updated tracker and the projection device.

[0037] Optionally, the processing device can perform spatial matching and pose calculation between the first real-time coordinates and the first initial coordinates to obtain the dynamic transformation matrix of the workpiece, and perform matrix operation with the first calibration parameters to obtain the third calibration parameters; similarly, the dynamic transformation matrix of the ball cage frame can be calculated using the second real-time coordinates and the second initial coordinates, and the fourth calibration parameters can be updated to obtain the fourth calibration parameters.

[0038] S204. Based on the third calibration parameter and the fourth calibration parameter, project the information to be projected from the projection device onto the object being projected.

[0039] The information to be projected refers to the content that needs to be projected onto the surface of the workpiece, such as machining outlines, color difference models, assembly holes, etc.

[0040] As one possible implementation, a virtual scene containing the object to be projected can be pre-constructed in the processing device. The virtual scene includes a virtual model of the object to be projected and a virtual camera. The virtual model contains model data that needs to be projected. The virtual model in the processing device can be converted into initial projection information from the perspective of the virtual camera. The initial projection information can be corrected by combining the third calibration parameter and the fourth calibration parameter to obtain the information to be projected. Then, the information to be projected is sent to the projection device, which projects the information to be projected onto the object to be projected.

[0041] In this embodiment, by real-time acquisition of the coordinate changes of the projected object and updating the first calibration parameter, displacement errors caused by force or vibration during workpiece processing are eliminated. By real-time acquisition of the coordinate changes of the marker points of the ball cage frame and updating the second calibration parameter, the projection system is able to move flexibly with the frame, eliminating the cumulative errors caused by frequent station changes in large-space operations. Projection based on the real-time updated third and fourth calibration parameters enables the projection information to be accurately and without delay adhered to the surface of the projected object even under complex working conditions where both the workpiece and equipment are moving, significantly improving the flexibility and positioning accuracy of the assembly and processing of large and complex components.

[0042] Figure 3 This application provides a flowchart for obtaining a third calibration parameter according to an embodiment of the present application. (Refer to...) Figure 3 The process of updating the first calibration parameters based on the first initial coordinate information and the first real-time coordinate information to obtain the third calibration parameters includes: S301. Perform spatial matching and pose calculation on the first initial coordinate information and the first real-time coordinate information to obtain the dynamic transformation matrix.

[0043] The dynamic transformation matrix can quantitatively describe the translation and rotation of the projected object from its initial static state to its current real-time state.

[0044] The first initial coordinate information and the first real-time coordinate information include the three-dimensional position coordinates of points with the same location in both the static state and the projection process. By using the iterative nearest point algorithm or the singular value decomposition algorithm to calculate the least squares solution between the two sets of coordinate data, the translation and rotation of the workpiece in space can be calculated, thus obtaining the dynamic transformation matrix.

[0045] S302. Perform matrix operations on the dynamic transformation matrix and the first calibration parameter to obtain the third calibration parameter.

[0046] The first calibration parameter represents the coordinate transformation relationship from the tracker to the projected object at the initial moment, while the dynamic transformation matrix represents the pose change of the projected object itself. The processing device can multiply the dynamic transformation matrix by the first calibration parameter, thereby superimposing the latest displacement and rotation of the workpiece onto the original coordinate transformation path. The third calibration parameter is the latest transformation relationship from the tracker coordinate system to the coordinate system of the projected object at the current moment, thus realizing real-time mathematical compensation for workpiece movement.

[0047] In this embodiment, the dynamic transformation matrix is ​​obtained by spatial matching and pose calculation, which can quantify the minute movement or rotation of the projected object in any direction; then, this change is directly integrated into the calibration parameters through matrix operations, avoiding the tedious process of repeating full calibration.

[0048] Figure 4 A flowchart illustrating a projection process provided in an embodiment of this application is shown, with reference to... Figure 4 The process of projecting the information to be projected from the projection device onto the object being projected, based on the third and fourth calibration parameters, includes: S401. Generate projection information based on the third calibration parameter, the fourth calibration parameter, and the initial projection information.

[0049] Initial projection information refers to the original data that needs to be displayed on the object being projected under standard or ideal conditions, such as machining outlines, hole diagrams, or assembly guides in a CAD (Computer-Aided Design) model.

[0050] The processing device can pre-build a virtual scene containing a virtual model of the object to be projected. The initial projection information can be the data content set in the virtual model that needs to be projected onto the object. The initial projection information in the virtual scene can be mapped to the real physical space through the third calibration parameter and the fourth calibration parameter to obtain the information to be projected.

[0051] Optionally, the initial projection information can be geometrically distorted and repositioned according to the third and fourth calibration parameters to generate projection information that can be accurately attached to the surface of the current workpiece. For example, if the workpiece moves to the right, the generated projection information will automatically shift to the left to compensate.

[0052] S402. Send the information to be projected to the projection device so that the information to be projected is projected onto the object being projected.

[0053] Optionally, the processing device can transmit the generated projection information to the projection device mounted on the ball cage frame in real time. After receiving the signal, the projection device projects the image through its internal optical engine. Since the projection information already includes the compensation amounts indicated by the third and fourth calibration parameters, even if the projection device moves with the frame or the workpiece vibrates, the projected light can accurately fall on the predetermined position of the projected object.

[0054] In this embodiment, by fusing the third and fourth calibration parameters to generate the information to be projected, it is ensured that the projected content mathematically and logically completes the dual compensation for workpiece displacement and equipment displacement. Secondly, by sending the corrected information to the projection device for output, the precise execution of the physical optical path is ensured, effectively eliminating projection ghosting, misalignment, or blurring caused by equipment or workpiece movement.

[0055] Figure 5 This application provides a flowchart for generating projection information according to an embodiment of the present application. (Refer to...) Figure 5 The process of generating the projection information based on the third calibration parameter, the fourth calibration parameter, and the initial projection information includes: S501. Determine the coordinate transformation matrix between the projection device and the projected object based on the third and fourth calibration parameters.

[0056] The third calibration parameter characterizes the transformation relationship from the tracker to the projected object, and the fourth calibration parameter characterizes the transformation relationship from the tracker to the projection device. The processing device can calculate the direct relative pose relationship between the projection device and the projected object through matrix inversion and multiplication operations, and obtain the coordinate transformation matrix.

[0057] S502. Based on the coordinate transformation matrix and the intrinsic parameters of the projection device, the initial projection information is transformed to obtain the information to be projected.

[0058] The intrinsic parameters of the projection device can be determined simultaneously when determining the first calibration parameters. The intrinsic parameters of the projection device include the inherent optical and geometric properties of the projection device itself, mainly including focal length, principal point coordinates, pixel size, and lens distortion coefficient.

[0059] Optionally, the initial projection information can be placed in virtual space first. Then, using the coordinate transformation matrix calculated in the previous step, the virtual model is transformed to a viewpoint with the projection device as the origin; next, combined with the intrinsic parameters of the projection device, perspective projection and distortion correction are performed to render the 3D model into a 2D image. The final generated 2D image, after precise geometric transformation and distortion compensation, is the information to be projected.

[0060] In this embodiment, by combining the coordinate transformation matrix with intrinsic parameters characterizing the device's own optical properties, the current real viewing angle of the projection device can be simulated in virtual space. This allows for automatic perspective transformation and distortion compensation of the initial projection information, ensuring that even when the projection device and the projected object move relative to each other, tilt, or change in distance, the generated projection information still conforms to the physical surface of the projected object. This eliminates image stretching, misalignment, or blurring caused by changes in viewing angle, achieving adaptive high-precision projection.

[0061] Optionally, the projection device includes: a surface projector and a line projector. The process of projecting the information to be projected from the projection device onto the object being projected includes: A control surface projector projects a surface image onto the object being projected, and / or a control line projector projects a laser line onto the object being projected.

[0062] The area array image includes a color difference model image for characterizing the state of the processing area or surface, and any information that can be expressed by an image, such as a region bounding image for characterizing the boundary of the processing area. The laser line includes line segments, curved segments, and processing trajectory lines for indicating the processing path, and may also include contour lines for indicating the processing contour. It should be understood that this application only provides some possible implementations of the area array image and the laser line. The area array image can also be other image information, and the laser line can also be other line information. The specific form is not limited in this application.

[0063] A surface projector is a projection device capable of projecting a two-dimensional area array image, such as a DLP projector (Digital Light Processing Projector) or an LCD projector (Liquid Crystal Display Projector), used to project large-area images or patterns onto the surface of an object. A line projector is a device capable of projecting one or more high-brightness laser lines, used to project extremely fine laser trails onto the surface of an object.

[0064] Color difference model images are 3D model projections rendered in different colors, used to visually display the flatness, thickness deviation, or machining allowance of a workpiece surface. Area outline images refer to contour lines that delineate the boundaries of machining or assembly areas.

[0065] A planar image is a two-dimensional image composed of a pixel matrix, covering a certain area. Optionally, based on actual operational needs, control commands can be used to drive the planar projector or the line projector separately, or simultaneously, to project the information to be projected.

[0066] It should be noted that the projection equipment includes both surface projectors and line projectors. When calibrating parameters, the calibration of both surface projectors and line projectors can be completed simultaneously through the above-mentioned step S201. When updating calibration parameters, the calibration parameters of both surface projectors and line projectors can also be dynamically updated simultaneously.

[0067] In this embodiment, a surface projector is used to project a color difference model and region outlines, providing operators with intuitive global visual feedback and processing guidance; a line projector is used to project processing trajectory lines and contour lines, providing high-precision local physical positioning references. This composite projection method balances the needs of large-scale information display and high-precision trajectory guidance, significantly improving the operational efficiency and positioning accuracy of complex workpiece processing and assembly.

[0068] Optionally, the projection positioning system may also include a scanner.

[0069] The aforementioned acquisition of the first initial coordinate information corresponding to the projected object includes: The feature data of the object being projected is scanned by a scanner, and the initial coordinate information is determined based on the feature data.

[0070] Feature data refers to the three-dimensional point cloud data or specific geometric topological features of the surface of the projected object obtained by a scanner.

[0071] Optionally, the operator can use a scanner to perform a full-range scan of the projected object in a stationary state to extract the point cloud data of the projected object, which contains multiple points and the three-dimensional position coordinates of the points.

[0072] The tracker and the projected object are calibrated to obtain the first calibration parameters, including: The first calibration parameters are obtained by calibrating based on the feature data and the model data of the projected object.

[0073] Optionally, the feature data and model data can be registered. By finding the optimal overlap position between the two sets of data, the pose difference between the real physical workpiece and the virtual digital model can be calculated. Combined with the spatial coordinates of the tracker during the scanning process, the first calibration parameters between the tracker and the projected object can be finally calculated.

[0074] In the second implementation, multiple preset marker points can be set on the projected object, and the first calibration parameter can be obtained through the preset marker points.

[0075] The process of acquiring the first initial coordinate information corresponding to the projected object includes: The first initial coordinate information is obtained by collecting the coordinates of multiple preset marker points on the projected object using a tracker.

[0076] Pre-set markers refer to specific optical reflective targets that are pre-pasted or attached to the surface of the object to be projected. Before operation, operators can distribute and paste multiple pre-set markers on the surface of the object to be projected, according to the workpiece's geometry and shape. These markers serve as spatial coordinate anchor points for the workpiece, providing stable visual recognition targets for subsequent high-frequency dynamic tracking.

[0077] When the projected object is stationary, the tracker can capture these preset marker points through its optical lens and calculate the three-dimensional coordinates of each marker point through stereo vision or coding recognition algorithms to obtain the first initial coordinate information.

[0078] The process of calibrating the tracker and the projected object to obtain the first calibration parameters includes: Spatial matching and pose calculation are performed on the coordinates of each preset marker point to obtain the first calibration parameters.

[0079] After acquiring the coordinates of each preset marker point fed back by the tracker, the processing equipment uses mathematical algorithms such as singular value decomposition or iterative nearest point to spatially register the measured 3D point cloud with the CAD model of the workpiece. By calculating the least squares solution between the two sets of points, the precise rotation and translation matrix of the projected object relative to the tracker coordinate system is calculated, and the matrix is ​​used as the first calibration parameter.

[0080] The process of calibrating the projection device and tracker to obtain the second calibration parameters includes: Based on the second initial coordinate information and the intrinsic parameters of the projection device, the coordinate system transformation relationship of the projection device in the spherical cage frame coordinate system is calculated and used as the second calibration parameter.

[0081] Because there is a rigid physical connection between the frame markers on the ball cage frame and the projection device, their relative positions are fixed. The processing device can use the second initial coordinate information fed back in real time by the tracker, combined with the pre-measured intrinsic parameters of the projection device, and through spatial resection and other pose calculation algorithms, to deduce the precise pose of the optical center and optical axis of the projection device relative to the ball cage frame coordinate system. The final transformation matrix is ​​used as the second calibration parameter.

[0082] The following describes the process of obtaining the first and second calibration parameters. A projection device projects a pattern onto a target, while a camera takes a picture of the target to obtain an image. The mathematical models for both are consistent; projection is essentially the reverse process of taking a picture. Therefore, the relationship calibration of the projection device, given the projected pattern, requires obtaining the three-dimensional coordinates of the projected target pattern. The projection model is a collinearity equation.

[0083] The projection model can be expressed by the following collinearity equation, which is given by equation (1): (1) Where (x, y) are the image plane coordinates of the image point; (x0, y0, f) are the interior orientation elements of the image; (XS, YS, ZS) are the object space coordinates of the camera station; (X, Y, Z) are the object space coordinates of the object point; ai, bi, ci (i = 1, 2, 3) are the nine direction cosines composed of the three exterior orientation elements phi, omega, and kappa of the image. (XS, YS, ZS, phi, omega, kappa) are the extrinsic parameters of the image, and (x0, y0, f) are the interior orientation elements of the image; these are the intrinsic and extrinsic parameters of the image.

[0084] In addition, during the shooting process, due to lens distortion, the influence of lens distortion also needs to be considered. The distortion model is represented by the following equation (2) and equation (3): (2) (3) Where (k1,k2,k3,p1,p2) are distortion parameters. Distortion correction can be achieved by substituting the distortion model into the collinearity equation above. After distortion correction, the pixel coordinates of the image are theoretically correct.

[0085] To solve the parameters in the above collinear equation, multiple sets of (X,Y,Z,x,y) are needed. For the problem of solving the projector, (x,y) can be known from the projected pattern, while (X,Y,Z) are to be determined.

[0086] Alternatively, it can be done through Figure 6 The projected image shown is calibrated using a pattern composed of multiple sets of horizontal and vertical blue lines (the color is determined by the tracker's camera filter; the projection tracker receives the most sensitive color, assumed to be blue here). Several circular markers are also designed to uniquely define the coordinate system. Through this pattern, the intersection points of the horizontal and vertical lines, the center points of the circles, and the corresponding unique IDs can be obtained, serving as the image coordinate set. .

[0087] The pattern is projected onto a flat surface (such as a wall) using a projector to obtain... Figure 7The projection effect is shown. Then, the left and right cameras of the tracker are used to capture the projection pattern to obtain the left and right images. Then, the straight line extraction method and the ellipse extraction method are used to extract the straight lines and ellipses on the image. The image coordinates of the intersection of the straight lines and the image coordinates of the center of the ellipse are extracted. Then, according to the coordinate positions of the five ellipse centers, the image coordinate values ​​are compared and sorted to obtain the unique ID value on the corresponding projection pattern. That is, the one-to-one correspondence between the five ellipse centers and the five ellipse centers of the projection pattern is realized. Finally, the homography transformation (the transformation between planes) can be used to solve the transformation relationship between the projection pattern and the captured image, and the following formula (4) is obtained.

[0088] (4) The following equation (5) is obtained by solving: (5) Taking a device with 8 degrees of freedom as an example, H can be obtained by solving the equation AH=0, which is a combination of 4 sets of non-collinear matching points. H can be expressed as the following equation (6).

[0089] (6) After obtaining H, other feature points on the projected pattern can be projected onto the acquired image through homography transformation. By finding the minimum distance value, the corresponding points on the acquired image are obtained. Finally, the above operation is performed on both the left and right images obtained by the tracker to obtain the corresponding points and unique ID values ​​of the left and right images. Using the existing intrinsic and extrinsic parameters of the tracker, the three-dimensional coordinate values ​​of these feature points are calculated and denoted as H. .

[0090] After obtaining multiple groups Then, the values ​​can be substituted into the collinearity equation and the unknowns can be solved to obtain the pose of the projection device in the current state and the intrinsic parameters of the projection device.

[0091] Simultaneously, the tracker can acquire the coordinate set of the frame markers on the ball cage frame in the tracker coordinate system. Based on the existing coordinate set of the markers on the ball cage frame, the two coordinate systems are transformed to obtain the pose parameters of the ball cage frame in the tracker coordinate system, which are the second calibration parameters.

[0092] It should be noted that, in order to achieve more accurate calibration, the above calibration process can be repeated, and projection can be performed at different positions and attitudes of the ball cage frame, tracker, and projection device to obtain multiple sets of data. The first calibration parameter, the second calibration parameter, and the intrinsic parameters of the projection device obtained each time are used as global unknowns. The least squares method is used to perform optimization to obtain the globally optimal first calibration parameter, the second calibration parameter, and the intrinsic parameters of the projection device.

[0093] The intrinsic parameters of the projection device are fixed after calibration. During the use of the device, the first and second calibration parameters can be updated based on the real-time coordinate information of the projected object and the tracker, thereby realizing dynamic tracking projection of the projection device.

[0094] Based on the same inventive concept, this application also provides a projection positioning system corresponding to the projection positioning method. The system architecture can be referred to. Figure 1 As shown, since the principle of the system in this application embodiment is similar to the projection positioning method described above in this application embodiment, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0095] Figure 8 The diagram illustrates the structure of a processing device provided in an embodiment of this application, including a processor 801, a storage medium 802, and a bus 803. The storage medium 802 stores machine-readable instructions executable by the processor 801. When the electronic device runs a projection positioning method as described in the embodiment, the processor 801 communicates with the storage medium 802 via the bus 803. The processor 801 executes the machine-readable instructions, and the preamble of the method item of the processor 801 executes the steps in the projection positioning method described above.

[0096] This application also provides a computer-readable storage medium storing a computer program that is executed by a processor, which performs the steps in the projection positioning method described above.

[0097] In this embodiment, the computer program, when run by the processor, can also execute other machine-readable instructions to perform other methods as described in the embodiments. For details on the specific execution steps and principles, please refer to the description of the embodiments, which will not be repeated here.

[0098] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0099] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0100] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0101] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0102] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0103] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A projection positioning method, characterized in that, A processing device for use in a projection positioning system, the projection positioning system comprising: the processing device, a tracker, a movable ball cage frame, and a projection device disposed on the ball cage frame, the method comprising: When the object being projected is stationary, the first initial coordinate information corresponding to the object being projected and the second initial coordinate information of the frame marker points on the ball cage frame are collected. The tracker and the object being projected are calibrated to obtain the first calibration parameters. The projection device and the tracker are calibrated to obtain the second calibration parameters. The first calibration parameters indicate the coordinate system transformation relationship between the tracker and the object being projected, and the second calibration parameters indicate the coordinate system transformation relationship between the tracker and the projection device. During the projection process using the projection device, the first real-time coordinate information corresponding to the projected object is collected in real time, and the second real-time coordinate information of the frame marker point is collected in real time through the tracker. Based on the first initial coordinate information and the first real-time coordinate information, the first calibration parameter is updated to obtain the third calibration parameter, and based on the second initial coordinate information and the second real-time coordinate information, the second calibration parameter is updated to obtain the fourth calibration parameter; Based on the third calibration parameter and the fourth calibration parameter, the information to be projected in the projection device is projected onto the object being projected.

2. The method according to claim 1, characterized in that, The step of updating the first calibration parameters based on the first initial coordinate information and the first real-time coordinate information to obtain the third calibration parameters includes: Spatial matching and pose calculation are performed on the first initial coordinate information and the first real-time coordinate information to obtain the dynamic transformation matrix; The third calibration parameter is obtained by performing matrix operations on the dynamic transformation matrix and the first calibration parameter.

3. The method according to claim 1, characterized in that, The step of projecting the information to be projected from the projection device onto the object to be projected based on the third calibration parameter and the fourth calibration parameter includes: The projection information is generated based on the third calibration parameter, the fourth calibration parameter, and the initial projection information; The information to be projected is sent to the projection device so that the information to be projected is projected onto the object being projected.

4. The method according to claim 3, characterized in that, The step of generating the projection information based on the third calibration parameter, the fourth calibration parameter, and the initial projection information includes: Based on the third calibration parameter and the fourth calibration parameter, determine the coordinate transformation matrix between the projection device and the projected object; The initial projection information is transformed based on the coordinate transformation matrix and the intrinsic parameters of the projection device to obtain the information to be projected.

5. The method according to claim 1, characterized in that, The projection equipment includes: a surface projector and a line projector; The step of projecting the information to be projected from the projection device onto the object being projected includes: Control the area projector to project an area array image onto the object being projected, and / or control the line projector to project a laser line onto the object being projected; The area array image includes a color difference model image for characterizing the processing area or surface state, and a region frame image for characterizing the boundary of the processing area. The laser line includes line segments, curved segments, and processing trajectory lines for indicating the processing path, as well as contour lines for indicating the processing contour.

6. The method according to claim 1, characterized in that, The projection positioning system also includes: a scanner; The acquisition of the first initial coordinate information corresponding to the projected object includes: The scanner scans the feature data of the projected object, and the first initial coordinate information is determined based on the feature data; The calibration of the tracker and the projected object to obtain the first calibration parameters includes: The first calibration parameter is obtained by calibrating based on the feature data and the model data of the projected object.

7. The method according to claim 1, characterized in that, The object being projected has multiple preset marker points; The acquisition of the first initial coordinate information corresponding to the projected object includes: The tracker collects the coordinates of the multiple preset marker points on the projected object to obtain the first initial coordinate information. The calibration of the tracker and the projected object to obtain the first calibration parameters includes: Spatial matching and pose calculation are performed on the coordinates of each preset marker point to obtain the first calibration parameters.

8. The method according to claim 1, characterized in that, The calibration of the projection device and the tracker to obtain the second calibration parameters includes: Based on the second initial coordinate information and the intrinsic parameters of the projection device, the coordinate system transformation relationship between the projection device and the ball cage frame is calculated and used as the second calibration parameter.

9. A projection positioning system, characterized in that, The projection positioning system includes: the processing device, the tracker, the movable ball cage frame, and the projection device disposed on the ball cage frame, wherein the processing device is used to perform the steps of the projection positioning method as described in any one of claims 1-8.

10. A processing apparatus, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the processing device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of a projection positioning method as described in any one of claims 1 to 8.

Images