A method for improving image splicing accuracy and positioning accuracy of a portable printing and copying machine

Abstract

The application provides a method for improving image splicing precision and positioning precision of a portable printing and copying machine. The method effectively overcomes image splicing and positioning precision errors caused by local measurement errors of a motion tracking sensor and sparseness of image features used for registration, thereby improving image scanning splicing quality and printing image splicing quality of the portable printing and copying machine. By using the method in the application, the image splicing or positioning precision is less than 0.2 mm, and there is no cumulative error, even if there is an error within 1% in local motion measurement or there is no effective visual feature in a 10cm*10cm range of a printing image.

Description

TECHNICAL FIELD

[0001] The application provides a method for improving image splicing accuracy and positioning accuracy of a portable printing and copying machine, and particularly relates to a method for improving image splicing and positioning accuracy of a portable printing and copying machine by using a linear image sensor, and belongs to the technical field of printing and copying machines. BACKGROUND

[0002] The patent ZL 202110482240.X proposes a design method of a portable printing and copying machine based on two-stage visual positioning from coarse to fine, and realizes automatic splicing of multi-row scanning images and multi-row printing images through visual positioning, thereby supporting scanning and printing functions of an arbitrary format image.

[0003] However, the existing method has two problems affecting the splicing effect of image scanning and printing: limited local motion measurement accuracy in the image scanning and printing process; and sparse image features used for registration in some scenes. Specifically, due to the limitation of local motion measurement accuracy, in the scene lacking of registration image features, the cumulative error of motion measurement is generated, which causes non-rigid deformation of a row of scanning or printing images, resulting in failure of image registration, thereby affecting the scanning image splicing effect or the splicing and positioning accuracy of printing.

[0004] Therefore, the application proposes a method for improving image splicing accuracy and positioning accuracy of a portable printing and copying machine by using a linear image sensor, which overcomes the problems of image splicing and positioning accuracy caused by local motion measurement error and sparse image features in such printing and copying machines. SUMMARY

[0005] The application proposes a method for improving image splicing accuracy and positioning accuracy of a portable printing and copying machine, which can improve the scanning image splicing quality and the printing image splicing quality. The main steps include:

[0006] Step 1: setting a linear image sensor and a motion tracking sensor in the hardware of the portable printing and copying machine.

[0007] Step 2: improving the image splicing accuracy of the portable printing and copying machine by calibrating the imaging model parameters of the linear image sensor, removing distortion from the collected images, and estimating the image splicing based on 2D rigid body transformation.

[0008] Step 3: determining the single-row printing content according to the positioning result of the linear image sensor to realize splicing printing.

[0009] Step 4: improving the accuracy of single-row printing positioning initialization by using a pre-set mark pattern on the surface of the printing medium.

[0010] Step 5: improving the accuracy of positioning tracking during printing by using a pre-set mark pattern on the surface of the printing medium.

[0011] wherein the setting of the linear image sensor and the motion tracking sensor in the portable printing copier hardware in step 1 specifically includes the following steps:

[0012] Step 1.1: Setting one or more linear image sensors in the portable printing copier to acquire original size images of the scanning or printing medium surface. Linear image sensor refers to contact image sensor (CIS) or linear CCD image sensor.

[0013] Step 1.2: Setting one or more motion tracking sensors in the portable printing copier to measure the local motion of the printing copier. Motion tracking sensor refers to photoelectric or laser mouse sensor, grating or capacitive or magnetic grating sensor, photoelectric encoder or IMU.

[0014] wherein the method for improving the image stitching accuracy of the portable printing copier in step 2 not only improves the quality of multi-line scanning image stitching, but also improves the quality of multi-line printing content stitching. Specifically includes the following steps:

[0015] Step 2.1: Calibrating the imaging model parameters of the linear image sensor. Calibration accuracy has a direct impact on improving image stitching accuracy, and must be recalibrated after the image sensor parameters change.

[0016] Step 2.2: Collecting linear image sensor data and motion tracking sensor data. Move the portable printing copier along a straight line, scan the image on the medium, and collect multiple frames of data from the linear image sensor at a certain resolution and equal interval according to the local motion measurement results of the motion tracking sensor.

[0017] Step 2.3: Using the imaging model of the linear image sensor and the data collected in step 2.2 to obtain a single row or local original size scanning image without distortion.

[0018] Step 2.4: Stitching the multi-row original size scanning images obtained by multiple executions of steps 2.2-2.3 based on 2D rigid body transformation to obtain a larger range of original size scanning images, realizing stitching scanning; or stitching the local scanning image and the printing medium surface image obtained by one execution of steps 2.2-2.3 based on 2D rigid body transformation to obtain the positioning result of the linear image sensor, realizing stitching printing.

[0019] wherein the calibration of the imaging model parameters of the linear image sensor in step 2.1 is described below. The imaging model parameter calibration steps are illustrated by taking contact image sensor (CIS) as an example. The related method is also applicable to linear CCD image sensor.

[0020] Step 2.11M: Establishing the scanning imaging model of the contact image sensor. When scanning an image using the contact image sensor, a plurality of frames of data are collected at equal intervals along the scanning movement direction at a certain resolution. The plurality of frames of data are directly combined together to form a scanning area array data with distortion. Let the coordinates of the image points in the scanning area array be p = [x y 1] T , the coordinates of the image points after distortion correction be p' = [x' y' 1] T , and the relationship between them be:

[0021]

[0022] where s x is the sampling size ratio of the contact image sensor along the scanning direction, and is 1 if the sampling resolution is the same as the original image resolution; s y is the pixel imaging ratio of the contact image sensor, and is related to the pixel arrangement interval of the sensor itself; and θ is the angle between the imaging direction of the image sensor and the scanning movement direction, and is related to the installation accuracy, and is generally approximately 90 degrees.

[0023] Let the coordinates of the corresponding point of the image point p' in the original image be p'' = [x'' y'' 1] T , and since the image after distortion correction and the original image satisfy the 2D rigid body transformation relationship, there is:

[0024]

[0025] where t x and t y are the translation components of the 2D rigid body transformation, and ψ is the rotation component of the 2D rigid body transformation. H is the affine transformation matrix.

[0026] Step 2.12M: Solving the affine transformation matrix H in Step 2.11M. At least three groups of matching point coordinates in the original image and the scanning area array are used for solving. Let the solved H be:

[0027]

[0028] Step 2.13M: Solving the imaging model parameters from the affine transformation matrix H obtained in Step 2.12M. According to the imaging model parameters of the contact image sensor are calculated.

[0029] where the affine transformation matrix H mentioned in Step 2.12M is solved based on the matching point coordinates in the chessboard picture of known size and the scanning area array image, and the direct linear transformation (DLT) method is used for solving, and the specific calculation steps are as follows:

[0030] Step A: Obtain a plurality of matching point pairs (p i , p i ) (i = 1, 2, …, n, n≥3) using a picture of a chessboard with known size and its distorted scanned area image.

[0031] Step B: According to the affine transformation model between the matching point pairs, we have:

[0032]

[0033] Step C: Accumulate the equations obtained in step B for the plurality of matching point pairs to obtain a linear equation system Ah = b. Wherein

[0034]

[0035]

[0036]

[0037] Step D: Solve the least square solution of the equation system established in step C to obtain h * , and further obtain the estimated value of H.

[0038] Optionally, for the affine transformation matrix H mentioned in step 2.12M, use the known planar image and its scanned image to obtain matching point pairs based on affine transformation, and solve it using the random sample consensus (RANSAC) and direct linear transformation (DLT) method. The specific steps are as follows:

[0039] Step a: Scan the known planar image using a contact image sensor to obtain a distorted scanned area image.

[0040] Step b: Perform Gaussian blur processing on the known planar image and the scanned area image obtained in step a, respectively, to avoid the influence of high-frequency measurement noise on the accuracy of matching point coordinates.

[0041] Step c: Extract feature points from the known planar image and the scanned area image after Gaussian blur in step b, respectively, and calculate the descriptors of the feature points.

[0042] Step d: Find the candidate matching relationship for the two groups of extracted feature points according to the similarity of the feature point descriptors.

[0043] Step e: Use the random sample consensus (RANSAC) and direct linear transformation method (DLT) to find a group of matching relationships that apply the same affine transformation and have the most inliers from the candidate matching relationships.

[0044] Step f: Use all the matching inliers obtained in step e to calculate the final affine transformation result using the direct linear transformation method, which is the estimated value of H.

[0045] Where, the imaging model parameters of the linear CCD image sensor are different from that of the CIS, and the single frame image has non-linear distortion. The specific calibration steps are as follows:

[0046] Step 2.11N: Non-linear distortion correction is performed on the linear CCD single frame image. The method for estimating the non-linear distortion model parameters and the distortion correction of the linear CCD image sensor are prior art.

[0047] Step 2.12N: After the image of the linear CCD image sensor is corrected, the s x , s y , and θ parameters are calibrated in the same way as the CIS.

[0048] Hereinafter, all the image stitching and positioning methods using the CIS are also applicable to the linear CCD image sensor, and therefore the CIS and the linear image sensor are equivalent in the following description.

[0049] For the image stitching by estimating the 2D rigid body transformation between two images in step 2.4, there are various sub-methods to improve the accuracy of image stitching, which are described in detail as follows.

[0050] Optionally, the photometric error minimization image registration method is used to calculate the 2D rigid body transformation of two images. The main steps are as follows:

[0051] Step 2.41M: Low-pass filtering, such as mean filtering or Gaussian blur filtering, is performed on the two images to be registered, so as to reduce the influence of measurement noise on the photometric error and gradient calculation, and improve the convergence characteristics of the optimization function;

[0052] Step 2.42M: Uniformly select points or corner points (such as Harris corner points) with large gradient changes of gray value from the source images to be registered, as measurement items to construct the optimization objective function based on photometric error:

[0053]

[0054] Where δ = [t x t y ψ] T is the optimization variable, representing the 2D rigid body transformation, e i is the photometric error, I1 is the gray value function of the point in the source image, I2 is the gray value function of the point in the target image, p 1，i is the homogeneous coordinates of the point in the source image, and p 2，i is the homogeneous coordinates of the point in the target image, satisfying:

[0055] p 2，i = T(δ)p 1，i

[0056]

[0057] Step 2.43M: Using the coarse registration results obtained from other methods as initial values of optimization variables, and solving the optimization objective function constructed in step 2.42M by Levenberg-Marquarelt or Gauss-Newton method to obtain the 2D rigid transformation between images.

[0058] Optionally, the splicing of multiple split sub-images and a single image is used to improve the image splicing accuracy. Due to the influence of the measurement accuracy of the motion tracking sensor, the undistorted images may have non-rigid deformation, which may cause the splicing failure or error between the undistorted images or between the undistorted images and the printed images. In order to solve this problem, the registration between two images is split into the registration between multiple sub-images and a single image, and the non-rigid registration between images is realized. The specific steps are as follows:

[0059] Step 2.41N: Splitting the registration source image into a series of sub-images according to a certain width along the scanning direction.

[0060] Step 2.42N: Judging whether each sub-image has degeneration or not: extracting feature points for photometric error calculation in the sub-image; for feature point i, calculating the photometric value gradient (gx i , gy i ) of the image at the point; counting the proportion of the number of feature points with gx i > gy i , if the value is greater than 0.8 or less than 0.2 or the total number of feature points is too small, it is considered that there is degeneration.

[0061] Step 2.43N: For the first sub-image, using the prior value as the initial value, for the subsequent sub-images, using the registration results of the previous sub-images as the initial values, sequentially performing the registration of each sub-image and the target image, and using the photometric error optimization method; for the sub-image judged to have degeneration in step 2.42N, directly using the initial value as the registration result of the sub-image and the target image.

[0062] Step 2.44N: According to the registration results of each sub-image, completing the splicing of two images or the positioning of the linear image sensor at different stages in the printing process.

[0063] Optionally, the 2D ICP (Iterative Closest Point) algorithm is used for image splicing to obtain the 2D rigid transformation between images. The steps are as follows:

[0064] Step 2.4lW: Binarize the image and extract some edge or high gradient line features, such as using the Canny method.

[0065] Step 2.42W: Calculate the normal vector of the line feature after binarization in Step 2.41W.

[0066] Step 2.43W: Using the 2D rigid body transformation obtained by other methods as the initial value, optimize the 2D rigid body transformation using the 2D ICP method. The objective function of the optimization is the point-to-line distance calculated based on the normal vector obtained in Step 2.42W.

[0067] Optionally, OCR (Optical Character Recognition) text features can be used for image stitching to obtain a 2D rigid body transformation between images. This method is suitable for image registration scenarios with a large number of text features. The steps are:

[0068] Step 2.41Q: Use an OCR program to extract text features from the two registered images, including characters, words, and phrases.

[0069] Step 2.42Q: Calculate the coordinates of the center point of the bounding box of the text features extracted in Step 2.41Q.

[0070] Step 2.43Q: Match the text features in the two registered images based on the consistency of the text content, and obtain multiple matching candidates. The coordinates of the matching features are the center point coordinates calculated in step 2.42Q.

[0071] Step 2.44Q: Using the matching candidates and feature coordinates from Step 2.43Q, calculate the optimal 2D rigid body transformation based on RANSAC and singular value decomposition.

[0072] In step 3, the specific implementation steps for determining the single-line printing content based on the positioning results of the linear image sensor to achieve spliced ​​printing, taking a contact image sensor as an example, are as follows:

[0073] Step 3.1: Use a contact image sensor to scan a segment of image for positioning. The scanning width must meet the positioning accuracy requirements, and record the scanning start time.

[0074] Step 3.2: Perform distortion correction on the image scanned in Step 3.1 using the sensor imaging model, and ensure that the coordinates of the origin of the first frame scan are (0, 0) in the distortion-corrected image.

[0075] Step 3.3: Stitch the distorted image from Step 3.2 with the image printed on the surface of the printing medium to obtain the positioning result of the contact image sensor. in is a 2D rigid transformation, which is the conversion relationship between the de-distorted image coordinate system and the printing medium coordinate system.

[0076] Step 3.4: Since the contact image sensor moves along the scanning direction, the position of the starting end of the contact image sensor in the de-distorted image at the current time is obtained according to the motion measurement result Δp (pixel displacement at the current time and the starting time of scanning) of the motion tracking sensor. s = (Δp, 0).

[0077] Step 3.5: The current position of the print head in the de-distorted image is calculated using the external parameters of the print head and the contact image sensor. (i = 1… endpoint number, is the external parameter of the print head endpoint i and the contact image sensor).

[0078] Step 3.6: The current coordinates of the print head in the printing medium plane are obtained The printed content is obtained by cutting the surface of the printing medium along the printing direction to generate the range of the printed image.

[0079] In step 3.5, in order to calculate the position of the print head using the positioning result of the contact image sensor, the spatial external parameter relationship between the print head and the contact image sensor needs to be calibrated. The specific steps are as follows:

[0080] Step 3.51: Prepare a calibration paper and pre-print an image for registration on it according to a predetermined size.

[0081] Step 3.52: Place the printer on the calibration paper and print an ink line for each print head.

[0082] Step 3.53: Move the printer and use the contact image sensor to continuously scan the image at a specified resolution, ensuring that the first frame is obtained in a stationary state and the scanning data width meets the accurate positioning requirements.

[0083] Step 3.54: Perform de-distortion processing on the scanned image in step 3.53, and ensure that the first frame scanning origin has a coordinate of (0, 0) in the de-distorted image.

[0084] Step 3.55: The de-distorted image obtained in step 3.54 is spliced with the pre-set image in step 3.51 to obtain wherein is a 2D rigid transformation.

[0085] Step 3.56: Move the printer and use the contact image sensor to scan the ink line printed in step 3.52 and the partial registration image pre-set in step 3.51, and the scanning data width must meet the accurate splicing requirements and perform de-distortion processing.

[0086] Step 3.57: Directly measure the pixel coordinates of the printed ink line end points in the undistorted image in step 3.56 (i = 1… number of printhead end points).

[0087] Step 3.58: Stitch the undistorted image in step 3.56 with the pre-set image in step 3.51 to get where is a 2D rigid body transformation.

[0088] Step 3.59: Calculate the coordinates of the printed printhead end points in the undistorted image in step 3.54 (i = 1… number of printhead end points), is the spatial extrinsic relationship between the printed printhead end points and the contact image sensor.

[0089] In step 4, the use of a pre-set mark pattern on the surface of the printing medium is mentioned to improve the accuracy of single-line printing positioning initialization. The purpose is to reduce the positioning initialization error caused by the measurement error of the motion tracking sensor and the sparsity of the image features. The specific method is to pre-set a mark pattern for positioning on the printing medium. The mark pattern is visible to the naked eye, or covered with ultraviolet (or infrared) fluorescent material and visible under ultraviolet (or infrared) light. The mark pattern refers to mark lines or mark points or specific patterns and combinations, and is convenient to identify. During printing, the imaging of these mark patterns by the contact image sensor completes the positioning initialization function.

[0090] In step 4, the use of a pre-set mark pattern on the surface of the printing medium is mentioned to improve the accuracy of single-line printing positioning initialization. One way to set the mark pattern is to use two sets of straight lines (meridian clusters and latitude clusters) that are equally spaced (such as an interval distance of 1 inch, which is 600 pixels at a scanning resolution of 600 dpi) and perpendicular to each other. In order to ensure that the single-frame image of the linear image sensor covers several straight lines in these two sets during printing, the distribution direction of the two sets of straight lines is set at an angle of about 45 degrees to the printing movement direction. The coordinates of the pre-set mark line meridian cluster in the printing medium coordinate system are The coordinates of the latitude cluster are The adjacent meridians or latitudes are equally spaced (such as 600 pixels at a scanning resolution of 600 dpi), and the meridians and latitudes are perpendicular to each other.

[0091] In step 4, the use of a preset mark pattern to improve the initialization accuracy of single-line printing positioning is completed using a single frame image of the contact image sensor. The idea is to limit the position and orientation of the contact image sensor to some discrete state space through the imaging of the mark line, and then select the best match from these discrete states as the positioning result using the imaging of the part of the printing medium other than the mark line. The specific steps are as follows:

[0092] Step 4.1a: During the image printing process, the distortion-free mark line image scanned by the contact image sensor and the printing image at the same time are overlapped using the extrinsic relationship between the contact image sensor and the printing head to generate a combined image for subsequent positioning. The first line of image printing does not need to be positioned or only needs to be positioned using other methods to align with the boundary of the printing medium.

[0093] Step 4.2a: Use the contact image sensor to collect a single frame image covering the mark line and the printed image at a certain resolution setting (for example, 600 dpi), and perform distortion correction.

[0094] Step 4.3a: Extract the intersection points of the distortion-corrected single frame image with the mark line, allowing for errors.

[0095] Step 4.4a: From the adjacent intersection points extracted in step 4.3a, select point pairs ab with a distance less than the interval between the latitude and longitude lines (600 pixels). Assume that the intersection line segment of the single frame image with a mark line square is ab, and assume that the latitude direction is the x-axis and the longitude direction is the y-axis. Since ab is less than the interval between the latitude and longitude lines, a and b must be on one of the longitude lines and on one of the latitude lines. Assuming that the intersection point of the intersection points of a, b, and the square with the latitude and longitude lines is the coordinate origin O, there will be two solutions that satisfy the observation results of the intersection points extracted in step 4.3a (assuming that the intersection points of a, b, and the square with the coordinate origin O are A and B, and in special cases, if OA = OB, the two solutions overlap).

[0096] The first solution is calculated as follows:

[0097]

[0098] The second solution is calculated as follows: symmetrical transformation of the intersection point coordinates of the straight line and the latitude and longitude lines according to the straight line x = y, or exchange the x and y coordinates of the intersection points, to obtain the intersection point coordinates of the second solution. The coordinates of the matching points in the distortion-corrected single frame image remain unchanged.

[0099] Step 4.5a: A square has 4 vertices, and 2 solutions are constructed at each vertex according to the intersection distribution of the extracted mark line in step 4.3a, so 8 solutions are constructed by ab and a square. Among them

[0100] The intersection coordinates of straight lines and meridians and parallels in the third and fourth solutions are obtained by symmetric transformation of the intersection coordinates in the first and second solutions based on the straight line x = meridian and parallel interval / 2, and the coordinates of the matching points in the undistorted single frame image remain unchanged;

[0101] The intersection coordinates of straight lines and meridians and parallels in the fifth and sixth solutions are obtained by symmetric transformation of the intersection coordinates in the first and second solutions based on the straight line y = meridian and parallel interval / 2, and the coordinates of the matching points in the undistorted single frame image remain unchanged;

[0102] The intersection coordinates of straight lines and meridians and parallels in the seventh and eighth solutions are obtained by symmetric transformation of the intersection coordinates in the third and fourth solutions based on the straight line y = meridian and parallel interval / 2, and the coordinates of the matching points in the undistorted single frame image remain unchanged.

[0103] Step 4.6a: Repeat steps 4.4a-4.5a to obtain more groups of feasible discrete solutions, each group having 8 solutions, using other adjacent intersection points extracted in step 4.3a with a distance less than the meridian and parallel interval.

[0104] Step 4.7a: Select a group of 8 solutions with the most matching intersection points and the smallest matching error from the groups of feasible discrete solutions established in steps 4.4a-4.6a, and then translate the intersection coordinates of straight lines and meridians and parallels in the 8 solutions by meridian or parallel interval (600 pixels)*n in the meridian or parallel direction (including any combination), and then rotate by 45 degrees to obtain the intersection coordinates of the actual feasible discrete solution. The coordinates of the matching points in the undistorted single frame image remain unchanged.

[0105] Step 4.8a: Traverse all feasible solutions in step 4.7a, and use the singular value decomposition method to obtain the rigid body transformation from the undistorted single frame image to the combined image according to the coordinates of the matching point pairs, and calculate the photometric error between the undistorted single frame image and the combined image, and select the one with the smallest photometric error as the final solution.

[0106] In step 4, the use of a pre-set mark pattern to improve the positioning initialization accuracy of single-line printing is one method, which is to use a contact image sensor to scan multiple local images to improve the positioning initialization accuracy. The specific steps are as follows:

[0107] Step 4.1b: During image printing, the undistorted mark line image scanned by the contact image sensor and the printing image at the same time are overlapped using the extrinsic relationship between the contact image sensor and the printing nozzle to generate a combined image for subsequent positioning.

[0108] Step 4.2b: Capture partial multi-frame images covering the guide lines and the printed image using the contact image sensor at certain resolution setting (e.g. 600 dpi), stitch the multi-frame images to generate a distortion-free scan image using the partial motion obtained from the motion tracking sensor and the imaging model of the contact image sensor.

[0109] Step 4.3b: Extract line features from the scan image generated in step 4.2b.

[0110] Step 4.4b: For a line feature pair consisting of two perpendicular line features extracted in step 4.3b, consider the case that they overlap with a set of latitude and longitude lines respectively, set the latitude coordinate as y = 0 and the longitude coordinate as x = 0, construct one matching point on each of the longitude and latitude lines; since there are four possible matching relationships, four feasible solutions are obtained from one line feature pair.

[0111] Step 4.5b: Translate the matching points on the longitude and latitude lines in the four feasible solutions obtained in step 4.4b by the interval between the longitude and latitude lines (600 pixels) * n in the direction of the longitude or latitude line (including any combination), and then rotate by 45 degrees to obtain more feasible discrete solutions.

[0112] Step 4.6b: Repeat steps 4.4b-4.5b using the other two perpendicular line features extracted in step 4.3b to obtain more feasible discrete solutions.

[0113] Step 4.7b: Traverse all feasible discrete solutions established in steps 4.4b-4.6b, obtain the rigid transformation from the distortion-free image to the combined image using the singular value decomposition method according to the coordinates of the established matching point pairs, calculate the photometric error between the distortion-free image and the combined image, and select the one with the smallest photometric error as the final solution.

[0114] In step 5, it is mentioned that the pre-set mark pattern on the surface of the printing medium is used to improve the positioning and tracking accuracy during printing, aiming to eliminate the cumulative positioning error caused by local motion measurement error or sparse image features during printing. The setting method of the mark pattern used in this step is the same as step 4. The specific positioning and tracking steps using the guide lines are as follows:

[0115] Step 5.1: During printing, capture a single frame of image using the contact image sensor and perform distortion correction.

[0116] Step 5.2: Extract the intersection point with the guide lines from the distortion-corrected single frame image obtained in step 5.1 and obtain its coordinates in the distortion-corrected single frame image.

[0117] Step 5.3: Search for the guide lines near the intersection point in the printing medium coordinate system according to the pre-correction positioning result and obtain their coordinates in the printing medium coordinate system.

[0118] Step 5.4: Using the intersection points obtained in step 5.2 and step 5.3 and the matching mark lines, adjust the 2D rigid body transformation between the rectified single frame image and the printing medium coordinate system by minimizing the point line distance to obtain the optimized image acquisition time positioning result.

[0119] The advantages and beneficial effects of the present application are that:

[0120] The present application proposes a method for improving the image stitching and positioning accuracy of a portable printing and copying machine using a linear image sensor, effectively overcoming the image stitching and positioning accuracy errors caused by the local motion measurement accuracy limitations of motion tracking sensors and the sparsity of image features used for registration in some scenes, thereby improving the image scanning and stitching quality of the portable printing and copying machine and the printing image stitching quality. Using the method in the present application, even if there is an error of less than 1% in local motion measurement, or there are no effective visual features in the printing image within a range of 10cm*10cm, the image stitching or positioning accuracy of <0.2mm can still be achieved, and there is no cumulative error. BRIEF DESCRIPTION OF DRAWINGS

[0121] Figure 1 is a bottom view of an implementation example of a portable printing and copying machine.

[0122] Figure 2 is an imaging model diagram of a contact image sensor.

[0123] Figure 3 is a method diagram for contact image sensor imaging model calibration using a checkerboard image.

[0124] Figure 4 is a method diagram for contact image sensor imaging model calibration using a planar image scanning result.

[0125] Figure 5 is a schematic diagram of multiple split subgraphs and a single image registration.

[0126] Figure 6 is a schematic diagram of two image registration using OCR text features.

[0127] Figure 7 is a schematic diagram of a method for improving printing positioning accuracy using mark lines.

[0128] Figure 8 is a schematic diagram of a mark line being disconnected at its intersection point.

[0129] Figure 9 is a schematic diagram of combining mark lines and printing medium surface images as a contact image sensor positioning and registration target image.

[0130] Figure 10This is a schematic diagram of the intersection of a single frame image from a contact image sensor and a marker line.

[0131] Figure 11 This is a schematic diagram illustrating the determination of the first and second sets of positioning solutions based on the intersection relationship between a single frame image from a contact image sensor and the square marker line.

[0132] Figure 12 This is a schematic diagram of obtaining the third and fourth positioning solutions based on the intersection relationship between a single frame image from a contact image sensor and the square marker line.

[0133] Figure 13 This is a schematic diagram of obtaining the fifth and sixth sets of positioning solutions based on the intersection relationship between a single frame image from a contact image sensor and the square marker line.

[0134] Figure 14 This is a schematic diagram for determining the seventh and eighth positioning solutions based on the intersection relationship between a single frame image from a contact image sensor and the square marker line.

[0135] Figure 15 This is a schematic diagram of using a pair of perpendicular lines and marker lines in multiple frames of images from a contact image sensor to find four sets of localization solutions.

[0136] Figure 16 This is a schematic diagram illustrating the use of a single frame image from a contact image sensor and marker lines to eliminate tracking and positioning errors.

[0137] Figure 17 This is a schematic diagram illustrating a method for improving printing positioning accuracy using marker points.

[0138] The labels in the diagram are explained as follows: 1-Horizontal scroll wheel, 2-Optical mouse sensor, 3-Contact image sensor, 4-Printer nozzle, 5-Scanning and printing movement direction;

Detailed Implementation Methods

[0139] Figure 1A bottom view of a portable printing and copying machine based on the hardware setting method of the present application is provided. The portable printing and copying machine is provided with: horizontal rollers (1) arranged at four corners of the base, for guiding stable movement of the machine body and limiting movement of the machine body only along the scanning and printing movement direction (5) when scanning and splicing printing is performed; photoelectric mouse sensors (2), two of which are used for tracking and measuring local movement of the machine body to obtain displacement of the machine body, with a relative error of <1%; contact image sensors (3), two of which are arranged at two sides of the machine body, perpendicular to the scanning and printing movement direction (5), and spaced apart from the scanning medium by 0.5-2 mm, with a scanning resolution of 600 dpi, for collecting images on the surface of the scanning medium or the printing medium, the contact image sensors having light sources which are turned on when image scanning is performed; and a printing head (4) which is spaced apart from the printing medium by about 2 mm and is used for printing images, with a printing resolution of 600 dpi.

[0140] For the portable printing and copying machine example in Figure 1 , in order to improve the image scanning, splicing and positioning accuracy thereof using data of the contact image sensor, an imaging model of the contact image sensor is first established. Figure 2 A space model of the scanning medium coordinate system and the contact image sensor is established. Due to installation errors, the included angle θ between the pixel linear array direction of the contact image sensor and the scanning and printing movement direction of the portable printing and copying machine is not completely perpendicular. In addition, there are proportional factors in the pixel resolution of the contact image sensor, the displacement measurement of the photoelectric mouse sensor and the standard size. In the printing and copying operation, the machine body is moved along the scanning and printing movement direction (5), and data of the contact image sensor is collected at equal intervals according to a certain resolution, and the original scanning area array is obtained after direct arrangement. Let the coordinates of the image points in the original scanning area array be p = [x y 1] T , and the coordinates of the corresponding points in the scanning medium plane be p" = [x" y" 1] T According to the sensor imaging model established in step 2.11M, we have

[0141]

[0142] where t x , t y and ψ are the initial position and orientation of the sensor, which depend on the space state of the sensor at the initial scanning time relative to the scanning medium and are different for each operation; s x , s y and θ are the imaging model parameters of the sensor, which depend on the hardware setting of the printing and copying machine itself, and are relatively stable due to stable hardware structure in a short time; and H is an affine transformation matrix.

[0143] For the portable printing and copying machine example in Figure 1In the example of a portable printer / copier, to improve the image scanning stitching and positioning accuracy using data from a contact image sensor, it is necessary to calibrate the sensor imaging model parameters s. x s y θ. In one implementation, as shown... Figure 3 The image shown is a checkerboard pattern and its distorted scanned area image. The checkerboard squares are 20cm x 20cm in size. A distorted scanned area image was obtained by scanning the checkerboard image using a contact image sensor. The intersection points of the squares in the checkerboard image were extracted, and their coordinates p″ in the checkerboard image were calculated. i =(x″ i ,y″ i (i = 1…n), and obtain the matching points and coordinates p in the scanned area array image. i =(x i y i (i = 1…n). Using three or more pairs of matching point coordinates, solve for H using the Direct Linear Transformation (DLT) method. Specifically, construct the linear equation system Ah = b, where…

[0144]

[0145]

[0146]

[0147] Find the least squares solution to the above system of equations to obtain an estimate of H, h. * Furthermore, based on The imaging model parameters of the contact image sensor were calculated.

[0148] for Figure 1 In the example of a portable printer / copier, to improve the image scanning stitching and positioning accuracy using data from a contact image sensor, it is necessary to calibrate the sensor imaging model parameters s. x s y θ. In one implementation, as shown... Figure 4The picture and its distorted scan plane image are shown. There are rich patterns in the picture, which are convenient for extracting and searching for matching visual feature points. After Gaussian blur processing of the original picture and the distorted scan plane image, high-frequency noise is removed, and two images for affine transformation estimation are obtained. Visual feature points in the two images are extracted using a visual feature point calculation method (such as the Harris method), and the descriptors of the visual feature points are calculated (such as the SIFT method). The similarity of the feature point descriptors is used to find matching candidates. The random sample consensus (RANSAC) and direct linear transformation (DLT) are used to find a set of matching relationships that apply the same affine transformation and have the most inliers from the candidate matching relationships, and then the optimal estimation of H is obtained. The random sample consensus (RANSAC) method is used to exclude the candidate matching feature point pairs that are approximately or consistent locally but do not conform to the affine transformation model, thereby improving the accuracy of affine transformation calculation. In the implementation, the visual feature extraction is not limited to the Harris method, and the visual feature descriptor calculation is not limited to the SIFT method. Other alternative methods can be used to extract visual feature points and calculate descriptors. Experiments show that using this method, the angle parameter θ of the sensor model has a calibration error of <0.3 degrees, and the scale factor s x y

[0149] The imaging model and parameters of the contact image sensor are used to de-distort the original scan plane image collected by the contact image sensor, so as to obtain an image that conforms to the original size, which is used for scan image stitching and printing positioning. Specifically, for the de-distorted image point p' = [x'y'1] T , the point p = [x y 1] T in the original scan plane image matched therewith is calculated by the method

[0150]

[0151] Then the value of each pixel in the de-distorted scan image can be calculated. This is a specific method for de-distortion using the imaging model of the contact image sensor.

[0152] The de-distorted contact image sensor scan image is used for stitching scanning or stitching printing with the image on the printing medium surface. The stitching or registration between two images is performed by estimating the 2D rigid body transformation therebetween, which describes the coordinate conversion relationship of the matching points in the two images. Therefore, the image registration accuracy directly affects the image stitching and positioning accuracy of the portable printing copier. The present application proposes a variety of methods to improve the image registration accuracy. Therefore, in reality, any method to improve the image registration accuracy is beneficial to improve the accuracy of the portable printing copier.

[0153] ​​In order to improve the image registration accuracy, the method of using multiple split sub-images and single image stitching is proposed in the case. Figure 5 A specific example of the method is given. Where the linear image sensor scans the image range is the sum of the parts covered by the dashed box, it needs to be stitched with the complete image on the scanning medium to locate the linear image sensor. Due to the measurement error of the motion tracking sensor, the de-distorted overall scanning image has non-rigid deformation, resulting in failure or large error in stitching with the scanning medium image based on 2D rigid transformation estimation. Therefore, the scanning image is first split into 5 sub-images along the scanning direction at equal intervals. In turn, the 5 sub-images are judged for whether there is degeneration: for the 3rd sub-image from the left, there is a horizontal degeneration phenomenon; the other sub-images do not have degeneration. From left to right, each sub-image is registered with the target image (scanning medium image): for the 1st sub-image, use the positioning result obtained by other positioning methods as the priori, use the photometric error optimization method to register with the target image to obtain the optimized positioning result; for the 2nd sub-image, use the registration result of the 1st sub-image as the priori, continue to register with the target image using the photometric error optimization method to obtain the optimized positioning result; for the 3rd sub-image, since there is degeneration, do not perform registration and directly use the registration result of the 2nd sub-image as the registration result of the 3rd sub-image with the target image; for the 4th sub-image, use the registration result of the 3rd (actually 2nd) sub-image with the target image as the priori, continue to register with the target image using the photometric error optimization method to obtain the optimized positioning result; for the 5th sub-image, similar processing is performed. In this way, the registration error of each sub-image is reduced, the positioning accuracy of the linear image sensor at different stages is improved, and the stitching accuracy of multi-line printing is further improved.

[0154] In order to improve the image registration accuracy, the method of using OCR text features to stitch two images is proposed in the case. Figure 6 A specific example of the method is given. There are common text features in the left and right two images that need to be registered. First, use the OCR program to extract the text features in the two registration images to obtain the text content, bounding box and center point coordinates of the text features, and use the center point coordinates as the coordinates of the text features. Then, match the text features in the left and right two images according to the consistency of the text content to obtain multiple matching pair candidates. For example, both the left and right two images include "Su Shi", "Mo Ting", "Xu Xing", and "Ma". Finally, use the coordinates of the candidate matching feature pairs to calculate the 2D rigid transformation with the most inliers and the smallest projection error based on RANSAC and singular value decomposition method, thereby excluding false matching candidates (such as "Mo Ting" in the left and right two images). This method has strong robustness, and the matching of text features has strong directionality and high uniqueness.

[0155] To solve the problem of positioning initialization error caused by the measurement error of motion tracking sensors and the sparsity of image feature registration, a pre-set mark pattern on the surface of the printing medium is used. Figure 7 A specific example of using mark lines to improve the printing positioning accuracy is shown. In the defined printing medium coordinate system, the mark lines are two groups of straight lines (meridian cluster and parallel cluster) with equal intervals of 1 inch and perpendicular to each other. The distribution direction of the two groups of straight lines is about 45 degrees to the printing movement direction. The coordinates of the meridian cluster of the mark lines pre-set in the printing medium coordinate system are The coordinates of the parallel cluster are The adjacent meridians or parallels are separated by 600 pixels in the 600 dpi resolution image, and the meridians are perpendicular to the parallels. The two groups of mark lines are printed by materials that reflect infrared light. In some embodiments, the meridians and the parallels are printed by materials with different widths, line types or reflecting different wave bands, so as to distinguish them in the image of the contact image sensor and improve the positioning success rate and efficiency. In order to avoid the imaging of the contact image sensor near the intersection of the two mark lines from causing mismatch with the mark lines, in some embodiments, one of the mark lines is disconnected near the intersection of the two mark lines, as shown in Figure 8 .

[0156] Single-frame imaging of the mark lines and the surface image of the printing medium by the contact image sensor is used to complete the positioning initialization. First, the mark lines and the surface image of the printing medium are combined to form a target image for positioning registration, and a specific example is shown in Figure 9 . When combining the mark lines and the surface image of the printing medium, the positioning result of the contact image sensor during the printing process of the surface image of the printing medium and the spatial relationship between the printing head and the contact image sensor are used. When using single-frame imaging of the mark lines for positioning initialization, the intersection of the contact image sensor and the mark lines is first extracted using the single-frame imaging after distortion. In the example of Figure 10 , a total of 6 mark line intersections a2, a1, a, b, b1, b2 are extracted from a certain frame of data. The distances of a2a1, ab, b1b2 are all less than the interval (1 inch) between adjacent parallel mark lines. Considering the case that ab intersects a mark line square, the intersection points are A and B, and the vertices of the meridian and parallel intersecting ab are O, as shown in Figure 11 . The intersection of the straight line AB and the mark line in the two solutions meets the distribution of the extracted mark line intersections. Further, according to the two solutions, other solutions are found in which the intersection of the square is ab, Figures 12-14 and the distribution of the other 6 solutions is shown. Therefore, a total of 8 alternative solutions are found by the two intersections with a distance less than the interval between adjacent parallel mark lines and a mark line square. Using the intersection pair a2a 1、b1b2The above process is repeated to obtain more candidate discrete solutions. From all the discrete solution groups, a group of 8 solutions with the most matching intersection points and the smallest matching error is selected. The 8 candidate solutions are then translated along the meridian or parallel direction (including limited arbitrary combinations) by 600*n pixels, and then rotated by 45 degrees to obtain all candidate discrete solutions. The 2D rigid body transformation of these candidate discrete solutions is calculated, and the photometric error of the undistorted single frame image and the combined image in these solutions is calculated using the 2D rigid body transformation, and the one with the smallest photometric error is selected as the final solution. Overall, the pose of the contact image sensor is quickly locked to a limited discrete state using the intersection points of the mark line, and then the optimal one is selected as the final solution through the method of minimizing the photometric error. Using this method, the positioning initialization time is effectively shortened to less than 10ms, and the positioning error is less than 0.1mm.

[0157] When using multi-frame imaging of the mark line to initialize positioning, first, the mark line image is combined with the surface image of the printing medium to generate a target image for registration, which is similar to the single-frame positioning initialization method. Then, the contact image sensor is used to capture local multi-frame images covering the mark line and the printed image, and the images are undistorted for positioning. The Hough transform is used to extract line features in the undistorted image of the contact image sensor. As shown in the example, Figure 15 Assuming that point a on the line feature overlaps point A on the meridian and satisfies OA=oa, and point b on the line feature overlaps point B on the parallel and satisfies OB=ob; then the 2D rigid body transformation of the undistorted image to the scanning medium is solved using the coordinates of the two sets of matching point pairs. Since there are 4 matching ways between a set of perpendicular line features and a set of meridians and parallels, 4 candidate solutions are obtained. The 4 candidate solutions are translated along the meridian or parallel direction (including limited arbitrary combinations) by 600*n pixels, and then rotated by 45 degrees to obtain more feasible discrete solutions. Other candidate discrete solutions are obtained through more mutually perpendicular line features. Finally, the photometric error of the undistorted image and the registration target image is calculated using the candidate discrete solutions, and the one with the smallest photometric error is selected as the final solution.

[0158] Examples of using pre-set mark lines on the surface of the printing medium to improve the positioning tracking accuracy during printing are shown in Figure 16Single frame images of the contact image sensor are acquired during the printing process, and are de-warped, and the intersection points with the fiducial lines are extracted. Since there is some error in the contact image sensor pose before rectification (represented by a 2D rigid body transformation from the de-warped single frame image to the print media coordinate system), the projected points of the extracted fiducial line intersection points to the print media coordinate system using the pose before rectification do not exactly overlap with the fiducial lines near the projected points. The distance sum (or distance square sum) of the projected points to the fiducial lines near the projected points is used as a cost function to optimize the contact image sensor pose, and the rectified pose is obtained. This method is used to reduce the tracking error in the single pass printing process due to the measurement error of the motion tracking sensor or the sparsity of the image features.

[0159] In addition to using the fiducial lines pre-set on the print media surface, other types of fiducial patterns such as fiducial points are used to improve the printing position initialization accuracy. One way of setting the fiducial points is shown in FIG. 4, in which a number of fiducial points are set on the print media surface with equal intervals (1 inch, 600 pixels under 600 dpi scanning resolution) in the horizontal and vertical directions. Figure 17 In the position initialization process, first, a number of consecutive frames of the contact image sensor are acquired at 600 dpi resolution, and are de-warped. Then, the fiducial point features in the de-warped images are extracted, and the pairs of fiducial point features with an interval near 600 pixels are found. Considering the case that a pair of points overlaps with a fiducial point square, there are 8 matching modes in total; more matching candidates are obtained by translating the coordinates of the matching points on the print media surface by 600*n pixels in the horizontal or vertical direction (including any combination). For each matching candidate, a 2D rigid body transformation from the de-warped image coordinate system to the print media coordinate system is obtained using the singular value decomposition method. More 2D rigid body transformations are obtained by repeating the above steps using other pairs of fiducial point features with an interval near 600 pixels. Finally, the optimal 2D rigid body transformation is selected using the photometric error minimization method, and the position initialization is completed.

[0160] In addition to using the fiducial lines pre-set on the print media surface, other types of fiducial patterns such as fiducial points are used to improve the printing position tracking accuracy. One way of setting the fiducial points is shown in FIG. 4, in which a number of fiducial points are set on the print media surface with equal intervals (1 inch, 600 pixels under 600 dpi scanning resolution) in the horizontal and vertical directions. Figure 17 In the position tracking process, first, a number of consecutive frames of the contact image sensor are acquired at 600 dpi resolution, and are de-warped. Then, the fiducial point features in the de-warped images are extracted, and the pairs of fiducial point features with an interval near 600 pixels are found. Considering the case that a pair of points overlaps with a fiducial point square, there are 8 matching modes in total; more matching candidates are obtained by translating the coordinates of the matching points on the print media surface by 600*n pixels in the horizontal or vertical direction (including any combination). For each matching candidate, a 2D rigid body transformation from the de-warped image coordinate system to the print media coordinate system is obtained using the singular value decomposition method. More 2D rigid body transformations are obtained by repeating the above steps using other pairs of fiducial point features with an interval near 600 pixels. Finally, the optimal 2D rigid body transformation is selected using the photometric error minimization method, and the position initialization is completed.

[0161] In some embodiments, the localization initialization is performed using joint imaging of the print medium surface by the linear image sensor and the camera. This combines the high localization accuracy of the linear image sensor and the wide perception range of the camera, and improves the success rate of the localization initialization. Taking the use of a marker line for localization initialization as an example, one method of using the linear image sensor and the camera for joint localization initialization is as follows: first, the imaging of the marker line by the linear image sensor limits the pose of the linear image sensor to some discrete states, and the pose of the camera is converted to the candidate states of the camera pose according to the extrinsic parameters of the camera and the linear image sensor; finally, the photometric error between the camera image and the print medium surface image is calculated using the candidate poses, and the state with the smallest photometric error is selected as the final localization result. Another method of using the linear image sensor and the camera for joint localization initialization is as follows: first, the camera pose is solved by the visual feature point extraction, matching and RANSAC PnP (Perspective-n-Point) method using the camera image and the print medium surface image, and then the pose of the linear image sensor is converted according to the extrinsic parameters of the camera and the linear image sensor; finally, the pose of the linear image sensor is optimized using the marker line intersection in the single frame of data collected by the linear image sensor with the above pose as the initial value, and the final localization result is obtained.

[0162] In some embodiments, the localization initialization is performed using joint imaging of the print medium surface by the linear image sensor and the camera. This combines the high localization accuracy of the linear image sensor and the wide perception range of the camera, and improves the success rate of the localization initialization. Taking the use of a marker line for localization initialization as an example, one method of using the linear image sensor and the camera for joint localization initialization is as follows: first, the imaging of the marker line by the linear image sensor limits the pose of the linear image sensor to some discrete states, and the pose of the camera is converted to the candidate states of the camera pose according to the extrinsic parameters of the camera and the linear image sensor; finally, the photometric error between the camera image and the print medium surface image is calculated using the candidate poses, and the state with the smallest photometric error is selected as the final localization result. Another method of using the linear image sensor and the camera for joint localization initialization is as follows: first, the camera pose is solved by the visual feature point extraction, matching and RANSAC PnP (Perspective-n-Point) method using the camera image and the print medium surface image, and then the pose of the linear image sensor is converted according to the extrinsic parameters of the camera and the linear image sensor; finally, the pose of the linear image sensor is optimized using the marker line intersection in the single frame of data collected by the linear image sensor with the above pose as the initial value, and the final localization result is obtained.

Claims

1. A method for improving the image stitching accuracy and positioning accuracy of a portable printer / copier, characterized in that, The steps include the following: Step 1: Configure the linear image sensor and motion tracking sensor in the portable printer / copier hardware; Step 2: Improve the image stitching accuracy of portable printers and copiers by calibrating the imaging model parameters of the linear image sensor, removing distortion from the acquired images, and performing image stitching based on 2D rigid body transformation estimation. Step 3: Determine the single-line printing content based on the positioning results of the linear image sensor to achieve splicing printing; Step 4: Improve the accuracy of single-line printing positioning initialization by using preset marking patterns on the surface of the printing media; Step 5: Improve the accuracy of positioning and tracking during the printing process by using preset marking patterns on the surface of the printing media; Step 2 specifically includes the following steps: Step 2.1: Calibrate the imaging model parameters of the linear image sensor; the calibration accuracy has a direct impact on improving the image stitching accuracy, and it must be recalibrated after the image sensor parameters change; Step 2.2: Acquire data from the linear image sensor and motion tracking sensor; move the portable printer / copier along a straight line to scan the image on the medium, and continuously acquire multiple frames of data from the linear image sensor at equal intervals with a certain resolution based on the local motion measurement results of the motion tracking sensor; Step 2.3: Using the imaging model of the linear image sensor and the data acquired in step 2.2, obtain a distortion-free single-line or local original-size scan image; Step 2.4: Perform 2D rigid body transformation-based stitching on the multiple rows of original-size scan images obtained from multiple executions of Step 2.2-2.3 to obtain a larger-area original-size scan image, thus realizing stitched scanning; or perform 2D rigid body transformation-based stitching on the local scan image obtained from a single execution of Step 2.2-2.3 and the image of the printing medium surface to obtain the positioning result of the linear image sensor, thus realizing stitched printing; In step 5, a pre-set marking pattern on the surface of the printing medium is used to improve the positioning and tracking accuracy during printing. The marking pattern setting method used in this step is the same as in step 4; the marking line is used as the marking pattern, and the specific steps are as follows: Step 5.1: During the printing process, a contact image sensor is used to acquire a single frame image, and distortion correction is performed. Step 5.2: Extract the intersection point with the marker line from the distortion-free single-frame image obtained in Step 5.1, and obtain its coordinates in the distortion-free single-frame image; Step 5.3: Based on the positioning results before correction, search for the marker line near the intersection point in the printing media coordinate system and obtain its coordinates in the printing media coordinate system; Step 5.4: Using the intersection points and matching marker lines obtained in Steps 5.2 and 5.3, the 2D rigid body transformation between the distortion-free single-frame image and the coordinate system of the printing medium is adjusted by minimizing the distance between the points and lines to obtain the optimized image acquisition time positioning result.

2. The method for improving image stitching accuracy and positioning accuracy of a portable printer / copier according to claim 1, characterized in that: Step 1 specifically includes the following steps: Step 1.1: Set up one or more linear image sensors in the portable printer / copier to acquire a raw-size image of the surface of the scanned or printed media; Step 1.2: Install one or more motion tracking sensors in the portable printer / copier to measure the local motion of the printer / copier.

3. A method for improving image stitching accuracy and positioning accuracy in a portable printer / copier according to claim 1 or 2, characterized in that: Linear image sensors refer to contact image sensors (CIS) or linear CCD image sensors; motion tracking sensors refer to photoelectric or laser mouse sensors, grating or capacitive or magnetic grating sensors, photoelectric encoders, or IMUs.

4. The method for improving image stitching accuracy and positioning accuracy of a portable printer / copier according to claim 1, characterized in that: In step 2.1, using a contact image sensor (CIS) as a linear image sensor, the steps for calibrating its imaging model parameters are as follows: Step 2.11M: Establish the scanning imaging model of the contact image sensor; when scanning images using the contact image sensor, acquire multiple frames of data at equal intervals with a certain resolution along the scanning movement direction; combine the multiple frames of data directly to form a distorted scanning array data; the imaging model describes the relationship between the scanning array data and the coordinates of matching points in the original image, and is represented by an affine transformation matrix. Let the coordinates of the image point in the scanned array be... The coordinates of the image point after distortion correction are The relationship between the two is as follows: ； in, It is the sampling size ratio of the contact image sensor along the scanning direction. If the sampling resolution is the same as the original image resolution, it is 1. It is the pixel imaging ratio of a contact image sensor, which is related to the pixel arrangement spacing of the sensor itself; It is the angle between the imaging direction of the image sensor and the scanning movement direction, which is related to the installation accuracy; Let the image point The coordinates of the corresponding point in the original image are Since the distorted image and the original image satisfy the 2D rigid body transformation relationship, we have: ； in, , It is the translation component of a 2D rigid body transformation. It is the rotational component of a 2D rigid body transformation; It is the affine transformation matrix; Step 2.12M: Solve the affine transformation matrix using at least three sets of non-collinear matching points in the original image and the scanned array, obtaining... ； Step 2.13M: Using the affine transformation matrix The imaging model parameters of the contact image sensor are calculated; based on , , , by affine transformation matrix The imaging model parameters of the contact image sensor were calculated.

5. The method for improving image stitching accuracy and positioning accuracy of a portable printer / copier according to claim 4, characterized in that: The affine transformation matrix in step 2.12M Based on the coordinates of matching point pairs in a known-size checkerboard image and a scanned area array image, the Direct Linear Transform (DLT) method is used to solve the problem. The specific calculation steps are as follows: Step A: Obtain multiple matching point pairs using a checkerboard image of known size and its distorted scanned area image. ; Step B: Based on the affine transformation model between matching point pairs, we have: ； ； ； Step C: Accumulate multiple matching point pairs of the equations obtained in Step B to obtain a system of linear equations. ;in, ； ； ； Step D: Find the least squares solution to the system of equations established in step C. And thus obtain The estimated value.

6. The method for improving image stitching accuracy and positioning accuracy of a portable printer / copier according to claim 4, characterized in that: For the affine transformation matrix mentioned in step 2.12M The matching point pairs are obtained by affine transformation-based registration using a known planar image and its scanned image, and then solved using the Random Sample Consensus (RANSAC) and Direct Linear Transform (DLT) methods. The specific steps are as follows: Step a: Use a contact image sensor to scan a known planar image to obtain a distorted scanned area image; Step b: Perform Gaussian blur processing on the known planar image and the scanned area array image obtained in step a, respectively, to avoid the influence of high-frequency measurement noise on the accuracy of the matching point coordinates; Step c: Extract visual feature points from the known planar image and the scanned area image after Gaussian blurring in step b, and calculate the descriptors of the feature points; Step d: Find alternative matching relationships for the two extracted sets of feature points based on the similarity of the feature point descriptors; Step e: Use Random Sample Consensus (RANSAC) and Direct Linear Transform (DLT) to find the set of matching relations with the most interior points that are subject to the same affine transformation from the candidate matching relations; Step f: Using all the matched interior points obtained in step e, calculate the final affine transformation result using the direct linear transformation method, and use it as... The estimated value.

7. The method for improving image stitching accuracy and positioning accuracy of a portable printer / copier according to claim 4, characterized in that: In step 2.1, for a linear CCD image sensor, its imaging model differs from that of a contact image sensor, resulting in nonlinear distortion in a single frame image. The specific calibration steps are as follows: Step 2.11N: Perform nonlinear distortion correction processing on the single-frame image of the linear CCD; Step 2.12N: After distortion correction of the image from the linear CCD image sensor, its... , , The parameter calibration method is the same as that for contact image sensors.

8. The method for improving image stitching accuracy and positioning accuracy of a portable printer / copier according to claim 1, characterized in that: In step 2.4, image stitching is performed by estimating the 2D rigid body transformation between the two images, specifically as follows: The 2D rigid body transformation of two images is calculated using an image registration method that minimizes photometric error; the steps are as follows: Step 2.41M: Perform low-pass filtering on the two images to be registered, including mean filtering or Gaussian blur filtering, to reduce the impact of measurement noise on photometric error and gradient calculation, and improve the convergence characteristics of the optimization function; Step 2.42M: Uniformly select points or corners with large gray-level gradient changes from the source image to be registered, and use them as measurement terms to construct an optimization objective function based on photometric error: ； in It is an optimization variable that represents a 2D rigid body transformation. It is a photometric error. The grayscale function for registering points in the source image. To register the grayscale function of points in the target image, The homogeneous coordinates of the points in the source image. Let the homogeneous coordinates of a point in the target image satisfy: ； ； Step 2.43M: Use the coarse registration results as the initial values ​​of the optimization variables, and solve the optimization objective function using the Levenberg-Marquarelt or Gauss-Newton method to obtain the 2D rigid body transformation between images.

9. The method for improving image stitching accuracy and positioning accuracy of a portable printer / copier according to claim 1, characterized in that: In step 2.4, the stitching accuracy is improved by combining multiple sub-images with a single image; the registration between two images is broken down into the registration between multiple sub-images and a single image, achieving non-rigid registration between images; the specific steps are as follows: Step 2.41N: Divide the registered source image into a series of sub-images along the scanning direction with a certain width; Step 2.42N: Determine if each sub-image exhibits degradation: Extract feature points from the sub-images for photometric error calculation; for feature points... Calculate the photometric gradient of the image at that point. ; Statistical subplot The percentage of feature points is considered to indicate degradation if the value is greater than 0.8 or less than 0.2, or if the total number of feature points is too small. Step 2.43N: For the first sub-image, use the prior value as the initial value; for subsequent sub-images, use the registration result of the previous sub-image as the initial value. Register each sub-image with the target image in sequence, using a photometric error optimization method for registration. For sub-images identified as having degradation in Step 2.42N, do not perform registration; directly use the initial value as the registration result between the sub-image and the target image. Step 2.44N: Based on the registration results of each sub-image, complete the positioning of the linear image sensor at different stages during the stitching or printing process of the two images.

10. The method for improving image stitching accuracy and positioning accuracy of a portable printer / copier according to claim 1, characterized in that: In step 2.4, 2D ICP is used to stitch the images together, resulting in 2D rigid body transformations between the images; the steps are as follows: Step 2.41W: Binarize the image and extract edge or high gradient line features; Step 2.42W: Calculate the normal vector of the line feature after binarization in Step 2.41W; Step 2.43W: Using the obtained 2D rigid body transformation as the initial value, optimize the 2D rigid body transformation using the 2D ICP method. The objective function of the optimization is the point-to-line distance calculated using the normal vector.

11. The method for improving image stitching accuracy and positioning accuracy of a portable printer / copier according to claim 1, characterized in that: In step 2.4, optical character recognition (OCR) text features are used to stitch images together, resulting in a 2D rigid body transformation between images; the steps are as follows: Step 2.41Q: Use an OCR program to extract text features from the two registered images, including characters, words, and phrases; Step 2.42Q: Calculate the coordinates of the center point of the extracted text feature bounding box; Step 2.43Q: Match the text features in the two registered images based on the consistency of the text content, and obtain multiple matching candidates. The coordinates of the matching features use the center point coordinates. Step 2.44Q: Using matching alternatives and feature coordinates, calculate the optimal 2D rigid body transformation based on RANSAC and singular value decomposition.

12. The method for improving image stitching accuracy and positioning accuracy of a portable printer / copier according to claim 1, characterized in that: Step 3 also includes the following steps: Step 3.1: Use a contact image sensor to scan a segment of image for positioning. The scanning width must meet the positioning accuracy requirements, and record the scanning start time. Step 3.2: Perform distortion correction on the image scanned in Step 3.1 using the sensor imaging model, and ensure that the coordinates of the origin of the first frame scan are (0, 0) in the distortion-corrected image; Step 3.3: Stitch the distorted image from Step 3.2 with the image already printed on the printing medium surface to obtain the positioning result of the contact image sensor. ;in It is a 2D rigid body transformation, which is the transformation relationship between the distortion-reduced image coordinate system and the printing medium coordinate system; Step 3.4: Since the contact image sensor moves along the scanning direction, based on the motion measurement results from the motion tracking sensor... This yields the position of the starting end of the contact image sensor in the distortion-free image at the current moment. ; Step 3.5: Calculate the current position of the printhead in the distortion-free image using the extrinsic parameters of the printhead and the contact image sensor. ; ; Step 3.6: Obtain the current coordinates of the print head in the printing media plane. The image to be printed is cut off from the surface of the printing medium by the range generated by its movement along the printing direction, thus obtaining the printed content.

13. The method for improving image stitching accuracy and positioning accuracy of a portable printer / copier according to claim 12, characterized in that: In step 3.5, in order to calculate the position of the printhead using the positioning results of the contact image sensor, it is necessary to calibrate the spatial extrinsic relationship between the printhead and the contact image sensor; the specific steps are as follows: Step 3.51: Prepare calibration paper and pre-print the image for registration on it according to the predetermined size; Step 3.52: Place the printer on the calibration paper and print one ink line for each printhead; Step 3.53: Move the printer and use a contact image sensor to continuously scan images at a specified resolution. It is necessary to ensure that the first frame is acquired while the printer is stationary and that the width of the scanned data meets the requirements for accurate positioning. Step 3.54: Perform distortion correction processing on the scanned image in step 3.53, and ensure that the coordinates of the origin of the first frame scan are (0, 0) in the distortion-corrected image; Step 3.55: The distortion-corrected image obtained in Step 3.54 is stitched together with the preset image obtained in Step 3.

51. ,in It is a 2D rigid body transformation; Step 3.56: Move the printer and use a contact image sensor to scan the ink lines printed in Step 3.52 and the pre-registered partial image in Step 3.

51. The width of the scanned data must meet the requirements for accurate stitching and distortion correction must be performed. Step 3.57: Directly measure the pixel coordinates of the endpoints of the printed ink lines in the image distorted in step 3.

56. ; Step 3.58: Stitch the distorted image from Step 3.56 with the preset image from Step 3.51 to obtain... ,in It is a 2D rigid body transformation; Step 3.59: Calculate the coordinates of the printhead endpoint in the distortion-free image from step 3.

54. , for the printhead tip The spatial extrinsic relationship between the contact image sensor and the sensor.

14. The method for improving image stitching accuracy and positioning accuracy of a portable printer / copier according to claim 1, characterized in that: In step 4, the accuracy of single-line printing positioning initialization is improved by using a pre-set marking pattern on the surface of the printing medium. Specifically, a marking pattern for positioning is pre-set on the printing medium. The marking pattern is visible to the naked eye, or it is covered with ultraviolet or infrared fluorescent material and is visible under ultraviolet or infrared light. The marking pattern refers to the graphic and combination of marking lines or marking points, and is easy to identify. During the printing process, a contact image sensor is used to image these marking patterns, completing the positioning initialization function.

15. The method for improving image stitching accuracy and positioning accuracy of a portable printer / copier according to claim 1, characterized in that: In step 4, a pre-set marking pattern on the printing medium surface is used to improve the initialization accuracy of single-line printing positioning. One method of setting the marking pattern is to use two sets of equally spaced, mutually perpendicular straight lines; ensuring that during the printing process, a single frame image from the linear image sensor simultaneously covers several lines in these two sets; the distribution direction of these two sets of straight lines is set at a 45-degree angle to the printing movement direction; the coordinates of the pre-set marking line meridian cluster in the printing medium coordinate system are... The coordinates of the latitude cluster are Adjacent meridians or parallels of latitude are spaced equally apart, and meridians and parallels of latitude are perpendicular to each other.

16. The method for improving image stitching accuracy and positioning accuracy of a portable printer / copier according to claim 1, characterized in that: In step 4, a preset marking pattern is used to improve the positioning initialization accuracy of single-line printing. One positioning initialization method utilizes a single-frame image from a contact image sensor. The position and orientation of the contact image sensor are confined to a discrete state space by imaging the marking lines. Then, the best match is selected from these discrete states as the positioning result using the imaging of the portion outside the marking lines on the printing medium. The specific steps are as follows: Step 4.1a: During the image printing process, the distortion-free marker line image scanned by the contact image sensor and the printed image at the same moment are overlapped using the extrinsic relationship between the contact image sensor and the print head to generate a combined image for subsequent positioning. Step 4.2a: Use a contact image sensor to acquire a single frame image covering the marker line and the printed image at a certain resolution setting, and perform distortion correction processing; Step 4.3a: Extract the intersection points with the marker lines in the distorted single-frame image from step 4.2a, allowing for errors; Step 4.4a: From the adjacent intersection points extracted in Step 4.3a, select point pairs ab whose distance is less than the latitude and longitude interval; let ab be the line segment where a single frame image intersects with a square marker line, and let the latitude direction be the x-axis and the longitude direction be the y-axis; since ab is less than the latitude and longitude interval, a and b must be one on the longitude and one on the latitude; let the intersection point of a, b and the square be the latitude and longitude intersection point of the coordinate origin O, then there will be two solutions that satisfy the intersection point observation results extracted in Step 4.3a; let A and B be the intersection points of a, b and the square in one solution, and in a special case if OA=OB, the two solutions overlap; The solution process for the first solution is as follows: solution <-null for OA ∈ [0, latitude and longitude interval] For any OB, st. AB = ab and OB >= OA and OB < latitude / longitude interval. If the solution is empty, or the intersection of line AB and the latitude / longitude line is closer to the observation result than the solution: solution <- the coordinates of the intersection of line AB and the latitude and longitude lines, and the coordinates of the matching points in the distortion-free single-frame image. return solution The second solution is calculated as follows: the coordinates of the intersection points of the straight line and the latitude and longitude lines in the first solution are symmetrically transformed according to the straight line x=y, or the x and y coordinate values ​​of the intersection points are swapped to obtain the coordinates of the intersection points in the second solution. The coordinates of the matching points in the distorted single-frame image remain unchanged. Step 4.5a: A square has 4 vertices. Using each vertex, construct two solutions that satisfy the intersection distribution of the marker lines extracted in Step 4.3a. Therefore, a total of 8 solutions can be constructed using ab and a square; among them... The coordinates of the intersection points of the straight line and the latitude and longitude lines in the third and fourth solutions are obtained by performing a symmetrical transformation on the intersection point coordinates in the first and second solutions based on the straight line x = latitude and longitude line interval / 2. The coordinates of the matching points in the distortion-free single-frame image remain unchanged. The coordinates of the intersection points of the straight line and the latitude and longitude lines in the fifth and sixth solutions are obtained by performing a symmetrical transformation on the intersection point coordinates in the first and second solutions based on the straight line y = latitude and longitude line interval / 2. The coordinates of the matching points in the distortion-free single-frame image remain unchanged. The coordinates of the intersection points of the straight line and the latitude and longitude lines in the seventh and eighth solutions are obtained by performing a symmetrical transformation on the intersection point coordinates in the third and fourth solutions based on the straight line y = latitude and longitude line interval / 2. The coordinates of the matching points in the distortion-free single-frame image remain unchanged. Step 4.6a: Repeat steps 4.4a-4.5a using other adjacent intersection points extracted in step 4.3a whose distance is less than the latitude and longitude interval to obtain more sets of feasible discrete solutions, with 8 solutions in each set; Step 4.7a: Select the set of 8 solutions with the most matching intersection points and the smallest matching error from the feasible discrete solution set established in steps 4.4a-4.6a. Then, translate the coordinates of the intersection points of the straight lines and the latitude and longitude lines in these 8 solutions along the direction of the longitude or latitude lines by the latitude and longitude line interval * n, and then rotate them by 45 degrees to obtain the intersection point coordinates of the actual feasible discrete solutions. The coordinates of the intersection points and the matching points in the distortion-free single frame image remain unchanged. Step 4.8a: Traverse all feasible solutions in Step 4.7a, use singular value decomposition to obtain the rigid transformation from the distorted single-frame image to the combined image based on the coordinates of the matching point pairs, calculate the photometric error between the distorted single-frame image and the combined image, and select the one with the smallest photometric error as the final solution.

17. The method for improving image stitching accuracy and positioning accuracy of a portable printer / copier according to claim 1, characterized in that: In step 4, the positioning initialization accuracy of single-line printing is improved by using a preset marking pattern. One method is to use a contact image sensor to scan local multi-frame images to improve the positioning initialization accuracy. The specific steps are as follows: Step 4.1b: During the image printing process, the distortion-free marker line image scanned by the contact image sensor and the printed image at the same moment are overlapped using the extrinsic relationship between the contact image sensor and the print head to generate a combined image for subsequent positioning. Step 4.2b: Use a contact image sensor to acquire local multi-frame images covering the marking line and the printed image at a certain resolution setting. Use the local motion obtained by the motion tracking sensor and the imaging model of the contact image sensor to stitch the multi-frame images to generate a distortion-free scan image. Step 4.3b: Extract line features from the scanned image generated in step 4.2b; Step 4.4b: For the line feature pair formed by the two mutually perpendicular line features extracted in Step 4.3b, consider the case where they overlap with a set of latitude and longitude lines respectively. Let the latitude coordinates be... The coordinates of the meridian are Construct one matching point on each of the longitude and latitude lines; since there are a total of 4 feasible matching relationships, 4 feasible solutions are obtained through a single line feature pair; Step 4.5b: Shift the matching points on the longitude and latitude lines in the four feasible solutions from Step 4.4b along the direction of the longitude or latitude line by the longitude and latitude line interval *n, and then rotate them by 45 degrees to obtain more feasible discrete solutions; Step 4.6b: Repeat steps 4.4b-4.5b using the other two perpendicular line features extracted in step 4.3b to obtain more feasible discrete solutions; Step 4.7b: Traverse all feasible discrete solutions established in steps 4.4b-4.6b, use singular value decomposition to obtain the rigid transformation from the distorted image to the combined image based on the coordinates of the established matching point pairs, calculate the photometric error between the distorted image and the combined image, and select the one with the smallest photometric error as the final solution.

Images