Scanning image splicing method and scanning pen

Abstract

The invention discloses a scanning image splicing method and a scanning pen, and the method comprises the steps: obtaining a scanning image imgsrc frame by frame; performing binarization processing on the scanning image imgsrc to obtain a binarized image imgbin; image matching is carried out on the binarized image imgbinCurr of the current frame of scanning image imgsrcCurr and the binarized image imgbinPre of the previous frame of scanning image imgsrcPre, and coordinates of a splicing point are determined; and according to the coordinate of the splicing point, splicing the current frame of scanning image imgsrcCurr and the previous frame of scanning image imgsrcPre, and obtaining a spliced image imgSetch. According to the scanning image splicing method, image matching is carried out based on the binary image imgbin, the image features of the binary image imgbin are fewer, the data processing amount in the matching process is greatly reduced, the difficulty of scanning image splicing is reduced, the hardware requirement is greatly reduced, the memory requirement is small, and the processing speed is high.

Description

Technical Field

[0001] The invention relates to the technical field of scanning image processing, and specifically provides a scanning image splicing method and a scanning pen. Background Art

[0002] As a widely used intelligent learning hardware, the smart scanning pen can scan and recognize text and display the scanned and recognized text content on the display screen.

[0003] Existing scanning pens need to stitch continuously scanned images together and then perform OCR recognition on the stitched images. Currently, the commonly used scan image stitching technologies mainly include extracting image features, matching and stitching the previous and next two frames of images, or using a deep model. Regardless of the method, the hardware requirements for the scanning device are extremely high to meet the processing power requirements for scan image stitching.

[0004] Since the existing scanning pens have high hardware requirements, the hardware cost of the scanning pens is increased to a certain extent. In addition, the scanning pens also need to perform OCR recognition and screen display functions, which further increases the hardware cost of the scanning pens.

[0005] In view of this, in order to reduce the complexity of the scanned image stitching technology and thus reduce the hardware cost of the scanning equipment, the present patent is specially proposed. Summary of the invention

[0006] In view of the above technical problems, the present invention proposes a scan image stitching method and a scanning pen. On the basis of ensuring the scan image stitching accuracy, the scan image stitching method is simpler and has lower hardware requirements, thereby reducing the hardware cost of the scanning device.

[0007] Specifically, the following technical solutions are adopted:

[0008] In a first aspect, the present invention provides a scanned image stitching method, comprising:

[0009] Get the scanned image imgsrc frame by frame;

[0010] The scanned image imgsrc is binarized to obtain a binary image imgbin;

[0011] Perform image matching on the binary image imgbinCurr of the current frame scanned image imgsrcCurr and the binary image imgbinPre of the previous frame scanned image imgsrcPre to determine the coordinates of the splicing points;

[0012] According to the stitching point coordinates, the current frame scanned image imgsrcCurr is stitched with the previous frame scanned image imgsrcPre to obtain a stitched image imgSitch.

[0013] As an optional embodiment of the present invention, in a scanned image stitching method of the present invention, performing image matching on a binary image imgbinCurr of a current frame scanned image imgsrcCurr and a binary image imgbinPre of a previous frame scanned image imgsrcPre to determine the coordinates of the stitching points includes:

[0014] Traverse the connected domain information of the current frame binary image imgbinCurr and the previous frame binary image imgbinPre;

[0015] Match a connected domain circumscribed rectangle in the previous frame binary image imgbinPre with a connected domain circumscribed rectangle in the current frame binary image imgbinCurr, respectively use the four vertices of the connected domain circumscribed rectangle in the previous frame binary image imgbinPre as alignment points to match with the connected domain circumscribed rectangle in the current frame binary image imgbinCurr, perform XOR operation on the pixel values ​​in the corresponding ROI area of ​​the previous frame binary image imgbinPre and the current frame binary image imgbinCurr, and obtain the difference pixel point sets cnt1, cnt2, cnt3, cnt4 with the four vertices as alignment points respectively;

[0016] Repeat the above steps until each connected domain circumscribed rectangle in the previous frame binary image imgbinPre is matched one by one with each connected domain circumscribed rectangle in the current frame binary image imgbinCurr, and obtain the coordinates of the splicing point of the minimum difference pixel point set after traversal matching.

[0017] As an optional embodiment of the present invention, in a scanned image stitching method of the present invention, the XOR operation processing of pixel values ​​in the corresponding ROI area for the previous frame binary image imgbinPre and the current frame binary image imgbinCurr includes:

[0018] Obtain the minimum bounding rectangle width min_rect_w and the minimum bounding rectangle height min_rect_h according to the connected domain bounding rectangle in the previous frame binary image imgbinPre and the connected domain bounding rectangle in the current frame binary image imgbinCurr;

[0019] When the four vertices a, b, c, d of the connected domain circumscribed rectangle in the previous frame binary image imgbinPre are used as alignment points to match the connected domain circumscribed rectangle in the current frame binary image imgbinCurr, the RIO areas (a_x, a_y, min_rect_w, min_rect_h), (b_x, b_y, min_rect_w, min_rect_h), (c_x, c_y, min_rect_w, min_rect_h), (d_x, d_y, min_rect_w, min_rect_h) are determined by the four vertices and the minimum circumscribed rectangle width min_rect_w and the minimum circumscribed rectangle height min_rect_h, respectively, where (a_x, a_y), (b_x, b_y), (c_x, c_y), (d_x, d_y) are the coordinates of the four vertices a, b, c, d of the connected domain circumscribed rectangle;

[0020] The four vertices a, b, c, and d of the connected domain circumscribed rectangle in the previous frame binary image imgbinPre are used as alignment points and matched with the connected domain circumscribed rectangle in the current frame binary image imgbinCurr, and the XOR operation of pixel values ​​is performed in the corresponding ROI areas (a_x, a_y, min_rect_w, min_rect_h), (b_x, b_y, min_rect_w, min_rect_h), (c_x, c_y, min_rect_w, min_rect_h), and (d_x, d_y, min_rect_w, min_rect_h).

[0021] As an optional embodiment of the present invention, a scanned image stitching method of the present invention includes:

[0022] The difference pixel point sets cnt1, cnt2, cnt3, cnt4 obtained by matching a connected domain circumscribed rectangle in the previous frame binary image imgbinPre with a connected domain circumscribed rectangle in the current frame binary image imgbinCurr are normalized by (min_rect_w*min_rect_h) to obtain the minimum difference pixel point set minCnt1 for this connected domain circumscribed rectangle matching;

[0023] Traverse and match the connected domain external rectangles of the current frame binary image imgbinCurr and the previous frame binary image imgbinPre to obtain the minimum difference pixel point sets minCnt2, ..., minCntn respectively;

[0024] Compare and select the alignment point coordinates corresponding to the minimum difference pixel set among minCnt1, minCnt2, ..., minCntn as the splicing point coordinates;

[0025] Optionally, after obtaining the coordinates of the splicing point, the method includes:

[0026] The horizontal offset of the current scanned image imgsrcCurr relative to the previous scanned image imgsrcPre is obtained according to the coordinates of the stitching points, that is, the horizontal offset of the current scanned image stitching;

[0027] The horizontal offset of the current scanned image splicing is compared with the horizontal offset of the last scanned image splicing. If the difference between the two is greater than the preset horizontal offset threshold, the splicing point coordinates determined by the current image matching are unreasonable and are discarded.

[0028] As an optional embodiment of the present invention, a scanned image stitching method of the present invention, after traversing the connected domain information of the current frame binary image imgbinCurr and the previous frame binary image imgbinPre, comprises:

[0029] If there is no connected domain in the previous frame scan image imgsrcPre, the coordinates of the splicing point are forced to (0, 0), that is, the current frame scan image imgsrcCurr completely covers the previous frame scan image imgsrcPre;

[0030] If there is no connected domain in the current frame binary image imgbinCurr, the splicing of the current frame is skipped.

[0031] As an optional embodiment of the present invention, a scanned image stitching method of the present invention, after traversing the connected domain information of the current frame binary image imgbinCurr and the previous frame binary image imgbinPre, comprises:

[0032] If the number of connected domains in the previous frame scanned image imgsrcPre exceeds the preset maximum number threshold of connected domains, and the bounding rectangles of each connected domain in the previous frame binary image imgbinPre do not match the coordinates of the splicing points with the bounding rectangles of each connected domain in the current frame binary image imgbinCurr, then the coordinates of the splicing points are forced to (0, 0), that is, the current frame scanned image imgsrcCurr completely covers the previous frame scanned image imgsrcPre;

[0033] If the number of connected domains in the current frame binary image imgbinCurr exceeds the preset maximum number threshold of connected domains, and the bounding rectangles of each connected domain in the previous frame binary image imgbinPre do not match the coordinates of the splicing points with the bounding rectangles of each connected domain in the current frame binary image imgbinCurr, the splicing of the current frame is skipped.

[0034] As an optional embodiment of the present invention, a scanned image stitching method of the present invention includes:

[0035] If the number of connected domains in the previous frame scanned image imgsrcPre and the number of connected domains in the current frame binary image imgbinCurr do not exceed the preset maximum number threshold of connected domains, and the bounding rectangles of each connected domain in the previous frame binary image imgbinPre and the bounding rectangles of each connected domain in the current frame binary image imgbinCurr cannot match the coordinates of the splicing points;

[0036] Then the preset horizontal offset threshold is enlarged, and the image matching of the binary image imgbinCurr of the current frame scan image imgsrcCurr and the binary image imgbinPre of the previous frame scan image imgsrcPre is repeated;

[0037] If the image matching is repeated for a number of times reaching the preset maximum matching number threshold, and the coordinates of the splicing point are still not obtained, it is determined whether the horizontal offset diffX1 of the last scanned image splicing is 0;

[0038] If the judgment result is yes, the coordinates of the splicing point are forced to (0, 0), that is, the current frame scan image imgsrcCurr completely covers the previous frame scan image imgsrcPre. If the judgment result is no, the coordinates of the splicing point are forced to (diffX1, 0).

[0039] As an optional embodiment of the present invention, in a scanned image stitching method of the present invention, stitching the current frame scanned image imgsrcCurr with the previous frame scanned image imgsrcPre according to the stitching point coordinates to obtain the stitched image imgSitch includes:

[0040] According to the width and the coordinates of the stitched image obtained by stitching the previous scanned image imgsrcPre, the stitching point of the current scanned image imgsrcCurr on the stitched image is calculated, and the horizontal offset diffX2 of the current scanned image imgsrcCurr relative to the previous scanned image imgsrcPre is obtained;

[0041] Calculate the overlapping range th=w-diffX2 between the current frame scanned image imgsrcCurr and the previous frame scanned image imgsrcPre;

[0042] When the current frame scan image imgsrcCurr is spliced ​​with the previous frame scan image imgsrcPre, along the X-axis direction of the current frame scan image imgsrcCurr, it is determined whether the area with a distance x from the splicing point along the X-axis direction satisfies x<th;

[0043] If the judgment result is yes, the corresponding area is pixel-fused according to the formula imgSitch=(imgSitch*(th-x)+imgCurr*x) / th. If the judgment result is no, the corresponding area of ​​the current frame scan image imgsrcCurr is overwritten.

[0044] As an optional embodiment of the present invention, in a scanned image stitching method of the present invention, the binarization processing of the scanned image imgsrc to obtain the binarized image imgbin includes:

[0045] The radius R is set according to the image width w and image height h of the scanned image imgsrc;

[0046] In the scanned image imgsrc, take the maximum pixel value imgmax within the radius R, and calculate the new pixel value imgtem by imgtem=abs(imgcurr-imgmax) for each pixel value within the radius R, where abs is the absolute value function;

[0047] The new pixel values ​​imgtem in the scanned image imgsrc are binarized using the Otsu method to obtain a binary image imgbin;

[0048] Optionally, the step of acquiring the scanned image imgsrc frame by frame includes:

[0049] According to the image width w, image height h, and scanning angle of the scanned image imgsrc, a perspective transformation mapping matrix is ​​generated in advance, and the scanned image imgsrc is used to generate a corrected scanned image through the mapping matrix, and the binarization processing is for the corrected scanned image;

[0050] Optionally, a binary threshold thresh is obtained by using the Otsu method for the corrected scanned image, and the number of pixel points less than the binary threshold thresh is counted. If it is less than w*h*η (η is a preset value, 0.5<η<1), the pixel does not need to be inverted, otherwise it is inverted;

[0051] Optionally, binarizing the scanned image imgsrc to obtain a binarized image imgbin includes:

[0052] Generate an image scaling mapping matrix in advance, new_w = w / N, new_h = h / N, where N is a setting value set by referring to the image width w and image height h of the scanned image imgsrc, and N>1;

[0053] The binary image imgbin is transformed into a new binary image imgbin1 through an image scaling mapping matrix.

[0054] In a second aspect, the present invention provides a scanning pen using the scanned image stitching method.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] A scanned image splicing method of the present invention performs binarization processing on a scanned image imgsrc, performs image matching based on the features of the binarized image imgbin after the binarization processing, obtains an optimal splicing point, and realizes the splicing of two frames of scanned images. A scanned image splicing method of the present invention performs image matching based on the binarized image imgbin, and the image features of the binarized image imgbin are fewer, so the data processing amount of the matching process is greatly reduced, the difficulty of scanned image splicing is reduced, the hardware requirements are greatly reduced, the memory requirements are small, and the processing speed is fast.

[0057] The present invention provides a scanning pen using the scanned image splicing method, which can reduce hardware requirements, thereby greatly reducing the cost of the scanning pen. In addition, the scanning pen of the present invention can be used as an accessory, without a display screen, and only transmits the scanned spliced ​​image to a smart device such as a mobile phone or a pad via Bluetooth / WIFI, and an app of another existing smart device performs various processing on the collected image, such as OCR; in this way, the scanning pen can adopt a more low-cost hardware solution and can improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A flow chart of a scanned image stitching method according to an embodiment of the present invention;

[0059] Figure 2 An example diagram of a previous frame binary image imgbinPre and a current frame binary image imgbinCurr according to an embodiment of the present invention;

[0060] Figure 3 An example diagram of the circumscribed rectangle of the connected domain in the previous frame binary image imgbinPre and the current frame binary image imgbinCurr according to the embodiment of the present invention;

[0061] Figure 4 An example diagram of four aligned vertices of a circumscribed rectangle of a connected domain according to an embodiment of the present invention;

[0062] Figure 5 A scanning image effect diagram obtained by stitching a scanning image stitching method according to the present embodiment;

[0063] Figure 6 A schematic structural diagram of an electronic device according to an embodiment of the present invention;

[0064] Figure 7 Schematic diagram of a computer-readable recording medium according to an embodiment of the present invention. DETAILED DESCRIPTION

[0065] To make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be described clearly and completely in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them.

[0066] Therefore, the following detailed description of the embodiments of the present invention is not intended to limit the scope of the invention claimed for protection, but merely represents some embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features and technical solutions in the embodiments may be combined with each other.

[0068] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0069] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "upper", "lower", etc. is based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the invention product is usually placed when in use, or the orientation or positional relationship commonly understood by those skilled in the art. Such terms are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0070] Embodiment 1

[0071] This embodiment provides a scanned image stitching method, including:

[0072] Get the scanned image imgsrc frame by frame;

[0073] The scanned image imgsrc is binarized to obtain a binary image imgbin;

[0074] Perform image matching on the binary image imgbinCurr of the current frame scanned image imgsrcCurr and the binary image imgbinPre of the previous frame scanned image imgsrcPre to determine the coordinates of the splicing points;

[0075] According to the stitching point coordinates, the current frame scanned image imgsrcCurr is stitched with the previous frame scanned image imgsrcPre to obtain a stitched image imgSitch.

[0076] A scanned image stitching method of the present embodiment performs binarization processing on the scanned image imgsrc, performs image matching based on the features of the binarized image imgbin after the binarization processing, obtains the optimal stitching point, and realizes the stitching of the two scanned images before and after. A scanned image stitching method of the present embodiment performs image matching based on the binarized image imgbin, and the image features of the binarized image imgbin are fewer, so the data processing amount of the matching process is greatly reduced, the difficulty of scanned image stitching is reduced, the hardware requirements are greatly reduced, the memory requirements are small, and the processing speed is fast.

[0077] As an optional implementation of this embodiment, in a scanned image stitching method of this embodiment, performing image matching on a binary image imgbinCurr of a current frame scanned image imgsrcCurr and a binary image imgbinPre of a previous frame scanned image imgsrcPre to determine the coordinates of the stitching points includes:

[0078] Traverse the connected domain information of the current frame binary image imgbinCurr and the previous frame binary image imgbinPre;

[0079] Match a connected domain circumscribed rectangle in the previous frame binary image imgbinPre with a connected domain circumscribed rectangle in the current frame binary image imgbinCurr, respectively use the four vertices of the connected domain circumscribed rectangle in the previous frame binary image imgbinPre as alignment points to match with the connected domain circumscribed rectangle in the current frame binary image imgbinCurr, perform XOR operation on the pixel values ​​in the corresponding ROI area of ​​the previous frame binary image imgbinPre and the current frame binary image imgbinCurr, and obtain the difference pixel point sets cnt1, cnt2, cnt3, cnt4 with the four vertices as alignment points respectively;

[0080] Repeat the above steps until each connected domain circumscribed rectangle in the previous frame binary image imgbinPre is matched one by one with each connected domain circumscribed rectangle in the current frame binary image imgbinCurr, and obtain the coordinates of the splicing point of the minimum difference pixel point set after traversal matching.

[0081] This embodiment traverses the current frame binary image imgbinCurr to obtain m connected domains, and traverses the previous frame binary image imgbinPre to obtain n connected domains. When performing image matching, the bounding rectangles of the m connected domains in the current frame binary image imgbinCurr are matched one by one with the bounding rectangles of the n connected domains in the previous frame binary image imgbinPre. When matching the bounding rectangles of every two connected domains, four vertices are used as alignment points. For example, the four vertices of the bounding rectangle of the connected domain 1 are (a1, b1, c1, d1), and the four vertices of the bounding rectangle of the connected domain 2 are (a2, b2, c2, d2). The alignment results include (a1, a2), (a1, b2), (a1, c2), (a1, d2), so the four difference pixel point sets cnt1, cnt2, cnt3, and cnt4 can be obtained by matching the bounding rectangles of every two connected domains. In this way, the bounding rectangles of the m connected domains in the current frame binary image imgbinCurr are matched one by one with the bounding rectangles of the n connected domains in the previous frame binary image imgbinPre, and m*n*4 sets of difference pixel points can be obtained. The coordinates of the alignment points of the bounding rectangles of the connected domains corresponding to the least set of difference pixel points are determined as the coordinates of the splicing points.

[0082] In a scanned image stitching method of this embodiment, performing an XOR operation on pixel values ​​of a previous frame binary image imgbinPre and a current frame binary image imgbinCurr in a corresponding ROI region includes:

[0083] Obtain the minimum bounding rectangle width min_rect_w and the minimum bounding rectangle height min_rect_h according to the connected domain bounding rectangle in the previous frame binary image imgbinPre and the connected domain bounding rectangle in the current frame binary image imgbinCurr;

[0084] When using the four vertices a, b, c, and d of the bounding rectangle of the connected component in the previous frame's binary image imgbinPre as alignment points to match the bounding rectangle of the connected component in the current frame's binary image imgbinCurr, the RIO regions (a_x, a_y, min_rect_w, min_rect_h), (b_x, b_y, min_rect_w, min_rect_h), (c_x, c_y, min_rect_w, min_rect_h), and (d_x, d_y, min_rect_w, min_rect_h) are determined respectively by the four vertices and the minimum bounding rectangle width min_rect_w and the minimum bounding rectangle height min_rect_h, where (a_x, a_y), (b_x, b_y), (c_x, c_y), and (d_x, d_y) are the coordinates of the four vertices a, b, c, and d of the bounding rectangle of the connected component.

[0085] Regarding the four vertices a, b, c, and d of the bounding rectangle of the connected component in the previous frame's binary image imgbinPre as alignment points to match the bounding rectangle of the connected component in the current frame's binary image imgbinCurr, exclusive OR operation processing of pixel values is performed in the corresponding ROI regions (a_x, a_y, min_rect_w, min_rect_h), (b_x, b_y, min_rect_w, min_rect_h), (c_x, c_y, min_rect_w, min_rect_h), and (d_x, d_y, min_rect_w, min_rect_h).

[0086] For specific examples, see Figure 2-Figure 4 as shown Figure 2 is a schematic of the previous frame's binary image imgbinPre and the current frame's binary image imgbinCurr. Figure 3 The rectangular outer frame outside the character "Tang" in it is the bounding rectangle of the connected component. Figure 4 In it, a, b, c, and d are the four alignment vertices of the bounding rectangle of the connected component.

[0087] Furthermore, a method for stitching scanned images in this embodiment includes:

[0088] For the sets of different pixel points cnt1, cnt2, cnt3, and cnt4 obtained by matching the bounding rectangle of a certain connected component in the previous frame's binary image imgbinPre with the bounding rectangle of a certain connected component in the current frame's binary image imgbinCurr, divide by (min_rect_w * min_rect_h) for normalization processing to obtain the set of the fewest different pixel points minCnt1 for this matching of the bounding rectangle of the connected component.

[0089] Traverse and match the connected domain external rectangles of the current frame binary image imgbinCurr and the previous frame binary image imgbinPre to obtain the minimum difference pixel point sets minCnt2, ..., minCntn respectively;

[0090] The coordinates of the alignment points corresponding to the minimum difference pixel set among minCnt1, minCnt2, ..., minCntn are compared and selected as the stitching point coordinates.

[0091] Optionally, a scanned image stitching method of this embodiment, after obtaining the stitching point coordinates, includes:

[0092] The horizontal offset of the current scanned image imgsrcCurr relative to the previous scanned image imgsrcPre is obtained according to the coordinates of the stitching points, that is, the horizontal offset of the current scanned image stitching;

[0093] The horizontal offset of the current scanned image splicing is compared with the horizontal offset of the last scanned image splicing. If the difference between the two is greater than the preset horizontal offset threshold, the splicing point coordinates determined by the current image matching are unreasonable and are discarded.

[0094] For example, the coordinates of the last best matching splicing point moved 4 pixels, and this time it moved 20 pixels. Even if the difference in pixels is the smallest, it is still an unreasonable match. The coordinates of the splicing point should be discarded, and image matching should be performed again, or the splicing of the current frame scan image imgsrcCurr should be abandoned.

[0095] As an optional implementation of this embodiment, a scanned image stitching method of this embodiment needs to correct the image stitching points during the image matching process. Specifically, after traversing the connected domain information of the current frame binary image imgbinCurr and the previous frame binary image imgbinPre, the method includes:

[0096] If there is no connected domain in the previous frame scan image imgsrcPre, the coordinates of the splicing point are forced to (0, 0), that is, the current frame scan image imgsrcCurr completely covers the previous frame scan image imgsrcPre;

[0097] If there is no connected domain in the current frame binary image imgbinCurr, the splicing of the current frame is skipped.

[0098] In a scanned image stitching method of this embodiment, after traversing the connected domain information of the current frame binary image imgbinCurr and the previous frame binary image imgbinPre, the method includes:

[0099] If the number of connected domains in the previous frame scanned image imgsrcPre exceeds the preset maximum number threshold of connected domains, and the bounding rectangles of each connected domain in the previous frame binary image imgbinPre do not match the coordinates of the splicing points with the bounding rectangles of each connected domain in the current frame binary image imgbinCurr, then the coordinates of the splicing points are forced to (0, 0), that is, the current frame scanned image imgsrcCurr completely covers the previous frame scanned image imgsrcPre;

[0100] If the number of connected domains in the current frame binary image imgbinCurr exceeds the preset maximum number threshold of connected domains, and the bounding rectangles of each connected domain in the previous frame binary image imgbinPre do not match the coordinates of the splicing points with the bounding rectangles of each connected domain in the current frame binary image imgbinCurr, the splicing of the current frame is skipped.

[0101] A scanned image stitching method of this embodiment includes:

[0102] If the number of connected domains in the previous frame scanned image imgsrcPre and the number of connected domains in the current frame binary image imgbinCurr do not exceed the preset maximum number threshold of connected domains, and the bounding rectangles of each connected domain in the previous frame binary image imgbinPre and the bounding rectangles of each connected domain in the current frame binary image imgbinCurr cannot match the coordinates of the splicing points;

[0103] Then the preset horizontal offset threshold is enlarged, and the image matching of the binary image imgbinCurr of the current frame scan image imgsrcCurr and the binary image imgbinPre of the previous frame scan image imgsrcPre is repeated;

[0104] If the image matching is repeated for a number of times and reaches the preset maximum matching number threshold, and the coordinates of the splicing point are still not obtained, it is determined whether the horizontal offset diffX1 of the last scanned image splicing is 0;

[0105] If the judgment result is yes, the coordinates of the splicing point are forced to (0, 0), that is, the current frame scan image imgsrcCurr completely covers the previous frame scan image imgsrcPre. If the judgment result is no, the coordinates of the splicing point are forced to (diffX1, 0).

[0106] Therefore, a scanned image stitching method of this embodiment can greatly improve the speed of image stitching and the processing efficiency of image stitching by correcting image stitching points during image matching.

[0107] In the scanned image stitching method of this embodiment, the current frame scanned image imgsrcCurr is stitched with the previous frame scanned image imgsrcPre according to the stitching point coordinates to obtain the stitched image imgSitch, which includes:

[0108] According to the width and the coordinates of the stitched image obtained by stitching the previous scanned image imgsrcPre, the stitching point of the current scanned image imgsrcCurr on the stitched image is calculated, and the horizontal offset diffX2 of the current scanned image imgsrcCurr relative to the previous scanned image imgsrcPre is obtained;

[0109] Calculate the overlapping range th=w-diffX2 between the current frame scanned image imgsrcCurr and the previous frame scanned image imgsrcPre;

[0110] When the current frame scan image imgsrcCurr is spliced ​​with the previous frame scan image imgsrcPre, along the X-axis direction of the current frame scan image imgsrcCurr, it is determined whether the area with a distance x from the splicing point along the X-axis direction satisfies x<th;

[0111] If the judgment result is yes, the corresponding area is pixel-fused according to the formula imgSitch=(imgSitch*(th-x)+imgCurr*x) / th. If the judgment result is no, the corresponding area of ​​the current frame scan image imgsrcCurr is overwritten.

[0112] A scanned image stitching method of this embodiment, when stitching the current frame scanned image imgsrcCurr with the previous frame scanned image imgsrcPre, performs pixel fusion on the overlapping part according to the formula imgSitch=(imgSitch*(th-x)+imgCurr*x) / th to achieve transition stitching of pixel points. The resulting stitched image has natural pixel transition and better stitching effect.

[0113] In a scanned image stitching method of this embodiment, the binarization process for the scanned image imgsrc to obtain the binarized image imgbin includes:

[0114] The radius R is set according to the image width w and image height h of the scanned image imgsrc;

[0115] In the scanned image imgsrc, take the maximum pixel value imgmax within the radius R, and calculate the new pixel value imgtem by imgtem=abs(imgcurr-imgmax) for each pixel value within the radius R, where abs is the absolute value function;

[0116] Each new pixel value imgtem in the scanned image imgsrc is binarized using the Otsu method to obtain a binary image imgbin.

[0117] Since the scanned image imgsrc is not uniform and may have shadows, this embodiment uses the above-mentioned method of binarizing the scanned image imgsrc. The background and text gradients are large, imgmax is the maximum pixel value, and after the abs(imgcurr-imgmax) operation, the background approaches 0, the text area gradient is large, and the difference is relatively large.

[0118] In addition, if there are hardware limitations, the pixel skipping method can be used to speed up the pixel traversal processing. For example, the maximum pixel value imgmax in the 2*2 range is only calculated once, and the four values ​​are equal.

[0119] There are many binary algorithms, and different algorithms can be used according to actual conditions. For example, if the difference in the captured image is caused by the difference in hardware structure, a faster binary algorithm may be used. In a scanned image stitching method of this embodiment, the quality of the image after binarization is more important, and the difference between the binary images of the previous and next frames is kept as small as possible.

[0120] Optionally, in a scanned image stitching method of this embodiment, the step of acquiring the scanned image imgsrc frame by frame includes:

[0121] According to the image width w, image height h, and scanning angle of the scanned image imgsrc, a perspective transformation mapping matrix is ​​generated in advance, and the scanned image imgsrc generates a corrected scanned image through the mapping matrix. The binarization process is for the corrected scanned image. Due to the limitation of the scanning angle, the directly obtained scanned image imgsrc may have an angle deviation, so correction processing is required.

[0122] Optionally, a scanned image stitching method of this embodiment includes: obtaining a binary threshold thresh by the Otsu method for the corrected scanned image, counting the number of pixel points less than the binary threshold thresh, if it is less than w*h*η (η is a preset value, 0.5<η<1), then the pixel does not need to be inverted, otherwise it is inverted. For example, counting the number of pixel points less than thresh, if it is less than w*h*55%, then the pixel does not need to be inverted, otherwise it is inverted, imgcurr=255-imgcurr, which can reduce the number of processed pixels, reduce the processing amount, and improve processing efficiency.

[0123] Optionally, a scanned image stitching method of this embodiment, after binarizing the scanned image imgsrc to obtain a binarized image imgbin, includes:

[0124] Generate an image scaling mapping matrix in advance, new_w = w / N, new_h = h / N, where N is a setting value set by referring to the image width w and image height h of the scanned image imgsrc, and N>1;

[0125] The binary image imgbin is transformed into a new binary image imgbin1 through an image scaling mapping matrix.

[0126] For example, if you need to detect the stitching points faster and use smaller memory, you can generate the image scaling mapping matrix in advance, new_w=w / 2, new_h=h / 2; imgbin obtains the new binary image imgbin1 through the mapping matrix (because it is already a binary image, if interpolation is used for scaling, it will become a grayscale image, so the value is directly taken according to the coordinate original image. Therefore, the correspondence between the scaled image coordinates and the original image coordinates can be generated in advance, that is, the mapping matrix, so that the pixel value can be directly obtained from the original image); if the original image width and height w and h are too small, reducing them by half will affect the detection accuracy of the stitching points. You can choose according to the actual situation.

[0127] In addition, if image scaling is performed, the stitching point coordinates obtained during image matching need to be multiplied by N (such as N=2 in the specific example) to obtain the stitching point coordinates of the original image. Stitching is performed at the original image size to prevent the image from being too small to affect OCR.

[0128] In a scanned image stitching method of this embodiment, the above-mentioned image scaling process is optional. When the size of the acquired image is appropriate, this operation can reduce the use of memory and also improve the running speed. The final stitching of the original image does not affect the OCR recognition rate.

[0129] In a scanned image stitching method of the present embodiment, a perspective transformation mapping matrix and a scaling mapping matrix are generated in advance, and the mapping matrix is ​​directly placed in the code. This can reduce algorithm operations during execution and directly map the matrix to the target, which is the fastest. Of course, part of the memory is occupied to store the matrix data, but these two steps obviously prioritize speed and do not occupy much memory.

[0130] A scanned image stitching method of this embodiment includes: after stitching the current frame scanned image imgsrcCurr with the previous frame scanned image imgsrcPre, the data needs to be retained, imgbinPre=imgbinCurr; that is, the content of the current frame will become the previous frame in the next stitching.

[0131] See also Figure 5 , which is a diagram showing the effect of a scanned image spliced ​​using a scanned image splicing method according to this embodiment.

[0132] This embodiment also provides a scanning pen that adopts the scanned image stitching method, which can reduce hardware requirements and thus greatly reduce the cost of the scanning pen. In addition, the scanning pen of this embodiment can be used as an accessory, without a display screen, and only transmits the scanned stitched image to a smart device such as a mobile phone or a pad via Bluetooth / WIFI, and an app of another existing smart device performs various processing on the collected image, such as OCR; in this way, the scanning pen can adopt a more low-cost hardware solution and can improve the user experience.

[0133] Of course, the scanned image stitching method of this embodiment is not limited to the low-profile scanning pen solution, and can be used for scanning pen puzzles.

[0134] Embodiment 2

[0135] The following describes an electronic device embodiment of the present invention, which can be regarded as a specific physical implementation of the method and device embodiments of the present invention. The details described in the electronic device embodiment of the present invention should be regarded as a supplement to the above method or device embodiments; details not disclosed in the electronic device embodiment of the present invention can be implemented with reference to the above method or device embodiments.

[0136] Figure 6 It is a structural schematic diagram of an electronic device according to an embodiment of the present invention. The electronic device includes a processor and a memory. The memory is used to store a computer executable program. When the computer program is executed by the processor, the processor executes a scanned image stitching method according to the first embodiment.

[0137] like Figure 6 As shown, the electronic device is presented in the form of a general computing device. The processor may be one or more and work in coordination. The present invention does not exclude distributed processing, that is, the processor may be dispersed in different physical devices. The electronic device of the present invention is not limited to a single entity, but may also be the sum of multiple physical devices.

[0138] The memory stores a computer executable program, which is usually a machine-readable code. The computer-readable program can be executed by the processor to enable the electronic device to perform the method of the present invention, or at least part of the steps in the method.

[0139] The memory includes a volatile memory, such as a random access memory unit (RAM) and / or a cache memory unit, and may also be a non-volatile memory, such as a read-only memory unit (ROM).

[0140] Optionally, in this embodiment, the electronic device further includes an I / O interface, which is used for the electronic device to exchange data with an external device. The I / O interface can represent one or more of several types of bus structures, including a storage unit bus or a storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of a variety of bus structures.

[0141] It should be understood that Figure 6 The electronic device shown is only an example of the present invention, and the electronic device of the present invention may also include elements or components not shown in the above examples. For example, some electronic devices also include display units such as display screens, and some electronic devices also include human-computer interaction elements such as buttons, keyboards, etc. As long as the electronic device can execute the computer-readable program in the memory to implement the method of the present invention or at least part of the steps of the method, it can be considered as an electronic device covered by the present invention.

[0142] Figure 7 Schematic diagram of a computer readable recording medium according to an embodiment of the present invention. Figure 7 As shown, a computer executable program is stored in a computer readable recording medium, and when the computer executable program is executed, a scanning image stitching method of the first embodiment of the present invention is implemented. The computer readable recording medium may include a data signal propagated in a baseband or as part of a carrier, which carries a readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable recording medium may also be any readable medium other than a readable recording medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, device, or device. The program code contained on the readable recording medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0143] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0144] Through the above description of the implementation mode, it is easy for those skilled in the art to understand that the present invention can be implemented by hardware capable of executing a specific computer program, such as the system of the present invention, and the electronic processing unit, server, client, mobile phone, control unit, processor, etc. contained in the system. The present invention can also be implemented by computer software that executes the method of the present invention, such as control software executed by a microprocessor, an electronic control unit, a client, a server, etc. However, it should be noted that the computer software that executes the method of the present invention is not limited to being executed by one or a specific hardware entity, and it can also be implemented in a distributed manner by unspecified specific hardware. For computer software, the software product can be stored in a computer-readable recording medium (which can be a CD-ROM, a USB flash drive, a mobile disk, etc.), and can also be distributed and stored on the network, as long as it enables the electronic device to execute the method according to the present invention.

[0145] The above embodiments are only used to illustrate the present invention and are not intended to limit the technical solutions described in the present invention. Although the present invention has been described in detail with reference to the above embodiments, the present invention is not limited to the above specific implementation methods. Therefore, any modification or equivalent replacement of the present invention; and all technical solutions and improvements thereof that do not depart from the spirit and scope of the invention are included in the scope of the claims of the present invention.

Claims

1. A scanned image stitching method, characterized in that: include: Get the scanned image imgsrc frame by frame; The scanned image imgsrc is binarized to obtain a binary image imgbin; Perform image matching on the binary image imgbinCur r of the current frame scan image imgsrcCur r and the binary image imgbinPre of the previous frame scan image imgsrcPre to determine the coordinates of the splicing points; According to the stitching point coordinates, the current frame scanned image imgsrcCur r is stitched with the previous frame scanned image imgsrcPre to obtain a stitched image imgSitch.

2. A scanned image stitching method according to claim 1, characterized in that: The image matching of the binary image imgbinCurr of the current frame scanned image imgsrcCurr with the binary image imgbinPre of the previous frame scanned image imgsrcPre to determine the coordinates of the splicing points includes: Traverse the connected domain information of the current frame binary image imgbinCur r and the previous frame binary image imgbinPre; Match a connected domain circumscribed rectangle in the previous frame binary image imgbinPre with a connected domain circumscribed rectangle in the current frame binary image imgbinCur r, respectively use the four vertices of the connected domain circumscribed rectangle in the previous frame binary image imgbinPre as alignment points to match with the connected domain circumscribed rectangle in the current frame binary image imgbinCurr, perform XOR operation on the pixel values ​​in the corresponding ROI area of ​​the previous frame binary image imgbinPre and the current frame binary image imgbinCurr, and obtain the difference pixel point sets cnt1, cnt2, cnt3, cnt4 with the four vertices as alignment points respectively; Repeat the above steps until each connected domain circumscribed rectangle in the previous frame binary image imgbinPre is matched one by one with each connected domain circumscribed rectangle in the current frame binary image imgbinCurr, and obtain the coordinates of the splicing point of the minimum difference pixel point set after traversal matching.

3. A scanned image stitching method according to claim 2, characterized in that: The XOR operation processing of pixel values ​​in the corresponding ROI area for the previous frame binary image imgbinPre and the current frame binary image imgbinCurr includes: Obtain the minimum bounding rectangle width min_rect_w and the minimum bounding rectangle height min_rect_h according to the connected domain bounding rectangle in the previous frame binary image imgbinPre and the connected domain bounding rectangle in the current frame binary image imgbinCurr; When the four vertices a, b, c, d of the connected domain circumscribed rectangle in the previous frame binary image imgbinPre are used as alignment points to match the connected domain circumscribed rectangle in the current frame binary image imgbinCur r, the RIO areas (a_x, a_y, min_rect_w, min_rect_h), (b_x, b_y, min_rect_w, min_rect_h), (c_x, c_y, min_rect_w, min_rect_h), (d_x, d_y, min_rect_w, min_rect_h) are determined by the four vertices and the minimum circumscribed rectangle width min_rect_w and the minimum circumscribed rectangle height min_rect_h, respectively, where (a_x, a_y), (b_x, b_y), (c_x, c_y), (d_x, d_y) are the coordinates of the four vertices a, b, c, d of the connected domain circumscribed rectangle; The four vertices a, b, c, and d of the connected domain circumscribed rectangle in the previous frame binary image imgbinPre are used as alignment points and matched with the connected domain circumscribed rectangle in the current frame binary image imgbinCur r, and the XOR operation of pixel values ​​is performed in the corresponding ROI areas (a_x, a_y, min_rect_w, min_rect_h), (b_x, b_y, min_rect_w, min_rect_h), (c_x, c_y, min_rect_w, min_rect_h), and (d_x, d_y, min_rect_w, min_rect_h).

4. A scanned image stitching method according to claim 3, characterized in that: include: The difference pixel point sets cnt1, cnt2, cnt3, cnt4 obtained by matching a connected domain circumscribed rectangle in the previous frame binary image imgbinPre with a connected domain circumscribed rectangle in the current frame binary image imgbinCurr are normalized by (min_rect_w*min_rect_h) to obtain the minimum difference pixel point set minCnt1 for this connected domain circumscribed rectangle matching; Traverse and match the connected domain circumscribed rectangles of the current frame binary image imgbinCur r and the previous frame binary image imgbinPre to obtain the minimum difference pixel point sets minCnt2, ..., minCntn respectively; Compare and select the alignment point coordinates corresponding to the minimum difference pixel set among minCnt1, minCnt2, ..., minCntn as the splicing point coordinates; Optionally, after obtaining the coordinates of the splicing point, the method includes: The horizontal offset of the current scanned image imgsrcCurr relative to the previous scanned image imgsrcPre is obtained according to the coordinates of the stitching points, that is, the horizontal offset of the current scanned image stitching; The horizontal offset of the current scanned image splicing is compared with the horizontal offset of the last scanned image splicing. If the difference between the two is greater than the preset horizontal offset threshold, the splicing point coordinates determined by the current image matching are unreasonable and are discarded.

5. A scanned image stitching method according to claim 4, characterized in that: After traversing the connected domain information of the current frame binary image imgbinCurr and the previous frame binary image imgbinPre, the method includes: If there is no connected domain in the previous frame scan image imgsrcPre, the coordinates of the splicing point are forced to (0, 0), that is, the current frame scan image imgsrcCurr completely covers the previous frame scan image imgsrcPre; If there is no connected domain in the current frame binary image imgbinCurr, the splicing of the current frame is skipped.

6. A scanned image stitching method according to claim 4, characterized in that: After traversing the connected domain information of the current frame binary image imgbinCurr and the previous frame binary image imgbinPre, the method includes: If the number of connected domains in the previous frame scanned image imgsrcPre exceeds the preset maximum number threshold of connected domains, and the bounding rectangles of each connected domain in the previous frame binary image imgbinPre do not match the bounding rectangles of each connected domain in the current frame binary image imgbinCur r, the coordinates of the splicing point are forced to (0, 0), that is, the current frame scanned image imgsrcCur r completely covers the previous frame scanned image imgsrcPre; If the number of connected domains in the current frame binary image imgbinCurr exceeds the preset maximum number threshold of connected domains, and the bounding rectangles of each connected domain in the previous frame binary image imgbinPre do not match the coordinates of the splicing points with the bounding rectangles of each connected domain in the current frame binary image imgbinCurr, the splicing of the current frame is skipped.

7. A scanned image stitching method according to claim 6, characterized in that: include: If the number of connected domains in the previous frame scanned image imgsrcPre and the number of connected domains in the current frame binary image imgbinCurr do not exceed the preset maximum number threshold of connected domains, and the bounding rectangles of each connected domain in the previous frame binary image imgbinPre and the bounding rectangles of each connected domain in the current frame binary image imgbinCurr cannot match the coordinates of the splicing points; Then the preset horizontal offset threshold is enlarged, and the image matching of the binary image imgbinCurr of the current frame scan image imgsrcCur r and the binary image imgbinPre of the previous frame scan image imgsrcPre is repeated; If the image matching is repeated for a number of times reaching the preset maximum matching number threshold, and the coordinates of the splicing point are still not obtained, it is determined whether the horizontal offset diffX1 of the last scanned image splicing is 0; If the judgment result is yes, the coordinates of the splicing point are forced to (0, 0), that is, the current frame scan image imgsrcCurr completely covers the previous frame scan image imgsrcPre. If the judgment result is no, the coordinates of the splicing point are forced to (diffX1, 0).

8. A scanned image stitching method according to claim 2, characterized in that: The step of splicing the current frame scanned image imgsrcCur r with the previous frame scanned image imgsrcPre according to the splicing point coordinates to obtain the spliced ​​image imgSitch comprises: According to the width and the coordinates of the stitched image obtained by stitching the previous scanned image imgsrcPre, the stitching point of the current scanned image imgsrcCurr on the stitched image is calculated, and the horizontal offset diffX2 of the current scanned image imgsrcCurr relative to the previous scanned image imgsrcPre is obtained; Calculate the overlapping range th=w-diffX2 between the current frame scanned image imgsrcCurr and the previous frame scanned image imgsrcPre; When the current frame scan image imgsrcCurr is spliced ​​with the previous frame scan image imgsrcPre, along the X-axis direction of the current frame scan image imgsrcCurr, it is determined whether the area with a distance x from the splicing point along the X-axis direction satisfies x<th; If the judgment result is yes, the corresponding area is pixel-fused according to the formula imgSitch=(imgSitch*(th-x)+imgCur r*x) / th. If the judgment result is no, the corresponding area of ​​the current frame scan image imgsrcCur r is overwritten.

9. The scanned image stitching method according to claim 1, characterized in that: The binarization process for the scanned image imgsrc to obtain the binarized image imgbin comprises: The radius R is set according to the image width w and image height h of the scanned image imgsrc; In the scanned image imgsrc, take the maximum pixel value imgmax within the radius R, and calculate the new pixel value imgtem by imgtem=abs(imgcurr-imgmax) for each pixel value within the radius R, where abs is the absolute value function; The new pixel values ​​imgtem in the scanned image imgsrc are binarized using the Otsu method to obtain a binary image imgbin; Optionally, the step of acquiring the scanned image imgsrc frame by frame includes: According to the image width w, image height h, and scanning angle of the scanned image imgsrc, a perspective transformation mapping matrix is ​​generated in advance, and the scanned image imgsrc is used to generate a corrected scanned image through the mapping matrix, and the binarization processing is for the corrected scanned image; Optionally, a binary threshold thresh is obtained by using the Otsu method for the corrected scanned image, and the number of pixel points less than the binary threshold thresh is counted. If it is less than w*h*η (η is a preset value, 0.5<η<1), the pixel does not need to be inverted, otherwise it is inverted; Optionally, binarizing the scanned image imgsrc to obtain a binarized image imgbin includes: Generate an image scaling mapping matrix in advance, new_w = w / N, new_h = h / N, where N is a setting value set by referring to the image width w and image height h of the scanned image imgsrc, and N>1; The binary image imgbin is transformed into a new binary image imgbin1 through an image scaling mapping matrix.

10. A scanning pen using the scanned image stitching method as claimed in any one of claims 1 to 9.

Images