Image processing method of scanning equipment and dictionary pen

Abstract

The invention provides an image processing method of scanning equipment and a dictionary pen. The image processing method of the scanning equipment comprises the following steps of: partitioning each scanning image frame input in a content scanning process along the height direction of the scanning image frame to obtain at least two regions distributed along the height direction; based on the image feature of each region, calculating an image feature metric value of the region, and selecting the region with the highest image feature metric value as a target region; and on the basis of the image content of the overlapped part in the target area of each scanning image frame, determining a geometric transformation relationship between the scanning image frames, and splicing the scanning image frames according to the geometric transformation relationship. According to the image processing method of the scanning equipment, the scanned image frame is partitioned, then the image feature metric values are extracted respectively, and the area with the maximum image feature metric value is used for processing the subsequent shot image, so that the moire influence in the image is reduced, and the scanning and splicing accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image processing method for a scanning device and a dictionary pen. Background Technology

[0002] With the continuous innovation of science and technology in the field of education, dictionary pens, as a portable electronic learning tool, offer a more convenient way to look up information compared to traditional paper dictionaries and manually input electronic dictionaries.

[0003] The process of a dictionary pen from scanning to obtaining text is as follows: image capture - stitching - line segmentation - OCR recognition (i.e., text recognition). Specifically, as the scanning motion proceeds, the scanned image frames captured by the camera are continuously stitched together to form a complete scanned image. Then, the stitched image is segmented to remove redundant images, retaining only the target line. Finally, OCR recognition is performed on the segmented image containing the target line. However, in related technologies, if moiré patterns exist in the scanned image frames during the image stitching process, it will lead to inaccurate stitching positions, resulting in content loss. Furthermore, in the subsequent OCR recognition process, there may be missing words, misspelled words, and typos. As a result, the accuracy of word extraction is greatly reduced, requiring users to scan multiple times to obtain accurate words, leading to a poor user experience. Summary of the Invention

[0004] This invention provides an image processing method for a scanning device to solve the defects in the prior art where the scanning image frame is affected by moiré patterns and the stitching position is inaccurate, thereby achieving accurate stitching in moiré pattern scenes.

[0005] This invention provides an image processing method for a scanning device, comprising: Each scanned image frame entered during the content scanning process is divided along its own height direction to obtain at least two regions distributed along the height direction; Based on the image features of each region, calculate its image feature metric value, and select the region with the highest image feature metric value as the target region. Based on the image content of the overlapping parts within the target areas of each scanned image frame, the geometric transformation relationship between each scanned image frame is determined, and the scanned image frames are stitched together according to the geometric transformation relationship.

[0006] The image processing method of the scanning device according to an embodiment of the present invention reduces the influence of moiré patterns in the image and improves the accuracy of scanning stitching by partitioning the scanned image frame, extracting image feature measurement values ​​from each partition, and using the region with the largest image feature measurement value for processing subsequent captured images.

[0007] According to an embodiment of the present invention, the step of calculating an image feature metric value based on the image features of each region and selecting the region with the highest image feature metric value as the target region includes: Calculate the HOG feature value of the region, use the HOG feature value as the image feature metric, and select several regions with the largest HOG feature values ​​as the target regions. or, The norm of the HOG feature vector of the region is calculated, and the norm is used as the image feature metric. Several regions with the largest norm and reaching the set metric value are selected as target regions.

[0008] According to an embodiment of the present invention, the step of calculating an image feature metric value based on the image features of each region and selecting the region with the highest image feature metric value as the target region includes: The regions are filtered, and the image feature metrics of each filtered region are calculated. or, Detect moiré patterns and calculate the image feature metrics of the remaining regions after removing the moiré patterns.

[0009] According to one embodiment of the present invention, the detection of moiré patterns includes: Moiré region detection can be performed using frequency domain analysis, spatial domain analysis, or deep learning methods.

[0010] According to an embodiment of the present invention, the detection of moiré regions based on frequency domain analysis methods, spatial domain analysis methods, or deep learning methods includes: Calculate the characteristic parameters of each region, wherein the characteristic parameters include at least one of information entropy value, variance, peak ratio, sparsity and uniformity; The classifier is used to filter the Moore region based on the feature parameters.

[0011] According to one embodiment of the present invention, it further includes: If the target area is not selected and the text feature measurement value in the scanned image frame is greater than a set value, a prompt message will be issued to prompt the user to rescan the current position and adjust the scanning angle until the target area is selected. or, If the target area is not selected, each scanned image frame is re-partitioned along its own height direction until the target area is selected.

[0012] According to one embodiment of the present invention, the step of partitioning each scanned image frame recorded during the content scanning process along its own height direction to obtain at least two regions distributed along the height direction includes: Divide the scanned image frame into equal parts along its height direction to obtain multiple regions of the same size.

[0013] According to one embodiment of the present invention, the step of dividing the scanned image frame into multiple regions of equal size along the height direction includes: The height of each region is greater than the height of the font within the scanned image frame, and the font size is differentiated according to its size.

[0014] The present invention also provides an image processing apparatus for a scanning device, comprising: The partitioning module is used to partition each scanned image frame entered during the content scanning process along its own height direction to obtain at least two regions distributed along the height direction; The selection module is used to calculate the image feature metric value of each region based on its image features, and select the region with the highest image feature metric value as the target region. The stitching module is used to determine the geometric transformation relationship between the scanned image frames based on the image content of the overlapping parts in the target area of ​​each scanned image frame, and to stitch the scanned image frames together according to the geometric transformation relationship.

[0015] The present invention also provides a dictionary pen, comprising: The dictionary pen itself; A processor is disposed inside the dictionary pen body. The processor is used to respond to the user's sliding operation of scanning content through the dictionary pen, and to process each scanned image frame entered by the dictionary pen during the content scanning process according to the image processing method of the scanning device mentioned above to obtain the overall scanned image. A display screen is disposed on the surface of the dictionary pen body for displaying the overall scanned image.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the image processing method of the scanning device as described above.

[0017] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the image processing method of the scanning device as described above. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the image processing method of the scanning device provided by the present invention.

[0020] Figure 2 This is a schematic diagram of a scanned image frame provided by the present invention.

[0021] Figure 3 This is a schematic diagram of the partitioning of the scanned image frame provided by the present invention.

[0022] Figure 4 This is a comparative schematic diagram of the processing results of the image processing method of the scanning device provided by the present invention.

[0023] Figure 5 This is a schematic diagram of the image processing device of the scanning equipment provided by the present invention; Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] The following phenomenon exists in the related technology: During the image stitching process, if there are moiré patterns in the scanned image frame, the stitching position will be inaccurate, resulting in content loss. Furthermore, in the subsequent OCR recognition process, there may be missing words, misspelled words, and typos, which greatly reduces the accuracy of word extraction. Users need to scan multiple times to complete accurate word extraction, resulting in a poor user experience.

[0026] The main reason why moiré patterns affect image stitching is that the fundamental assumption of image stitching algorithms is that the scene is static, and images from different perspectives have a constant appearance in the overlapping area. However, moiré patterns are spatially variable. When the relative angle and distance between the camera and the subject change, the pattern, density, and direction of the moiré patterns will change drastically, making the moiré patterns on adjacent scanned image frames potentially quite different. This, in turn, has a serious impact on feature detection and matching, image alignment, and even image fusion.

[0027] To address the issue of inaccurate stitching positions mentioned above, this application proposes an image processing method for a scanning device. Please refer to [link to relevant documentation]. Figure 1 ,include: S100. Divide each scanned image frame entered during the content scanning process into regions along its own height direction to obtain at least two regions distributed along the height direction.

[0028] refer to Figure 2 Along the vertical direction, the central region of the scanned image frame is significantly affected by moiré patterns, while the top and bottom regions are relatively less affected. Therefore, if the entire scanned image frame is stitched together as a single unit, the interference from the moiré patterns in the central region may lead to inaccurate stitching positions. In this case, dividing the scanned image frame into at least two regions along the vertical direction allows for separate filtering of these regions to eliminate areas with significant interference, thus improving the accuracy of subsequent stitching.

[0029] In one example, the image frame can be divided equally along its height to obtain multiple regions of the same size. Please refer to [reference needed]. Figure 3 By dividing the image frame into equal parts, each region has the same pixel size. This means that during subsequent calculations, all regions are of the same magnitude and dimension, allowing for an objective and fair comparison of image feature metrics across regions, and accurately locating the region most suitable as the stitching standard.

[0030] Furthermore, equally dividing the scanned image frames reduces the processor's computational load and improves real-time scanning performance. If an adaptive non-equal division method is used (e.g., detecting the content of the scanned image frame before deciding how to divide it), additional computing power is required for preprocessing. However, with a fixed-ratio equal division method, the processor does not need to perform complex analysis of the scanned image frame content; it only needs to perform simple coordinate offsets based on the image resolution to complete the memory address allocation. This greatly simplifies the partitioning logic, shortens the processing time of a single frame, and helps maintain a smooth frame rate during high-speed scanning.

[0031] Furthermore, dividing the scanned image frames equally ensures the robustness of the stitching reference. If a non-equal division method is used, it may result in elongated regions with excessively small areas. During subsequent stitching, these excessively small regions may fail because they contain too little overlap, making it impossible to calculate a sufficient number of matching point pairs, thus causing stitching failure. By setting equal division, the minimum area of ​​each region can be pre-planned, ensuring that regardless of which region is ultimately selected as the reference, it has sufficient field of view and data volume to support subsequent image registration and fusion.

[0032] When dividing a scanned image frame into equal parts, the font size within the frame can be used for differentiation, ensuring that the height of each region is greater than the height of the font. This can be achieved through basic font size recognition within the scanned image frame. When the scanned image frame contains fonts of different sizes, it can be divided according to the smallest font size. Alternatively, the font size can be pre-stored in the scanning device based on typical paper document font sizes, and then automatically divided during the scanning process. This method of division ensures that each region contains as much of a complete line of text as possible, facilitating subsequent stitching.

[0033] Of course, if the scanned image frame is mechanically divided according to strict mathematical dividing lines, the dividing line may pass exactly through a line of text (for example, dividing the upper and lower parts of a line of Chinese characters into two different regions), which will interfere with subsequent image registration and feature extraction. To solve the above problem, this application embodiment, based on the equal division of the scanned image frame, can further adopt a content-aware adaptive fine-tuning strategy, including the following processing steps: The processor first calculates the theoretical coordinates of the equal divisions in the height direction based on the set number of partitions. For example, for an image with height H, the theoretical cutting lines are located at positions such as H / N and 2H / N.

[0034] Construct a floating search range: Centered on each theoretical bisector, preset a floating threshold range, such as ±Δh pixels. Δh can be a preset value or a value calculated adaptively based on the image resolution, usually slightly greater than half the height of a line of text.

[0035] Horizontal projection analysis: Within the floating search range, horizontal projection analysis is performed on the image data to count the sum of grayscale values ​​or the number of edge feature points for each row of pixels. Typically, the sum of pixels in the row containing text is higher, while the sum of pixels in the blank spaces between rows is lower.

[0036] Determine the actual dividing line: Find the minimum point of the projected value (i.e., the trough position) within the floating search interval, and establish the row coordinates corresponding to the minimum point as the final actual dividing line.

[0037] Through the above steps, the actual dividing line will intelligently snap into the blank gap between text lines, thus avoiding cutting off complete characters or text lines.

[0038] S200. Based on the image features of each region, calculate its image feature metric value, and select the region with the highest image feature metric value as the target region.

[0039] Based on the partitioning, step S200 requires further selection of the region with the highest image feature metric value as the target region. A higher graphic feature metric value indicates more text or background image content within the current region, and consequently, more feature points for image registration. To improve the accuracy of the graphic feature metric value calculation, each region can be filtered separately beforehand.

[0040] The filtering process can include the following methods: The first filtering method is frequency domain filtering. Moiré patterns typically appear as periodic high-frequency noise in an image. In the frequency domain (after Fourier transform), they appear as specific bright spots. A Fast Fourier Transform (FFT) is performed on the image to convert it from the spatial domain to the frequency domain. Due to their repetitive and high-frequency characteristics, moiré patterns form obvious peaks on the spectrogram. A Fourier transform is then performed on the region image. Band-stop or notch filters are designed to retain the low-frequency components representing the outlines of text or background images, while suppressing the specific high-frequency components representing moiré patterns. An inverse Fourier transform is then performed to restore the image. This is the most effective method for removing periodic noise, directly removing interfering fringes and thus eliminating the erroneous gradients they generate. Alternatively, only the thalassemia components representing text can be retained.

[0041] The second filtering method is edge-preserving smoothing filtering, which is a spatial domain method. While ordinary blurring (such as mean filtering and Gaussian filtering) can remove moiré patterns, it also blurs the edges of the target. This application uses edge-preserving filtering, which considers both pixel spatial distance and pixel value differences. Since moiré patterns typically manifest as small-range texture fluctuations with small but frequent pixel value differences, while object edges involve abrupt pixel value jumps, bilateral filtering can be used to smooth low-contrast textures while preserving strong edge gradients. Alternatively, guided filtering can be used, utilizing the structural information of a guiding image (usually the original image) to smooth the image. This is faster than bilateral filtering and better preserves the integrity of the gradient structure when smoothing moiré patterns.

[0042] The third filtering method: multi-scale pyramid downsampling. Since moiré patterns are an aliasing phenomenon closely related to image resolution, their formation is often due to insufficient sampling frequency. By constructing a Gaussian pyramid, the image is downsampled. Reducing the image resolution, for example, to 1 / 2 or 1 / 4 of the original, averages out high-frequency moiré details at the lower resolution, causing them to disappear or become uniform gray blocks.

[0043] The fourth filtering method involves adjusting the granularity of feature extraction to filter out the influence of moiré patterns. Specifically, increasing the CellSize (cell size) is beneficial. If the texture period of the moiré pattern is smaller than the cell size, and the cell size is large enough, the gradients in various directions generated by the moiré pattern may cancel each other out during histogram statistics and normalization. Therefore, by appropriately increasing the size of the Cell and Block, the focus is shifted to a more macroscopic shape rather than a microscopic texture.

[0044] Of course, the filtering of regions is not limited to the examples above. After filtering a region, the interference of moiré patterns can be eliminated or reduced. Based on this, the image feature metric value is calculated, and the region with the highest image feature metric value is selected as the target region, which can improve the accuracy of target region selection.

[0045] In addition to filtering the area as described above, you can also first detect the moiré pattern area, remove the moiré pattern area, and then select the target area from the remaining areas.

[0046] The methods for detecting moiré patterns include, but are not limited to, methods based on frequency domain analysis, spatial domain analysis, or deep learning.

[0047] To improve the accuracy and robustness of detection, this embodiment proposes a hybrid detection scheme based on multi-feature fusion and classifier selection. This includes: calculating feature parameters for each region, where the feature parameters include at least one of information entropy, variance, peak-to-peak ratio, sparsity, and uniformity; and using a classifier to select the Moore regions based on these feature parameters.

[0048] Specifically, moiré patterns typically exhibit complex, repetitive textures, with a grayscale distribution that differs from that of flat areas or natural edges. For example, a higher information entropy value usually indicates a greater amount of information or a more complex texture in the region, helping to distinguish text from moiré patterns. Furthermore, moiré patterns often exhibit significant pixel value fluctuations due to alternating light and dark stripes. Variance effectively reflects the contrast strength of the texture within a region, helping to eliminate background areas with insignificant grayscale changes. Additionally, by performing Fourier transforms or discrete cosine transforms on candidate regions to obtain a spectrum, moiré patterns in the frequency domain typically manifest as specific high-frequency peaks. The peak-to-peak ratio is defined as the ratio of the maximum energy peak (excluding the DC component) to the total energy of the spectrum. This parameter effectively identifies moiré patterns with periodic repetitive structures. Finally, sparsity representation can be used to represent regular textures like moiré patterns. In a specific transform domain, moiré patterns exhibit sparsity, meaning most of the energy is concentrated on a few coefficients. Calculating the sparsity of the transform coefficients can distinguish between random noise and regular moiré patterns. For example, the energy or second moment of a region can be calculated based on the gray-level co-occurrence matrix (GLCM). Uniformity reflects the degree of uniformity of the gray-level distribution of an image. Although moiré patterns are textures, their stripe arrangement has a high degree of directionality and regularity. Their texture uniformity characteristics are significantly different from those of text and background images.

[0049] Building upon the above, a classifier is used to filter moiré regions based on feature parameters. After obtaining one or more of the aforementioned feature parameters, a feature vector is constructed and input into a pre-trained classifier for discrimination. The classifier can be a Support Vector Machine (SVM), Random Forest, Naive Bayes classifier, or a lightweight neural network based on a Multilayer Perceptron (MLP). The classifier training process includes: in an offline phase, collecting positive sample image patches containing moiré patterns and negative sample image patches without moiré patterns (such as text, solid color backgrounds, and natural textures), calculating the aforementioned feature parameters as training data, and performing supervised training on the classifier to obtain a classification model. Based on this, the constructed feature vector is input into the classifier, and the classifier outputs the probability or label that the region belongs to the "moiré region". If it is determined to be a moiré region, the location coordinates of the region are recorded.

[0050] After filtering the region or removing moiré patterns, the image feature metrics of the region can be further calculated.

[0051] The method for calculating image feature metrics in this application includes, but is not limited to: The first calculation method is based on region selection using image gradient and texture intensity. In this case, the image feature metric is also the image gradient and texture intensity. Specifically, the gradient information of image pixels is used to measure the feature richness or sharpness of a region. The higher the gradient value, the richer the edge, contour, or texture information contained in the region.

[0052] In this calculation method, the image feature metric is calculated based on HOG (Histogram of Oriented Gradients) feature values. The HOG feature value is the image feature metric. Essentially, the Histogram of Oriented Gradients statistically analyzes the gradient directions of local regions, describing the local texture and contour features of an image in histogram form. The core logic is: the contour of an object is composed of edges, and the direction and intensity of the edges can be represented by gradients; statistically analyzing the distribution of gradient directions describes the contour. The HOG feature values ​​of all candidate regions are calculated, and the regions with the largest HOG feature values ​​are selected as the target regions. The specific calculation steps for HOG are as follows: (1) Preprocessing: Convert the image to grayscale and then perform gamma correction (normalized brightness) to reduce the impact of illumination changes on the gradient.

[0053] (2) Calculate the gradient: Use the Sobel operator to calculate the gradient magnitude and direction of each pixel.

[0054] (3) Construct cell histograms, divide the image into multiple small square (e.g., 8×8) pixel cell units, divide the gradient direction of each cell unit into several bins (gradient direction intervals), calculate the gradient contribution in each bin, and weight it with gradient magnitude. The larger the magnitude, the greater the contribution to that direction. Finally, obtain the n-dimensional histogram of each cell unit.

[0055] (4) Block normalization: Group adjacent cell units into blocks (e.g., 2×2 cells form a block, i.e., 16×16 pixels). Normalize the histogram of all cells within the block (e.g., L2 normalization) to eliminate the influence of local differences in lighting and contrast.

[0056] (5) Concatenate feature vectors: Concatenate all normalized block features in order to obtain the HOG feature vector of the entire image.

[0057] Here, the hog feature value can be understood as the value under a specific bin, such as the value corresponding to 90 degrees or the value corresponding to 0 degrees.

[0058] Of course, HOG eigenvalues ​​can also be understood as the norm of HOG eigenvectors, in which case the multidimensional vector is converted into a scalar.

[0059] After obtaining the image feature metrics of all regions, sort them in descending order and select one or more of the top-ranked regions as the target regions.

[0060] The second calculation method is region selection based on visual saliency. This method is suitable for finding regions in an image that differ most from the background and are most likely to attract human attention. In this embodiment, the image feature metric is the saliency score.

[0061] The third calculation method is region selection based on information entropy. This method is suitable for finding the regions with the most information and the most complex details in an image. In this embodiment, the image feature metric is the regional grayscale entropy, and the regions with the highest entropy values ​​are selected as the target regions.

[0062] The fourth calculation method: Region selection based on deep learning confidence. In this embodiment, the image feature metric is the target detection confidence. Calculating the image feature metric specifically includes: inputting each region into a pre-trained convolutional neural network model; obtaining the response value of the feature map output by the model at a specific feature layer, or obtaining the probability value of belonging to the target category output by the fully connected layer. The probability value or the mean of the feature response is used as the image feature metric. The highest metric value means that the neural network considers that the region is most likely the expected target object. The expected target object here can be text or a picture.

[0063] S300: Based on the image content of the overlapping parts in the target area of ​​each scanned image frame, determine the geometric transformation relationship between each scanned image frame, and stitch the scanned image frames together according to the geometric transformation relationship.

[0064] In S300, feature point extraction is performed only on the target regions of the previous and current frame images. Feature extraction algorithms can include SIFT (Scale Invariant Feature Transform), SURF (Fast Robust Feature Transform), ORB, or AKAZE. Based on the extracted feature descriptors, matching feature point pairs are found in the overlapping areas of the two target regions. Furthermore, erroneous matching point pairs can be eliminated using the random sample consensus method. Based on the filtered correct matching point pairs, the geometric transformation matrix between adjacent image frames is calculated.

[0065] In one embodiment, after removing the moiré pattern area, the norm of the HOG feature vector of the remaining area is calculated, and the norm is used as the image feature metric. Several areas with the largest image feature metric values ​​that reach a set metric value are selected as target areas. In this case, if the image feature metric values ​​of the remaining areas do not reach the set metric value after removing the moiré pattern area, it indicates that an ideal target area has not been selected. In this case, the image processing method of the scanning device in this embodiment further includes: S400, if no target area is selected and the text feature metric value in the scanned image frame is greater than the set value, a prompt message is issued to prompt the user to rescan the current position and adjust the scanning angle until a target area is selected.

[0066] Considering that moiré patterns are related to the user's scanning angle, the failure to select a target area might be due to the current scanning angle causing a large number of moiré patterns. Therefore, the system uses sound or light prompts to remind the user to adjust the scanning angle and scan again. The additional condition "the text feature measurement value in the scanned image frame is greater than the set value" ensures that the reason the target area is currently selected is due to the scanning angle, not because the scanned object is a blank area.

[0067] In another embodiment, the failure to select a target area may also be due to an unreasonable image partitioning method, resulting in the text feature measurement values ​​in each area not meeting expectations. Based on this, the image processing method of the scanning device in this application embodiment further includes: S500, for the unselected target area, re-partitioning each scanned image frame along its own height direction until a target area is selected. This process can increase the height of each area, thereby ensuring that each area has sufficient information.

[0068] The scanning device can preset the number of target regions, which is less than the total number of regions formed by the division. If insufficient feature points can be selected in step S300 for stitching, the number of target regions can be gradually increased until the stitching requirements are met. The maximum number of expanded target regions can be equivalent to the total number of regions formed by the division.

[0069] According to the image processing method of the scanning device in the embodiments of this application, the scanning image frame is divided into partitions, and then image feature measurement values ​​are extracted from each partition. The region with the largest image feature measurement value is used to process subsequent captured images, thereby reducing the influence of moiré patterns in the image and improving the accuracy of scanning stitching.

[0070] Related technologies do not partition scanned image frames in image processing methods, nor do the image processing methods of the scanning device in the embodiments of this application. Figure 2When scanning the corresponding scanning medium, please refer to the scanning results. Figure 4 In the former case, the stitched scanned image frames result in content loss, leading to the display of "hiding under a mushroom" or "19.9". However, the stitched scanned image frames in this embodiment can display the complete content such as "hiding under a mushroom" or "2019.9".

[0071] The target image obtained after stitching can be processed by line cutting and OCR recognition, and then fed into a translation model for word lookup and translation; it can also be fed into a question-searching model to complete question searching; or it can be fed into an essay correction model for essay correction, etc., depending on the application launched by the user.

[0072] The image processing apparatus of the scanning device provided by the present invention will be described below. The image processing apparatus of the scanning device described below can be referred to in correspondence with the image processing method of the scanning device described above. That is, the content of all image processing methods of the scanning device can be used to explain the image processing apparatus of the scanning device in the embodiments of this application, and some repeated content will not be described again.

[0073] For the image processing apparatus of the scanning device in this application embodiment, please refer to [link to application]. Figure 5 ,include: The partitioning module 10 is used to partition each scanned image frame entered during the content scanning process along its own height direction to obtain at least two regions distributed along the height direction. The selection module 20 is used to calculate the image feature metric value of each region based on its image features, and select the region with the highest image feature metric value as the target region. The stitching module 30 is used to determine the geometric transformation relationship between the scanned image frames based on the image content of the overlapping parts in the target area of ​​each scanned image frame, and to stitch the scanned image frames together according to the geometric transformation relationship.

[0074] According to an embodiment of this application, the selection module 20 is further configured to perform filtering processing on the regions, calculate the hog feature values ​​of all regions, use the hog feature values ​​as image feature metrics, and select several regions with the largest hog feature values ​​as target regions.

[0075] According to an embodiment of this application, the selection module 20 is further used to detect moiré pattern regions; after removing the moiré pattern regions, the norm of the hog feature vector of the remaining regions is calculated respectively, the norm is used as the image feature metric, and several regions with the largest image feature metric and reaching the set metric value are selected as target regions.

[0076] According to an embodiment of this application, the image processing device of the scanning device further includes a prompting module, which is used to issue a prompting message when the target area is not selected and the text feature measurement value in the scanned image frame is greater than a set value, so as to prompt the user to rescan the current position and prompt the user to adjust the scanning angle until the target area is selected.

[0077] According to an embodiment of this application, the image processing apparatus of the scanning device further includes a correction module, which is used to repartition each scanned image frame along its own height direction when no target area is selected, until the target area is selected.

[0078] Figure 6 A schematic diagram of the physical structure of an electronic device is provided. This electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840. The processor 810, communications interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute an image processing method for a scanning device. This method includes: partitioning each scanned image frame entered during the content scanning process along its own height direction to obtain at least two regions distributed along the height direction; calculating the image feature metric value of each region based on its image features, and selecting the region with the highest image feature metric value as the target region; determining the geometric transformation relationship between the scanned image frames based on the image content of the overlapping parts within the target regions of each scanned image frame, and stitching the scanned image frames together according to the geometric transformation relationship.

[0079] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0080] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the image processing method of the scanning device provided by the above methods. The method includes: partitioning each scanned image frame recorded during the content scanning process along its own height direction to obtain at least two regions distributed along the height direction; calculating its image feature metric value based on the image features of each region, and selecting the region with the highest image feature metric value as the target region; determining the geometric transformation relationship between each scanned image frame based on the image content of the overlapping part in the target region of each scanned image frame, and stitching the scanned image frames together according to the geometric transformation relationship.

[0081] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements an image processing method of the scanning device provided by the above methods. The method includes: partitioning each scanned image frame entered during the content scanning process along its own height direction to obtain at least two regions distributed along the height direction; calculating an image feature metric value based on the image features of each region, and selecting the region with the highest image feature metric value as the target region; determining the geometric transformation relationship between each scanned image frame based on the image content of the overlapping part in the target region of each scanned image frame, and stitching the scanned image frames together according to the geometric transformation relationship.

[0082] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0083] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image processing method for a scanning device, characterized in that, include: Each scanned image frame entered during the content scanning process is divided along its own height direction to obtain at least two regions distributed along the height direction; Based on the image features of each region, calculate its image feature metric value, and select the region with the highest image feature metric value as the target region. Based on the image content of the overlapping parts within the target areas of each scanned image frame, the geometric transformation relationship between each scanned image frame is determined, and the scanned image frames are stitched together according to the geometric transformation relationship.

2. The image processing method of the scanning device according to claim 1, characterized in that, The process of calculating image feature metrics based on the image features of each region and selecting the region with the highest image feature metrics as the target region includes: Calculate the HOG feature value of the region, use the HOG feature value as the image feature metric, and select several regions with the largest HOG feature values ​​as the target regions. or, The norm of the HOG feature vector of the region is calculated, and the norm is used as the image feature metric. Several regions with the largest norm and reaching the set metric value are selected as target regions.

3. The image processing method of the scanning device according to claim 1, characterized in that, The process of calculating image feature metrics based on the image features of each region and selecting the region with the highest image feature metrics as the target region includes: The regions are filtered, and the image feature metrics of each filtered region are calculated. or, Detect moiré patterns and calculate the image feature metrics of the remaining regions after removing the moiré patterns.

4. The image processing method of the scanning device according to claim 3, characterized in that, The detected moiré pattern area includes: Moiré region detection can be performed using frequency domain analysis, spatial domain analysis, or deep learning methods.

5. The image processing method of the scanning device according to claim 4, characterized in that, The detection of moiré regions based on frequency domain analysis methods, spatial domain analysis methods, or deep learning methods includes: Calculate the characteristic parameters of each region, wherein the characteristic parameters include at least one of information entropy value, variance, peak ratio, sparsity and uniformity; The classifier is used to filter the Moore region based on the feature parameters.

6. The image processing method of the scanning device according to any one of claims 3 to 5, characterized in that, Also includes: If the target area is not selected and the text feature measurement value in the scanned image frame is greater than a set value, a prompt message will be issued to prompt the user to rescan the current position and adjust the scanning angle until the target area is selected. or, If the target area is not selected, each scanned image frame is re-partitioned along its own height direction until the target area is selected.

7. The image processing method of the scanning device according to claim 1, characterized in that, The step of partitioning each scanned image frame recorded during the content scanning process along its own height direction to obtain at least two regions distributed along the height direction includes: Divide the scanned image frame into equal parts along its height direction to obtain multiple regions of the same size.

8. The image processing method of the scanning device according to claim 7, characterized in that, The process of dividing the scanned image frame into equal parts along its height direction yields multiple regions of equal size, including: The height of each region is greater than the height of the font within the scanned image frame, and the font size is differentiated according to its size.

9. An image processing apparatus for a scanning device, characterized in that, include: The partitioning module is used to partition each scanned image frame entered during the content scanning process along its own height direction to obtain at least two regions distributed along the height direction; The selection module is used to calculate the image feature metric value of each region based on its image features, and select the region with the highest image feature metric value as the target region. The stitching module is used to determine the geometric transformation relationship between the scanned image frames based on the image content of the overlapping parts in the target area of ​​each scanned image frame, and to stitch the scanned image frames together according to the geometric transformation relationship.

10. A dictionary pen, characterized in that, include: The dictionary pen itself; A processor is disposed inside the dictionary pen body. The processor is used to process, in response to the user's sliding operation of scanning content with the dictionary pen, each scanned image frame entered by the dictionary pen during the content scanning process according to the image processing method of the scanning device according to any one of claims 1 to 8, to obtain an overall scanned image. A display screen is disposed on the surface of the dictionary pen body for displaying the overall scanned image.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the image processing method of the scanning device as described in any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the image processing method of the scanning device as described in any one of claims 1 to 8.

Images