Fingertip-operated scanner, image synthesis method, and information input method

Abstract

To provide a fingertip-operated scanner that enables seamless cooperation with other devices, thereby improving convenience, and an image synthesis method and information input method using the same. [Solution] The fingertip-operated scanner has an image synthesis processing unit 18 that synthesizes multiple images to generate a composite image based on multiple images acquired by an area sensor 11 and information detected by a position sensor and angle sensor 13. It also has an optical mouse sensor 14 that generates mouse pointing information based on an operation unit 21, and a communication module 20 that transmits the composite image or processed information based on the composite image, and the mouse pointing information to an external electronic device. The main body 1 or PC 30 has an OCR processing unit 32 that recognizes character information included in the composite image.

Description

[Technical Field]

[0001] The present invention relates to mobile scanner technology for digitizing information such as characters, codes, and graphic images written on paper and inputting the information directly into electronic devices such as computers, smartphones, and tablets. In particular, this paper relates to a small, lightweight fingertip-operated scanner (FS: Fingertip Mouse Scanner), which solves the problems of conventional large scanners and software scanners that require manual photography, efficiently captures only the necessary parts from the surfaces of irregular paper prints and various objects, and can be wirelessly connected to smartphones and tablets, as well as an image synthesis method and information input method using this scanner. [Background technology]

[0002] In recent years, information printed on paper, such as official documents, contracts, books, newspapers, magazines, photographs, and receipts, has been rapidly digitized for the convenience of storage, management, and retrieval. Along with this trend toward digitalization, software scanners that use scanners and smartphone cameras have become widespread. However, these conventional methods have the following issues:

[0003] Typical desktop scanners are large and require a lot of space to be installed, as they are designed to scan the entire document. Furthermore, their functionality for scanning only specific areas is limited. On the other hand, software scanners that use the built-in camera of a smartphone require manual capture, making it difficult to adjust the required range and optimize lighting conditions, and image quality is prone to degradation due to camera shake. Furthermore, continuous capture while handheld is difficult for average users.

[0004] Furthermore, many conventional scanners lacked the mouse functionality to directly input scanned image information to the cursor position of the user application on a PC or electronic device, which meant users had to frequently switch between operating the scanner and using the mouse, which was time-consuming.

[0005] For this reason, the present applicant has proposed a mouse scanner that integrates a mouse and a scanner, as shown in Patent Document 1. However, the mouse scanners in Patent Documents 1 and 2 are primarily for personal computers and are operated by placing your hand over them, which leaves them insufficient in terms of further reducing size and weight. In particular, they are too large to be used as pointing devices for devices with small screens such as smartphones and tablets, making them difficult to use.

[0006] Additionally, imaging technologies using six-axis motion sensors and area sensors in smartphones and other devices include technologies for generating panoramic images by stitching together wide-area images and technologies for recognizing the 3D shapes of objects. While these technologies, like scanners, use multiple images and position and orientation information to synthesize them, their purpose is to generate wide-angle image information or acquire spatial information. Such prior art technologies are fundamentally different from the objective of the present invention, which is to selectively capture specific text and graphic images on paper media or objects in any shape and input them directly into the cursor position of a user application. These technologies do not solve the problem of recognizing specific information (such as characters or codes), converting them into electronic text data, and inputting them directly into an application, and do not directly contribute to the efficiency and convenience of information input desired by users. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2018-57010 A [Patent Document 2] Japanese Patent Application Publication No. 2017-130984 Summary of the Invention [Problem to be solved by the invention]

[0008] As described above, the main problems with the conventional techniques are as follows: First, typical desktop scanners require installation space due to their large size and are designed to scan the entire object, making it difficult to selectively scan only a specific narrow area. Second, software scanners that use the built-in camera of a smartphone rely on manual photography, making them susceptible to the effects of camera shake and lighting conditions, and making it difficult to accurately adjust the required range.

[0009] Furthermore, no means was provided for directly and seamlessly inputting image information or text information obtained by scanning into the cursor position of a user application running on a personal computer or other electronic device, and the separation of scanning operations and input operations significantly reduced overall operability.

[0010] Furthermore, imaging technologies using six-axis motion sensors and area sensors exist, including technologies for generating panoramic images by stitching together wide-area images and technologies for recognizing the 3D shape of objects. However, these technologies primarily aim to generate wide-angle image information or acquire spatial information, and do not address the information input problem addressed by the present invention, which is to "selectively capture" specific characters, codes, or graphic images of any shape present on paper media or objects and "directly input them into an application." Furthermore, simply making conventional combined mouse scanners smaller and thinner presented new challenges, such as a shallow depth of field and image distortion, making them unsuitable for use on mobile devices like smartphones and tablets. Thus, the technology for efficiently scanning and digitizing only the necessary information printed on various objects, not just paper media, remains an unresolved issue.

[0011] The present invention has been proposed to solve the above-mentioned problems of the prior art. The object of the present invention is to provide a fingertip-operated scanner and an operation method thereof that is even smaller and lighter, and in particular, that solves the issue of imaging quality that comes with thinning, while enabling seamless integration with other devices such as smartphones and tablets, thereby improving convenience. [Means for solving the problem]

[0012] In order to solve the above problems, the present invention provides a fingertip-operated scanner having a main body that is worn on a fingertip, characterized by having the following configuration. (1) An area sensor attached to the main body that scans the surface of an object to obtain image information. (2) A position sensor that detects movement information of the main body, and an angle sensor that detects angle information of the main body. (3) An LED light source provided on the main body to illuminate the scanning area. (4) An operating section of the scanner provided on the main body. (5) An image synthesis unit that synthesizes the multiple images acquired by the area sensor and information detected by the position sensor and the angle sensor to generate a synthesized image. (6) A communication module that transmits the composite image or processing information based on the composite image to an external electronic device. (7) The area sensor includes a high-density area sensor, a thin macro lens laminated on the lower surface of the high-density area sensor, and a transparent plate laminated on the lower surface of the macro lens.

[0013] In the present invention, the following configuration can also be adopted. (1) The image synthesis unit a sensor data integration and motion tracking unit that synchronizes and integrates a plurality of image data successively acquired by the area sensor and real-time movement information and posture information of the main body detected by the position sensor and the angle sensor; an image alignment and combination unit that estimates a relative positional relationship between the plurality of images based on the integrated information and combines the images while correcting image distortion; and an output pre-processing unit that performs output pre-processing, including distortion correction, perspective correction, keystone correction, and image enhancement processing, on the composite image combined by the image alignment and combination unit.

[0014] (2) an input linking unit that inputs the composite image to a user application of the external electronic device; The external electronic device includes an OCR processing unit that is provided in the external electronic device and that recognizes character information included in image data input to the input linkage unit.

[0015] (3) an OCR processing unit that recognizes character information included in the composite image; and an input linking unit that directly inputs character information recognized by the OCR processing unit into a cursor position of a user application of the external electronic device.

[0016] (4) A ring-shaped finger insertion portion to be worn on a user's finger is provided on the underside of the main body, a sliding surface on the top surface of the main body that is movable in contact with the object to be read; The area sensor and the LED light source are provided facing the object to be read from the sliding surface.

[0017] (5) The main body has a lid that can be opened and closed to cover the sliding surface, and the surface of the lid is decorated.

[0018] (6) In addition to the communication module, a wired communication means using a USB cable is provided.

[0019] (7) The image data or text information transmitted via the communication module is in a format that can be used by one or more of the following applications (a) to (d) running on an external electronic device: (a) A translation application that performs translation processing (b) A visual support application that converts recognized text information into audio and outputs it. (c) A code reading application that recognizes barcodes or QR codes (registered trademarks) (d) Input assistance applications that assist with filling out certain digital forms

[0020] (8) a mouse operation unit provided on the main body; an optical mouse sensor that generates mouse pointing information based on the position sensor and the mouse operation unit; a communication module for transmitting mouse pointing information based on the optical mouse sensor to an external electronic device; Equipped with.

[0021] The image synthesis method of the present invention includes: a step of synchronizing and integrating a plurality of pieces of image data sequentially acquired by the area sensor and real-time movement information and attitude information of the main body detected by the position sensor and the angle sensor; estimating a relative positional relationship between the plurality of images based on the integrated information, and combining the images while correcting image distortion; and performing output pre-processing on the combined image, including distortion correction, perspective correction, keystone correction, and image enhancement.

[0022] The information input method of the present invention is an information input method using the fingertip-operated scanner of the present invention, characterized in that it scans a predetermined area on the surface of an object, recognizes character information from the scanned image, and automatically inputs the character information at the cursor position of a user application in an external electronic device. [Effects of the Invention]

[0023] According to the present invention, it is possible to provide a fingertip-operated scanner that enables seamless integration with other devices, thereby improving convenience, and an image synthesis method and information input method using the same. In particular, according to the present invention, the area sensor comprises a high-density surface sensor, a thin macro lens laminated on the underside of the high-density surface sensor, and a transparent plate laminated on the underside of the macro lens, thereby eliminating the need for the conventional optical lens prism system and enabling the entire scanner to be made thinner.

[0024] Furthermore, by using a high-density surface sensor and a thin macro lens, it is possible to capture images while moving the sensor in any direction, without being restricted by the direction of movement relative to the subject, compared to conventional scanners that use line sensors.This means that target information can be stably acquired while flexibly following the curvature and inclination of the object surface, not limited to the paper surface. Furthermore, the transparent plate is configured to introduce illumination light uniformly through the plate while maintaining a constant distance from the object to be imaged, thereby ensuring an appropriate amount of light while minimizing the thickness of the housing, and enabling clear imaging of characters, codes, or graphic images written on paper or the surface of an object. Furthermore, by using the movement and orientation information obtained from each sensor to perform image linking and distortion correction on the captured images, it is possible to selectively extract only the necessary information from any area of ​​the target surface, electronically format it, and then input it directly into an application.

[0025] As a result, the present invention makes it possible to realize an information input device that is "selective," "instant," "portable," and "highly accurate," which was difficult to achieve with conventional mouse-type scanners or general-purpose camera devices, and provides a new style of information processing, such as allowing partial character strings on a document, a barcode on a package, or a figure on a clothing tag to be imported into a smartphone or tablet simply by tracing it on the spot. [Brief explanation of the drawings]

[0026] [Figure 1] FIG. 1 is a side view of a fingertip-operated scanner according to a first embodiment. [Figure 2] FIG. 1 is a plan view of a first embodiment. [Figure 3] FIG. 2 is a schematic cross-sectional view of an area sensor portion of the first embodiment. [Figure 4] FIG. 1 is a block diagram showing a configuration of a first embodiment. [Figure 5] 4 is a flowchart showing the operation of the first embodiment. [Figure 6]FIG. 10 is a block diagram showing the configuration of a second embodiment. [Figure 7] FIG. 10 is a side view showing the appearance of a third embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0027] [1. First embodiment] [1-1. Configuration of the first embodiment] A first embodiment of the present invention will be described in detail below with reference to the drawings. In this embodiment, a mouse function is incorporated into the fingertip-operated scanner, enabling it to be used as a mouse scanner. In this embodiment, information from an optical mouse sensor is used as the position sensor for the scanner. A six-axis motion sensor is also used as the angle sensor. A six-axis motion sensor calculates position (distance traveled) from acceleration data and estimates attitude (rotation, tilt) from angular velocity data, but in this embodiment, the position detection function of the six-axis motion sensor is used as the position sensor for the scanner. This is because position information from an optical mouse sensor is faster and more accurate than position detection by a six-axis motion sensor. However, if the mouse function is not incorporated into the fingertip-operated scanner, the six-axis motion sensor may be used as both a position sensor and an angle sensor, or other position sensors or angle sensors may be used.

[0028] FIG. 1 is a side view showing the appearance of the first embodiment, FIG. 2 is a plan view of the same, and FIG. 3 is a schematic cross-sectional view of an area sensor portion. The fingertip-operated scanner (hereinafter referred to as the scanner) of this embodiment comprises a box-shaped main body 1 and a ring-shaped finger insertion section 2 for attaching the main body 1 to a finger. The finger insertion section 2 is fixed to the back surface of the main body 1, and is provided with a mechanism for adjusting the inner diameter of the finger insertion section 2 to fit the thickness of the user's finger.

[0029] This inner diameter adjustment mechanism may have any configuration, and for example, in a metal finger insertion section 2, a notch can be made in part of the finger insertion section 2, and the width of the notch can be adjusted by bending the finger insertion section 2. Also, part or all of the finger insertion section 2 can be made of hook-and-loop fastener. The surface of the main body 1 opposite the finger insertion section 2 is formed flat, and serves as a sliding surface 3 for the paper or screen to be scanned when the scanner is in use.

[0030] The main body 1 is provided with a lid 4 that covers the sliding surface 3. The lid 4 may be an openable / closable type with a hinge on one side of the main body 1, or a removable cover type. When using the scanner of this embodiment, the lid 4 is opened to expose the sliding surface 3, and the main body 1 is rotated around the finger insertion section 2 so that the sliding surface 3 faces downward, and the scanner is brought into close contact with the surface of a piece of paper, a tablet, or the like. In this state, characters or images of an area larger than the area of ​​the sliding surface 3 can be read by drawing a spiral shape with the fingertip or by moving the fingertip back and forth and left and right over the area to be scanned.

[0031] The main body 1 is provided with a chain 5 to prevent it from being lost or falling off, and this chain 5 is connected to a ring 6 worn on the finger. The surfaces of the cover 4 and the main body 1 are provided with decorative patterns, so that when not in use the scanner looks like an ornament, making it comfortable to carry around.

[0032] As shown in FIGS. 3 and 4, the main body 1 is provided with the following devices that constitute a mouse and a scanner. (A) Area sensor 11 The main body 1 is provided with an area sensor 11 for acquiring image information of the surface of an object. As one of the important features of the present invention, the area sensor 11 has a configuration in which the following multiple elements are integrally stacked. Specifically, the area sensor 11 is composed of a high-density two-dimensional area image sensor 11a, a thin macro lens 11b stacked on the underside of the area image sensor 11a, and a transparent plate 11c stacked on the underside of the thin macro lens 11b. The surface image sensor 11a is a high-sensitivity CMOS sensor, such as the Sony IMX675, that offers a wide dynamic range and high sensitivity. This characteristic allows it to capture high-resolution snapshot images in a short time while maintaining a predetermined field of view and resolution, even during high-speed movements such as tracing a fingertip, while suppressing blur. This provides abundant image information that enables high-precision image synthesis in the subsequent image synthesis process.

[0033] The thin macro lens 11b, layered on the underside of the surface-type image sensor 11a, utilizes metasurface and micro-aspherical lens technologies. The thin macro lens 11b is extremely thin, measuring less than 1 mm, and is capable of focusing light from a subject onto the surface-type image sensor 11a at a close distance with a focal length of a few millimeters. For example, the metasurface lens (e.g., an ultra-thin resin lens using nanoimprint technology) developed by Canon Inc. or the Optical metasurface lens platform (product example: the Metalenz VIS lens family) developed by Metalenz, Inc., USA, can achieve a similarly thin and highly accurate imaging function. Furthermore, these lenses are significantly thinner than conventional spherical or aspherical lenses, yet offer excellent aberration correction and resolution, making them particularly effective for close-up photography. The adoption of this thin macro lens 11b eliminates the need for the conventional complex and thick optical lens prism system, significantly contributing to the overall slimming and miniaturization of the scanner. In addition, the thin macro lens 11b is designed to form an upright, minimally distorted image on the sensor even when the subject is close by, thereby reducing the burden of distortion correction in subsequent image processing and contributing to improved final image quality.

[0034] For example, lenses using the aforementioned metasurface technology (metalenses) operate on a different principle from conventional lenses, which focus light by utilizing the refraction of glass or plastic. Specifically, they focus light by arranging nano-sized structures called "metamaterials," which are smaller than the wavelength of light, on a flat surface and controlling the phase of the light. This principle eliminates the need for thickness and curved surfaces required for conventional lenses, theoretically making it possible to create lenses thinner than a human hair. Furthermore, metalenses can be mass-produced using semiconductor manufacturing processes, significantly contributing to the thinning, miniaturization, weight reduction, and cost reduction of the entire scanner, which is the goal of this embodiment. In addition, research and development toward practical application of metalense technology is progressing, and practical application is approaching every day. Metalenses technology is expected to be used in fields such as smartphones, surveillance cameras, and VR / AR devices, and some optical sensors, such as LiDAR sensors, have already been commercialized. These circumstances support the technical feasibility and effectiveness of applying metalenses to the optical system of this embodiment.

[0035] In this embodiment, the following additional configurations and functions are provided to address the issues of shallow depth of focus that arise when using the thin macro lens 11b, and the problem of focus deviation caused by unevenness or tilt of the paper surface to be read. (1) Aberration correction module and sensor surface optimization The thin macrolens 11b can be configured not only as a single element, but also by stacking or arranging multiple thin lens elements in an array to physically reduce optical aberrations such as chromatic aberration, distortion, and field curvature over a wide field of view. In particular, for chromatic aberration, a combination of materials with different refractive properties or a metalens design optimized for a specific wavelength band can be applied. Furthermore, the surface image sensor 11a maintains focus across the entire imaging surface by either slightly curving the surface or dividing it into multiple inclined segments in accordance with the focal plane shape of the thin macro lens 11b. This optimization of the sensor surface reduces blurring at the periphery of the image due to the curvature of the focal plane caused by the thin macro lens 11b and the shallow depth of field.

[0036] (2) Spacer layer 11d It is also possible to interpose a spacer layer 11d of a predetermined thickness (for example, about 0.3 mm to 1 mm) between the surface-type imaging element 11a and the thin macro lens 11b. This spacer layer 11d ensures an optical distance, thereby extending the effective focal length of the thin macro lens 11b and consequently expanding the depth of field. This makes it difficult for focus shifts to occur even when the object being read has minute irregularities or tilts, and stable image quality can be maintained. The spacer layer 11d may be a simple gap or a transparent material.

[0037] (3) Elastic support structure and flexibility of the transparent plate 11c The transparent plate 11c laminated on the underside of the thin macro lens 11b serves to protect the entire imaging system, including the surface image sensor 11a and the thin macro lens 11b, from physical shocks, dust, and dirt during scanning. In addition, this transparent plate 11c has an optically flat surface, and comes into contact with the object to be read as the sliding surface 3 when the scanner is in use. This transparent plate 11c is preferably attached to the main body 1 by a support structure that allows elastic displacement, such as a spring support or elastomer mount. This allows the sliding surface 3 to displace slightly in response to the minute irregularities, waves, and tilt of the object to be read, always maintaining an appropriate imaging distance. It is also possible to use a low-friction, flexible resin material such as thin polycarbonate or PET as the material for the transparent plate 11c, which can flexibly accommodate the curved surface deformation of the object to be read, achieving both smooth scanning operation and stable focal length. Because the transparent plate 11c comes into direct contact with the object to be read, the distance between the surface image sensor 11a and the object to be imaged is always kept constant, which prevents the object from becoming out of focus during scanning and makes it possible to obtain high-resolution images with consistently stable image quality, greatly contributing to improving the accuracy of subsequent OCR processing and image analysis. The area sensor 11 captures images (snapshots) at predetermined imaging intervals or moving distances in response to a user's operation of tracing the surface of the object to be scanned (a scan path of any shape), as will be described later by a control unit. The captured images are monochrome or color, and the resolution and exposure conditions are automatically adjusted.

[0038] To optimize the imaging performance of the area sensor 11, the main body 1 is equipped with an LED illuminator 12 located close to the area sensor 11, uniformly illuminating the target. This LED illuminator 12 is constructed using a white or RGB LED, with a color temperature of daylight white (5000K to 6500K) being preferable for OCR applications. The light from the LED illuminator 12 is homogenized through an acrylic or glass light guide plate or diffusion film to reduce hot spots (localized brightness variations). This homogenized light is uniformly irradiated from a predetermined angle (e.g., off-axis) onto the target object that contacts the slide surface 3 during fingertip scanning. This enhances the shading and contrast of the text and graphic images captured while maintaining an appropriate overall exposure balance and minimizing the effects of camera shake and shadows during scanning. Such uniform and stable illumination reduces differences in exposure conditions between the snapshot images to be combined, significantly improving the accuracy of the subsequent image combination process and ultimately the recognition rate of the OCR process.

[0039] (a) Angle sensor 13 The movement of this scanner is constantly measured by the angle sensor 13 and a position sensor provided in the optical mouse sensor 14 (described later). Because the operation of a finger-mounted scanner depends on the movement of the user's fingertip, it is difficult to maintain a constant angle relative to the document surface. Therefore, the rotation or tilt of the fingertip can cause the direction of the movement vector to deviate from the document coordinate system, and trapezoidal distortion can occur in the captured image. In this embodiment, the angle sensor 13 is used to detect changes in the fingertip's angle, such as the yaw angle, pitch angle, and roll angle, and reflect these changes in the movement amount calculation. This allows for correction of the stitching direction and distortion of the scanned image, enabling accurate position detection and the generation of highly accurate scanned images even when the finger is moving freely.

[0040] In the finger-mounted scanner of the present invention, the position and angle detection means can be optimally switched or their roles can be divided depending on whether or not the optical mouse sensor 14 is included. In the most preferred embodiment equipped with the optical mouse sensor 14, the advantages of each sensor are clearly divided. Specifically, the optical mouse sensor 14, which is comprised of an illumination light source, an image sensor, and an image processing circuit, detects the movement amount (Δx, Δy) of the operation surface with high precision and uses this information as the primary position information. At the same time, the angle sensor 13 detects the inclination (pitch and roll angles) and rotation (yaw angle) of the fingertip and uses this information as angle information. Combining this highly accurate position information and angle information enables image distortion correction and accurate stitching. While a gyro sensor alone can be used as the angle sensor 13, a 6-axis motion sensor combining a 3-axis gyro and a 3-axis acceleration sensor is more preferable.

[0041] On the other hand, in an embodiment that does not include the optical mouse sensor 14, position detection cannot depend on the optical mouse sensor 14, so a six-axis motion sensor that combines a three-axis gyro sensor and a three-axis acceleration sensor estimates both position and angle information. In this case, by combining the movement amount and angle information obtained from the six-axis motion sensor, the movement trajectory and attitude of the scanner can be recorded in chronological order and used for image stitching processing. Based on this information, the photographing position and posture of the image taken by the area sensor 11 are recorded in chronological order for each image.

[0042] (c) Optical mouse sensor 14 The scanner of this embodiment also functions as an optical mouse. For this purpose, an optical mouse sensor 14 is provided inside the main body 1. Any conventionally known optical mouse sensor can be used as this optical mouse sensor 14. For example, it is as follows. (1) Light source (LED) Generally, infrared LED or visible LED (wavelength around 650 nm) (2) Image sensor (area-type CMOS or dedicated sensor) High-speed camera (high frame rate CMOS) operating at 1000-6000 fps A low resolution is fine (e.g. 30x30 pixels) (3) DSP or ASIC (dedicated to image processing) Movement vectors are calculated in real time using autocorrelation and pattern matching processing of captured images. (4) Lens system Unlike the area sensor 11 of a normal scanner, it has a shallow depth of field (focal length of about a few mm)

[0043] In this type of optical mouse sensor 14, an illumination light source (LED) irradiates the operating surface of a document or other object, creating a characteristic pattern that highlights the minute irregularities and patterns on the surface. An image sensor (CMOS sensor) captures images of this pattern through a lens system at a high speed of several thousand frames per second. An image processor (DSP or ASIC) compares the successive captured images and detects positional deviations of the characteristic pattern through autocorrelation processing and pattern matching processing. Based on this, a two-dimensional movement vector (Δx, Δy) is calculated. The movement vector is calculated frequently and sequentially accumulated to form a movement trajectory from the start point of operation. Unlike inertial navigation methods that use acceleration sensors, this optical surface tracking method is free from the drift phenomenon that causes errors to accumulate, enabling highly reliable position detection even during long-term operation.This scanner uses this optical mouse sensor 14 as its main high-precision position sensor, and by estimating its own position based on its movement trajectory, it is possible to obtain with high precision the position information necessary for the stitching process of scanned images.

[0044] In this scanner, the components of the optical mouse function and the scanner function can be installed independently of each other, or some of them can be shared. For example, it is possible to share components in the communication unit described below, and this invention does not exclude such sharing. However, because the main imaging components, area sensor 11 and optical mouse sensor 14, have significantly different required performance (resolution, frame rate, focal length, etc.), it is technically easier to install them independently as separate systems.

[0045] (D) Main control unit 15 Unlike conventional panoramic photography techniques or simple linked scanning, the advanced image processing at the core of this embodiment supports multi-directional stroking gestures (arbitrary shape scanning) using a fingertip, enabling the desired detection area to be controlled over a wide area with high precision. This processing is performed by the main control unit 15 provided in the scanner. Each processing unit constituting the main control unit 15 of this scanner will be described below.

[0046] (A) Image capture processing unit 16 The image capture processor 16 directly controls the operation of the area sensor 11, angle sensor 13, optical mouse sensor 14, and LED light 12 provided on the main unit 1, and acquires sensor data suitable for subsequent image integration and OCR processing. Specifically, it issues a command to the area sensor 11 to capture snapshot images at a predetermined imaging interval or movement distance in response to the user's fingertip tracing (scan path: a scanning trajectory of any shape) on the surface of the object to be scanned. Based on the position and orientation (velocity and acceleration) information obtained from the angle sensor 13 and optical mouse sensor 14, the image capture timing and exposure conditions (exposure time, gain, resolution, etc.) of the area sensor 11 are automatically corrected for scenes with significant subject blur or rotation, ensuring optimal image capture. Furthermore, these imaging conditions are automatically adjusted according to the surrounding environment and the illumination conditions from the LED light 12, enabling stable image capture that is less susceptible to changes in illuminance and shadows. The acquired image data and motion sensor data are supplied to the sensor data integration processor 17, described below.

[0047] (B) Sensor data integration processing unit 17 Image data sequentially acquired from the area sensor 11 and position and orientation data based on the position (movement distance), rotation angle, and tilt acquired from the angle sensor 13 and optical mouse sensor 14 are integrated with high precision using a sensor fusion algorithm such as a Kalman filter or a complementary filter. This allows the position and orientation at the time of taking each image snapshot to be accurately recorded, and the spatial positional relationship of the image can be precisely reproduced even in the case of irregular scanning operations including complex finger movements or hand shake.

[0048] (C) Image synthesis processing unit 18 The image synthesis processor 18 combines multiple image snapshots based on the image data supplied from the sensor data integration processor 17 (including image data from each sensor in the case of a multiple sensor array configuration), two-dimensional movement information from the optical mouse sensor 14, and angle information from the angle sensor 13, to generate a highly accurate two-dimensional composite image. The image synthesis processor 18 executes, for example, the following processes. Note that not all of these processes are necessarily required for the present invention, and appropriate selection can be made depending on the image and character reading accuracy required by the finger-worn scanner. (a) Feature point extraction and matching Feature points such as SIFT (Scale-Invariant Feature Transform) and ORB (Oriented FAST and Rotated BRIEF) are extracted from each image snapshot, and matching is performed between adjacent or overlapping images, allowing the relative positional relationship of each image to be estimated from the image content. (b) Geometric correction and digital correction of optical aberrations (image registration) The system combines movement information obtained from the optical mouse sensor 14, angle information obtained from the angle sensor 13, and the relative relationship (e.g., homography transformation) obtained by feature point matching. This allows the image to be accurately positioned on a two-dimensional coordinate system while correcting for camera shake and trapezoidal distortion. Regarding optical aberrations caused by the thin macro lens, digital correction using a lookup table or similar method ensures uniform image quality across the entire composite image. (c) Overall optimization process To align the positional relationships between multiple images, optimization techniques such as bundle adjustment are used, which suppresses the accumulation of errors even during long scans or complex paths, enabling seamless merging of wide-area 2D images. (d) Overlapping area processing When combining images, blending processes such as feathering are performed on overlapping areas to obtain a natural composite image with inconspicuous boundaries. (e) Missing area detection Areas on the scan path where image data is insufficient are detected based on the movement information from the optical mouse sensor 14, the angle information from the angle sensor 13, and the loss of image data. This makes it possible to notify the user of areas where scanning is insufficient and evaluate the quality of the final image. (f) Perspective correction and keystone correction The entire composite image is corrected to unfold onto a flat surface, even when scanned from an oblique angle or when dealing with objects on a curved surface, providing a minimally distorted 2D image suitable for OCR processing and search. (g) Image Enhancement To improve OCR accuracy, the system performs sharpening, binarization, contrast adjustment, noise reduction, etc. It also applies optical aberration correction and AI-based sharpening processing to achieve uniform, clear image quality across the entire composite image.

[0049] (E) Image compression unit 19 This scanner has the problem that if the multiple still snapshot images successively acquired by the area sensor 11 during scanning are stored as high-resolution data as is, the amount of data becomes enormous, increasing the transfer load and processing load. Therefore, in this embodiment, a dedicated JPEG compression IC chip or a dedicated circuit block on the microprocessor is provided within the main body 1, and image data compression processing is performed in real time. The image compression unit 19 reduces the data volume while appropriately adjusting the balance between image quality and compression rate based on a standard image compression method such as JPEG for each snapshot image. This allows subsequent communication and storage processes to be performed efficiently while maintaining image quality, even during continuous scanning operations.

[0050] (F) Communication module 20 The compressed image data is transmitted in real time to an external device by the communication module 20. In this embodiment, a wireless communication module 20 conforming to Bluetooth (for example, Bluetooth Low Energy or Bluetooth Classic) is used as the communication module 20, and is configured to automatically connect to a smartphone, tablet device, or the like at the same time as scanning, and sequentially transfer the acquired images. In addition, for environments that require high-resolution image transfer, low latency, and high reliability, such as business applications and high-precision OCR processing, data transfer via a wired connection via a USB cable is also possible. In this way, by supporting both wireless and wired communication methods, flexible operation is possible according to the user's usage environment and required performance.

[0051] (G) Operation unit 21 This scanner is provided with an operation unit 21 that accepts various operational inputs from the user. In the basic configuration of this scanner, taking into consideration the constraints of an ultra-compact device, it is preferable that the operation unit 21 has two types of input: power on / off and scan start / end. However, if necessary, it can also be configured to have input functions such as left click, right click, and scrolling. Furthermore, these extended operations can also be realized by using the touch panel function of an external device such as a smartphone or tablet.

[0052] The operation unit 21 can be configured in the following manner. (1) Configuration with operation button 7 A basic configuration in which a power ON / OFF button and a scan start / stop button are provided on the side or top of the main body 1 is preferable. This allows for simple operability suited to ultra-compact devices. However, it is also possible to add multiple physical operation buttons according to function, such as a left-click button, a right-click button, and a scroll button, as needed. Such physical operation buttons 7 provide a clicking sensation and tactile feedback during operation, which has the effect of improving visibility and operability.

[0053] (2) Configuration with voice recognition function The device itself is equipped with a built-in microphone, and its speech recognition engine analyzes user-generated voice commands to accept operational commands such as starting and stopping a scan, saving images, and switching applications. This hands-free operation dramatically improves information accessibility, especially for users with physical disabilities. For example, visually impaired users can intuitively operate the device using natural language commands such as "start scanning" without having to search for a physical button on the device. It also eliminates the physical barrier of pressing a button for users with physical disabilities or arthritis who have difficulty using their fingers. In this way, voice recognition is an extremely effective means of realizing universal design, enabling smooth access to information for more people, regardless of visual or physical limitations.

[0054] (3) Configuration using touch sensors or gesture recognition By providing a touch sensor (e.g., a capacitive touchpad) in a specific area of ​​the main unit 1, operations such as finger sliding, tapping, and flicking can be accepted. It is also possible to use the output of the angle sensor 13 built into the main unit 1 to recognize specific scanner movements (e.g., shaking, tilting, rotating, etc.) as gestures and convert them into operation commands. These configurations reduce the number of physical buttons and enable more intuitive or contactless operation.

[0055] The operation unit 21 may be configured to be switchable by software settings or physical switches depending on the user's usage situation or application.

[0056] (H) PC30 side configuration In this embodiment, a compressed image sent from a scanner is received by an external device such as a personal computer, tablet, or smartphone (hereinafter collectively referred to as "PC 30") and expanded to the original image before compression. OCR processing is performed based on the expanded image, and the resulting character data is used in an application on PC 30. Specifically, the resulting character data is inserted at the cursor position of the application running on PC 30. For this purpose, the PC 30 is provided with the following configuration.

[0057] (I) Image development unit 31 This is a functional block that receives compressed image data sent from the scanner and decodes (expands) it back to its original image format (e.g., bitmap format for JPEG). This process is performed in software using the computing power of the CPU or GPU of the PC 30, or in hardware using a dedicated decoder IC. The expanded image data is supplied to the OCR processing unit 32, described later, in a format suitable for high-precision OCR processing.

[0058] (K) OCR processing unit 32 The generated high-precision composite image is processed by an OCR (Optical Character Recognition) engine and converted into text information. OCR processing is usually started automatically after scanning is complete, and performs image analysis, character region detection, and character recognition in stages. Existing engines such as Tesseract and Google Vision can also be used for character recognition. The converted text is automatically sent as character input to the application cursor position specified by the scanner's mouse function before the user performs the scan operation. At this time, the scanner also functions as a mouse and keyboard compliant with the Bluetooth HID (Human Interface Device) profile, and outputs the character string obtained by OCR to the PC30 or smart device as virtual keyboard input. Specifically, the character string obtained through OCR processing is inserted into the text box or document file where the user has placed the cursor in advance. In this way, the process from capturing the image of the intended location to outputting the text to the application can be achieved automatically without the need for user operation.

[0059] (K) Input link unit 33 The input linking unit 33 is a functional block for seamlessly inserting the character data obtained by the OCR processing unit 32 into an application running on the PC 30. Specifically, the input linking unit 33 recognizes the input focus, i.e., the cursor position, of the application on the PC 30 through the mouse function of the scanner, and automatically inputs the recognized character string at that position.

[0060] The input linking unit 33 can include the following functions. (a) Character input emulation function This function is realized by the scanner acting as a virtual keyboard device via the keyboard report in the Bluetooth HID (Human Interface Device) profile. This allows character data obtained by OCR processing to be sent to the PC 30 as keyboard input, and the text is automatically entered into the input field (cursor position) of a specified application without the user having to operate the keyboard. Examples of applications that can be used include text editors, spreadsheet software, and browser input forms.

[0061] (b) Input focus synchronization control function This function prevents erroneous output or erroneous input to other windows by synchronizing the timing of the OCR output with the input field selected by the user using the mouse function prior to the scanning operation. This is achieved by utilizing the fact that the scanner operates as a mouse conforming to the Bluetooth HID profile and can grasp the cursor position on the PC 30.

[0062] (c) Output mode selection function The input linking unit 33 can select the following output modes according to the user's settings. (1) Real-time mode: The OCR results are output immediately to the PC30 application. (2) Confirmation mode: The OCR results are temporarily stored, and after the user has confirmed or edited them, they are output at any time. (3) Collaboration mode: By transferring OCR results to a specific application via the application programming interface (API) or dedicated protocol provided by the application, more advanced data collaboration or output control can be achieved.

[0063] This allows for flexible response even in business applications where stability of the output destination and prevention of input errors are required, and makes it possible to automate the entire process from image acquisition to character recognition and output to the user application.

[0064] [1-2. Operation of the First Embodiment] A series of operations from image reading using the scanner according to this embodiment to character input into an application will be specifically described with reference to the flowchart shown in FIG.

[0065] (A) Sensor data acquisition and control When a user operates the scanner attached to his / her finger to start scanning, the image capture processing unit 16 in the main body 1 directly controls the operation of the area sensor 11, angle sensor 13, optical mouse sensor 14 and LED light 12, and starts acquiring the necessary sensor data (step S1).

[0066] First, the LED lighting 12 is provided close to the area sensor 11 and appropriately illuminates the image and characters on the running surface 3, which is the object of imaging. This provides the area sensor 11 with auxiliary light for capturing images with stable image quality.

[0067] Next, the image capture processing unit 16 issues an image capture command to the area sensor 11 to capture an image (snapshot) at a predetermined capture interval or movement distance in response to the user's operation of tracing the surface of the object to be scanned (scan pass). Based on this command, the area sensor 11 sequentially captures images and characters displayed on paper or a display in contact with the sliding surface 3. The captured images are monochrome or color, and the resolution and exposure conditions are automatically adjusted by the image capture processing unit 16.

[0068] At the same time, the angle sensor 13 and the optical mouse sensor 14 constantly measure the movement of the main body 1 and estimate the attitude (rotation, tilt) from the angular velocity data detected by the angle sensor 13. The image capture position is also detected according to the position information from the optical mouse sensor 14. The image capture processing unit 16 automatically corrects the imaging timing and exposure conditions of the area sensor 11 based on the position and attitude (velocity and acceleration) information obtained from the angle sensor 13 and the optical mouse sensor 14 in scenes where the subject is significantly shaken or rotated, thereby controlling the image capture to ensure optimal image capture. The imaging conditions of the area sensor 11 (exposure time, gain, resolution, etc.) are also automatically adjusted according to the surrounding environment and the illumination conditions from the LED lighting 12, enabling stable image capture that is less susceptible to changes in illuminance and shadows.

[0069] The image data and motion sensor data thus acquired are then supplied to the sensor data integration processing unit 17 .

[0070] (a) Integration of sensor data and estimation of the position and orientation of the main body 1 The sensor data integration processing unit 17 integrates with high precision the image data of the area sensor 11 supplied from the image capture processing unit 16 and the position and orientation data based on the position (travel distance), rotation angle, and tilt acquired from the angle sensor 13 using a sensor fusion algorithm such as a Kalman filter or a complementary filter (step S2). This integration process estimates the scanner's real-time position (X, Y, Z) and orientation (roll, pitch, yaw) in three-dimensional space. This accurately records the position and orientation at the time each image snapshot was taken, allowing the spatial relationship of the images to be accurately reproduced even with irregular scanning operations, including complex finger movements and camera shake. The integrated image data and position / orientation information are then supplied to the image synthesis processor 18.

[0071] (c) Image synthesis and correction The image synthesis processor 18 combines multiple image snapshots to generate a highly accurate synthesized image based on the image data and position / orientation information supplied from the sensor data integration processor 17. This processor first extracts robust feature points from each image snapshot at high speed through feature point extraction / matching processing (step S3), and then performs feature point matching between adjacent images and overlapping images to estimate the accurate relative position / orientation relationship between each image from the image content.

[0072] Next, a geometric correction process (step S4) integrates the initial position and orientation information obtained from the angle sensor 13 with the relative relationship between images obtained by feature point matching (e.g., homography transformation), and accurately aligns the images on a common coordinate system while correcting for distortions such as camera shake and perspective distortion. Furthermore, a global optimization process (step S5) uses a method similar to bundle adjustment to consistently optimize the positional relationships between multiple images, minimizing the accumulation of errors and seamlessly combining images over a wide range, even in long-term scans or complex paths.

[0073] In addition, overlapping area processing (step S6) does not simply delete overlapping areas that occur when combining images, but blends brightness and color tones to generate a natural composite image with inconspicuous boundaries, and optimizes the amount of data by efficiently deleting redundant image data and overlapping textures. Missing area detection processing (step S7) detects areas on the scan path where image data is insufficient or where sufficient overlap is not obtained based on movement information from angle sensor 13 and missing image information, enabling feedback to the user and quality evaluation of the final composite image.

[0074] The generated composite image undergoes pre-output processing in step S8, where perspective correction and keystone correction processes are applied to the image, accurately laying out the information on a flat surface even when scanned from an angle or with a curved object, providing a distortion-free image ideal for OCR processing and image search. Finally, the image enhancement function applies image enhancement processes such as sharpening, binarization, contrast adjustment, and noise reduction to maximize OCR performance. It should be noted that, at least in this embodiment, the "pre-output processing" in step S8 preferably includes keystone correction processing (perspective correction) and image enhancement processing (sharpening, contrast correction, noise removal, etc.).

[0075] The highly accurate composite image data thus obtained is then supplied to the image compression unit 19, where the following processing is performed. (D) Compression and transmission of image data The image compression unit 19 compresses the high-precision composite image supplied from the image composition processing unit 18 to achieve efficient data processing and transmission. This is performed in real time by a dedicated JPEG compression processing IC chip or a dedicated circuit block on the microprocessor provided within the main unit 1, reducing the data volume while appropriately adjusting the balance between image quality and compression rate. This allows image quality to be maintained even during continuous scanning operations, while subsequent communication and storage processing can be performed efficiently.

[0076] The compressed image data is then sent to the communication module 20. The communication module 20 mainly provides wireless communication functionality via a Bluetooth module, and transmits image data wirelessly in real time to an external device (PC 30) such as a smartphone or tablet. In addition, when high-quality data transfer or a stable connection is required, wired data transfer means using a USB cable can also be used, providing flexible connection methods.

[0077] (E) Image expansion, OCR processing, and character input on the PC30 side 4, an external device, PC 30, is wirelessly connected to scanner main body 1. PC 30 receives data sent from the scanner and executes a series of processes to insert the data into an application as character information. First, the communication module on the PC 30 side (not shown, but performing the receiving function on the PC 30 side) receives the compressed image data transmitted from the scanner's communication module 20. Next, the image expansion unit 31 decodes (expands) the received compressed image data back to its original image format. This process is performed by software using the computing power of the CPU or GPU of the PC 30, or by hardware using a dedicated decoder IC, and the expanded image data is then supplied to the OCR processing unit 32 in a format suitable for high-precision OCR processing.

[0078] The OCR processing unit 32 uses an OCR engine to perform character recognition processing on the high-precision expanded image supplied from the image expansion unit 31, and converts it into text information. The OCR processing is usually started automatically after scanning is completed, and image analysis, character area detection, and character recognition are performed in stages (step S9). The text resulting from the conversion is then supplied to the input linkage unit 33.

[0079] The input linking unit 33 is a functional block for seamlessly inserting the character data obtained from the OCR processing unit 32 into an application running on the PC 30. The input linking unit 33 recognizes the input focus, i.e., the cursor position, of the application on the PC 30 through the mouse function provided in the scanner, and automatically inputs the recognized character string at that position.

[0080] The input linkage unit 33 executes a character input emulation function. This function allows the scanner to operate as a virtual keyboard device. The scanner is connected to the PC 30 using the Bluetooth HID protocol. The input linkage unit 33 then generates character data recognized by OCR as a keyboard report. The input linkage unit 33 transmits the keyboard report to the PC 30 (step S10). The PC 30 accepts the keyboard report as input from a physical keyboard. As a result, text is automatically entered into the input field (cursor position) of the application without the user having to perform keyboard operations.

[0081] The input coordination unit 33 also has an input focus synchronization control function to prevent character data scanned by OCR from being input to an unintended location. In processing this function, the user first uses the mouse to select an input field in the application where they want to input text prior to performing a scan. This causes a cursor to appear in the input field, making it ready to accept character input from the keyboard, i.e., it has the "input focus." The input coordination unit 33 then identifies and stores the input field selected by the user (the position where the input focus is located) as the output destination. Then, immediately before outputting the OCR character data, the input coordination unit 33 determines whether the current input focus remains in the input field of the stored output destination. The input coordination unit 33 determines whether this determination is affirmative as a condition for executing character data output. This control reliably prevents the user from selecting an input field and then accidentally moving the input focus by unintentionally clicking elsewhere, for example, to prevent erroneous input to an incorrect output destination or to another window.

[0082] Furthermore, the input linkage unit 33 has an output mode selection function, and can select an output mode such as real-time mode, confirmation mode, or linkage mode according to the user's settings. This allows for flexible response even in business applications where stability of the output destination and prevention of input errors are required, and makes it possible to automate a series of processes from image acquisition to character recognition and output to a user application.

[0083] [1-3. Effects of the First Embodiment] The fingertip-operated scanner according to this embodiment provides the following significant advantages and effects.

[0084] (1) This technology achieves groundbreaking improvements in the efficiency and flexibility of information input and scanning operations. Specific text and graphic images on paper or objects can now be selectively captured with high speed and precision, even in irregularly shaped or complexly laid-out target areas (e.g., magazine clippings, charts, logos, etc.), using intuitive, multi-directional finger-stroking operations. This allows for the direct and instant input of OCR-processed text and images directly into the cursor position of an application on an external electronic device. This intuitive "scan, click, go" operation dramatically improves the efficiency of information acquisition and input compared to the conventional method of scanning the entire document.

[0085] (2) Achieves both thinness and wide-area, high-quality image capture. The area sensor 11, combined with a high-density surface sensor, a thin macro lens 11b, and a transparent plate 11c, significantly reduces the overall thickness of the scanner while capturing clear images across a wide imaging area. In particular, the shallow depth of focus and optical aberrations resulting from the characteristics of the thin macro lens 11b can be addressed by curving and segmenting the sensor surface, incorporating a spacer layer 11d, and using digital correction (deconvolution, chromatic aberration correction, and AI-based sharpening) in the image synthesis process. This effectively suppresses blurring and distortion around the image periphery, maintaining high-resolution image quality. This simultaneously achieves thinness, high image quality, and a wide field of view, something previously difficult to achieve with thin scanners.

[0086] (3) Stable, high-precision image acquisition is guaranteed for a variety of reading objects. By attaching the transparent plate 11c to the main body using an elastic support structure or by using a flexible material, the sliding surface 3 can follow the unevenness, undulations, and inclination of the paper surface to be read, making it possible to always maintain a constant distance between the area sensor 11 and the subject. This physically absorbs focus deviations, allowing stable, high-precision image acquisition for any object surface.

[0087] (4) Significantly reduces the burden on users. The mouse scanner integrates mouse operation and scanning, eliminating the need for users to switch between the scanner and mouse, allowing for a seamless process.

[0088] (5) High portability achieved through ultra-compact and lightweight design. Thanks to its small and lightweight design that can be worn on a fingertip and its wireless connectivity with smartphones and tablets, the scanner is extremely portable and can be used easily anywhere.

[0089] (6) High-precision OCR performance and a wide range of applications. This scanner significantly improves the OCR recognition rate by applying optimal image enhancement processing in addition to highly accurate image acquisition and synthesis. This means that it is expected to be used in a wide range of practical scenarios that take advantage of its characteristics, not only for simple document scanning but also for integration with OCR translation functions, reading functions to assist the visually impaired, high-speed reading of barcodes and QR codes (registered trademarks), and election voting assistance (for example, proxy entry on ballots).

[0090] (7) The scanner according to this embodiment is configured with a ring-shaped finger insertion section 2 attached to the underside of the main body 1. This configuration allows the main body 1 and the user's finger to become one during use, improving stability during operation and enabling the scanner to slide accurately to the intended position on the target. Furthermore, the area sensor 11 and the LED light source for illumination are positioned facing the target via the slide surface 3 on the top surface of the main body 1, ensuring a consistent distance and angle relative to the target, reducing distortion and blurring during image capture. This improves the quality of the captured image and the accuracy of subsequent image processing, particularly OCR processing.

[0091] (8) If the main body 1 is provided with an openable lid 4 and the surface of the lid 4 is decorated, closing the lid 4 when not in use can protect the area sensor 11 and the sliding surface 3, reducing the risk of malfunction or dirt during transport and improving durability and reliability. In addition, the decoration can enhance the design of the exterior, and when used in combination with a mobile information terminal or notebook PC 30, it can blend in with the surroundings and increase user satisfaction.

[0092] (9) In this embodiment, instead of providing physical operation buttons 7, a voice recognition unit is provided that recognizes user voice commands, allowing operations such as starting and stopping scans and transmitting data to be performed by voice. This configuration facilitates operation even for users with limited finger mobility or in environments with poor visibility, contributing to improved usability. Furthermore, by eliminating physical buttons, the structure of the main body 1 can be simplified, made lighter, and its dust-proof and water-resistant performance can be improved, achieving the high environmental resistance required of a portable information processing device.

[0093] (10) Strengthened proof of origin and copyright protection of image data. This scanner synthesizes captured fragmented images within the scanner itself to generate a complete image data sheet. This internal synthesis process makes it possible to embed unique identification information, such as the scanner's serial number or individual identification number, into the image data as a digital watermark. The embedded identification information serves as objective evidence that the image data was generated by the user using the scanner, making it easier for users to assert copyright over the digital images they generate and providing an effective means of protection against unauthorized duplication or theft by third parties.

[0094] [2. Second Embodiment] Figure 6 is a functional block diagram showing a second embodiment of the present invention. Compared to the first embodiment (see Figures 1 and 2), the second embodiment is characterized in that OCR processing is completed within the main body 1. To this end, the scanner according to this embodiment is equipped with an FPGA (Field-Programmable Gate Array) with an internal ARM core. This FPGA is an integrated circuit that combines the design freedom of hardware circuits with the flexible software processing capabilities of a processor. In the first embodiment, the fragmentary images acquired by the area sensor 11 are transmitted to an external electronic device (such as a smartphone), and the image synthesis process and OCR process are executed using the CPU of the electronic device. In contrast, in this embodiment, the FPGA built into the scanner performs image synthesis processing, character recognition processing using Japanese OCR, and QR Code (registered trademark) decoding processing, which enables the scanner to directly output completed text data and decoded data to an external electronic device.

[0095] [2-1. Configuration of the second embodiment] The internal configuration of the main body 1 of the second embodiment is configured as follows in addition to or in place of some of the components of the first embodiment. As in the first embodiment, the main body 1 is provided with an area sensor 11 (CMOS camera), an LED light 12, an angle sensor 13, and an optical mouse sensor 14. These sensors and lights are controlled by an image capture processing unit 16 in a main control unit 15, and data is acquired. The acquired sensor data is integrated by a sensor data integration processing unit 17, and the position and orientation of the scanner are estimated. The integrated image data and position and orientation information are then constructed as a highly accurate composite image by an image synthesis processing unit 18.

[0096] The most notable feature of the second embodiment is that an OCR engine (edge ​​AI) provided within the main body 1 directly performs OCR processing on the composite image generated by the image composition processing unit 18. This OCR engine is realized by providing an SoC or MCU constituting the main control unit 15 with Edge AI processing capabilities. For example, a lightweight OCR engine optimized for a high-performance core such as an ARM Cortex-A may be installed. The data converted into character information by OCR processing is transmitted to an external device via the communication module 20. The communication module 20 used here is compatible with USB or BLE and is capable of efficiently transmitting only text data.

[0097] The main body 1 is also provided with an operation unit 21 that accepts various operation inputs from the user. The PC 30 is provided with an input linking unit 33 for receiving character data sent from the scanner and inserting it into an application. Unlike the first embodiment, image data is not transferred to the PC 30, so the PC 30 does not need to have a processing unit for image compression / decompression or image synthesis.

[0098] [2-2. Operation of the Second Embodiment] The second embodiment differs from the first embodiment in that the series of operations from reading an image with a scanner to inputting characters into an application is completed within the main body 1, except for the initial step and the final step on the PC 30 side. First, as in the first embodiment, when the user operates the scanner to start scanning, the image capture processing unit 16 controls the area sensor 11, angle sensor 13, optical mouse sensor 14, and LED lighting 12 to acquire the necessary sensor data (snapshot image and motion sensor data).

[0099] Next, the sensor data integration processing unit 17 integrates these sensor data and estimates the real-time position and orientation of the scanner. Subsequently, the image synthesis processing unit 18 combines multiple image snapshots based on the integrated image data and position and orientation information to construct a highly accurate composite image. This is the main operation of the second embodiment. The OCR engine (Edge AI) in the main body 1 directly performs OCR processing on the highly accurate composite image generated by the image synthesis processing unit 18 and converts it into text information. This OCR processing is performed in real time or near real time, utilizing the Edge AI processing capabilities of the main control unit 15.

[0100] The character data obtained by the OCR process is sent directly to the communication module 20 without going through the image compression unit 19. The communication module 20 transmits this character data to an external device (PC 30) via USB or BLE.

[0101] On the PC 30 side, the character data sent from the scanner is received by the communication function of the PC 30. The input linkage unit 33 directly receives this character data and automatically inputs the recognized character string into the input focus (cursor position) of the application running on the PC 30. In this case, the input linkage unit 33 utilizes the mouse function of the scanner to seamlessly realize character input into the PC 30 through a character input emulation function, an input focus synchronization control function, and an output mode selection function.

[0102] 2-3. Effects of the Second Embodiment According to the second embodiment, in addition to the effects of the first embodiment, the following particularly significant effects can be obtained. (1) Significant reduction in the load on the PC30 and expansion of supported devices Because advanced image analysis, including OCR processing, is completed within the main unit 1, the PC 30 does not need to perform image extraction, complex image processing, or run an OCR engine. This reduces resource consumption, such as the CPU and memory, of the PC 30, making it easy to apply the scanner to lighter client environments such as smartphones and tablets. In other words, this configuration achieves high-speed image processing and recognition processing without placing a load on the CPU of the connected smartphone or PC 30. As a result, the scanner functions as a more high-performance, self-contained intelligent device, providing stable performance that is not dependent on the performance of the connected device.

[0103] (2) Improved security and communication efficiency Because only the text data is sent without transferring the original scanned image data to an external device, the risk of image data leaks is reduced, improving security. Furthermore, because the volume of text data is significantly smaller than that of image data, bandwidth consumption is reduced, significantly improving the efficiency of wireless communication. This improves response speeds in environments where real-time performance is required.

[0104] (3) Improved overall system responsiveness Because the entire process, from scanning to OCR and text transmission, is performed within the scanner, frequent transmission of image data to and from the external PC 30 is reduced, minimizing processing delays throughout the system, providing users with a smoother, more responsive user experience.

[0105] (4) Provide independent OCR functionality Because OCR can be performed by the scanner alone, basic character recognition functions can be used even if specific OCR software is not installed on the PC30, making it easier for users to use the OCR function.

[0106] 3. Third Embodiment [3-1. Configuration of the third embodiment] 7 is a schematic diagram showing a third embodiment of the present invention, illustrating a scanner housed in a finger cot-shaped holder 8. In the first and second embodiments, the scanner is attached to a user's finger via a ring-shaped finger insertion section 2, whereas this embodiment is characterized in that a finger cot-shaped holder 8 that can be attached to a finger is used as the finger insertion section 2.

[0107] The holder 8 of this embodiment is a cylindrical or bag-shaped member made of an elastic or flexible resin material and is attached so as to fit snugly around the user's finger. This resin material is preferably a material such as silicone rubber or thermoplastic elastomer (TPE), which provides an excellent fit and anti-slip effect to the finger, and may also be selected from those with antibacterial and stain-resistant properties. The main body 1 is housed inside the finger-cot-like holder 8, and is preferably provided with a fixing structure (e.g., friction-increasing processing, a fitting portion, etc.) to prevent the main body 1 from slipping within the holder 8. This positions the main body 1 so that it is in stable contact with the pad (underside) of the user's finger, allowing the user to faithfully move the scanner with natural finger movements.

[0108] An opening 9 is provided on the underside of the holder 8, i.e., the area corresponding to the sliding surface 3 of the main body 1, so that the scanner can come into contact with the surface to be read and perform a scanning operation. Alternatively, a transparent member that is transparent to visible light and infrared light may be attached instead of the opening 9. With this configuration, the sliding operation can be performed without impairing the accuracy of image reading and mouse position detection by the area sensor 11 and the optical mouse sensor 14.

[0109] Additionally, multiple ventilation openings 9 are formed on the top surface (the back of the finger side) of the holder 8. This ventilation structure prevents the user's fingertips from getting sweaty even during extended use, allowing the device to be worn comfortably.

[0110] The internal configuration of the main body 1 (area sensor 11, angle sensor 13, optical mouse sensor 14, main control unit 15, image compression unit 19, communication module 20, etc.) can be applied to either the first embodiment or the second embodiment, or a combination of them, and the main feature of this embodiment is its mounting form.

[0111] The attachment and detachment of the main body 1 to the holder 8 can be achieved by a snap-in method that takes advantage of the elasticity or flexibility of the holder 8, which allows for easy attachment and detachment without the need for tools. To prevent the main body 1 from falling off, a hole for passing a strap through may be provided in part of the holder 8, or a processing that makes it easy to grip to assist in attachment and detachment may be provided.

[0112] [3-2. Operation of the Third Embodiment] The operation of the scanner of this embodiment is basically the same as that of the first or second embodiment, except for the mounting mode. The user wears the scanner by inserting their finger into the resin finger cot-like holder 8. At this time, the main body 1 is stably positioned on the pad side of the finger due to the fixed structure inside the holder 8, and its sliding surface 3 is exposed to the outside or can be touched through the opening 9 of the holder 8 or a transparent member.

[0113] When a user performs a scanning operation, the holder 8 and main body 1 move together as a unit with the movement of the finger, and the sliding surface 3 scans the surface of the object to be scanned. This sense of unity allows the finger movement to be faithfully transmitted to the scanner, enabling highly accurate scanning and mouse movement detection. The opening 9 or transparent member of the holder 8 allows the area sensor 11 of the main body 1 to properly capture the object and the optical mouse sensor 14 to accurately detect the movement position. Even when used for a long period of time, the ventilation openings 9 on the top surface of the holder 8 ensure airflow between the fingers and the holder 8, effectively preventing the fingertips from getting sweaty and reducing discomfort to the user.

[0114] 3-3. Effects of the Third Embodiment According to the third embodiment, in addition to the respective effects of the first and second embodiments, the following particularly significant effects can be obtained. (1) Excellent stability, ease of operation, and ease of use when worn Unlike a finger ring, the finger cot-like holder 8 is highly adaptable to the shape and thickness of the finger, making it easier for more users to wear, and reducing the feeling of pressure on the finger, improving comfort during long-term use. The fixed structure inside the holder 8 prevents the main body 1 from slipping off the finger when worn, maintaining a sense of unity, so that the subtle movements of the finger are faithfully transmitted to the scanner, realizing high-precision and intuitive operation. In particular, the ventilation openings 9 on the top surface of the holder 8 and the use of antibacterial and stain-resistant materials effectively prevent the fingertips from getting sweaty and, by considering hygiene, allow for continuous use without discomfort, which greatly contributes to improving work efficiency and usability.

[0115] (2) Improved protection and durability of the main body 1 Since the main body 1 is housed inside the resin holder 8, the main body 1 is effectively protected from external shocks, falls, dust, etc., improving the durability of the entire product. This reduces the risk of breakage during everyday use.

[0116] (3) Versatility and stable operability Since the sliding surface 3 side of the holder 8 is an opening 9 or a transparent member, stable operation on a variety of surfaces is maintained without compromising the scanner's core functions of high-precision scanning and mouse movement detection performance.

[0117] (4) Ease of attachment and detachment and safety The snap-in type that takes advantage of the elasticity or flexibility of the holder 8 allows for easy attachment and detachment of the main body 1 without the need for tools. In addition, by providing a hole for a strap or processing that makes it easy to grip as needed, the grip when attached can be improved, preventing unexpected falling off during use or carrying, and contributing to improved safety.

[0118] (5) Flexible wearing methods As described above, the third embodiment provides a flexible wearing method that differs from conventional ring-shaped wearing parts, and by fitting naturally to the user's finger, more stable and comfortable operability can be achieved.

[0119] 4. Other Embodiments The present invention is not limited to the above-described embodiment, and other embodiments such as those described below are possible.

[0120] [4-1. Reading assistance devices for the visually impaired and partially sighted] The fingertip-operated scanner (mouse scanner) of this embodiment is used as a reading support device for the visually impaired and those with low vision. This embodiment solves the problems of conventional reading aids and provides a more intuitive and highly functional means of accessing information. [4-1-1.Configuration]

[0121] The scanner in this embodiment has a structure optimized for visually impaired people. Specifically, tactile guides (e.g., Braille or textured surfaces) are provided on the sides of the scanner body 1, allowing even visually impaired people to accurately grasp the position of their fingers and operate the scanner appropriately. Furthermore, a rubber frame or physical boundary can be provided so that the scanning range can be felt with the fingers.

[0122] This scanner also features an audio feedback function. This allows the scanner to notify the user by voice or vibration when the scanner has properly started scanning, is in the process of scanning, or has finished scanning. For example, information such as "Scanning Started," "Scanning Completed," and "Error" can be communicated through a Bluetooth speaker or audio output device connected to a smartphone. Operation Button 7 also uses physical buttons instead of the typical touchscreen, allowing users to easily perform operations such as "Start Scan," "Start Reading," and "Next Page" without relying on visual information.

[0123] [4-1-2. Software configuration] The scanner of this embodiment has the following software configuration. (a) OCR (Optical Character Recognition) Processing: Images scanned with a scanner are converted into text data using highly accurate OCR technology. For those with low vision in particular, the converted text can be expanded in font size and with contrast adjustment functions. Furthermore, it can also accommodate special fonts such as Braille fonts and UD (Universal Design) fonts.

[0124] (i) Real-time voice reading: Text data converted by OCR is instantly read aloud using Text-to-Speech (TTS). Users have the ability to freely adjust the voice speed, volume, and pitch. In addition, a language selection function is implemented that supports multiple languages, including Japanese, English, and Chinese, and users can select the reading method according to different scanning methods, such as line by line, paragraph by paragraph, or page by page.

[0125] (c) Scan Area Guide: During scanning, the smartphone screen and voice guidance provide real-time feedback to the user, such as "The scan position is off" or "The appropriate area has not been scanned." Utilizing information from the gyro sensor and position sensor built into the scanner, the system assists in accurate scanning by indicating the appropriate scan direction and speed.

[0126] (d) Smartphone / tablet integration: Connect to a smartphone or tablet via Bluetooth or USB, and save OCR results to the cloud or share them with others. The scanned text is automatically displayed and played back in conjunction with the smartphone's voice reading app or e-book reader. It also comes with a user interface that allows users to easily perform operations such as "rewind," "pause," and "word search" while the text is being read.

[0127] [4-1-3. Application functions] This embodiment can also have the following application functions.

[0128] (a) Real-time translation: After scanning text using OCR, real-time multilingual translation is performed using cloud translation APIs such as Google Translate and DeepL. Users can specify the language to be read aloud, making it possible to, for example, have an English book read aloud in Japanese or convert a French menu into English.

[0129] (i) Braille conversion: Automatically converts text acquired by OCR into Braille format, enabling linkage with external devices such as Braille displays and Braille printers. As an option, a function can be provided that utilizes the smartphone's vibration function to transmit Braille codes to the user like Morse code.

[0130] (c) AI assistant for the visually impaired: AI analyzes the content of scanned text and provides summaries and supplementary explanations. For example, it generates appropriate answers based on the scanned content in response to user questions such as "Can you give me a three-line summary of the contents of this book?" or "Please explain the meaning of this word in detail."

[0131] [4-1-4. Specific usage scenarios] The scanner of this embodiment is useful in the following specific usage scenarios. When visually impaired people read books, they scan the pages of a book with a scanner, and the text is extracted using OCR and sent to a smartphone or tablet. The scanned content is automatically read aloud, and users can control the reading with voice commands such as "read the next line" or "read again." When a person with low vision reads a newspaper or magazine: A section of the article is scanned and the content is enlarged and displayed on the smartphone screen. The background color and font can be changed to make it easier for the user to read, and only the difficult-to-read parts can be selected and read aloud. Obtaining information on the go (in places where Braille is not available): Visually impaired people can scan bus timetables or restaurant menus with a scanner and instantly convert them into speech to understand the content. They can also scan the text on signs and advertisements and receive audio guidance on the content.

[0132] [4-1-5. Technical elements for realization] To realize this embodiment, the following technical elements are important. Hardware technology: High-resolution image sensor (a small sensor capable of high-resolution OCR), uniform LED lighting (enables stable reading even in low-visibility environments), integrated position sensor and gyro (high-precision correction of misalignment during scanning), low-power configuration that enables long-term use (battery management technology). Software technology: Real-time OCR engine (utilizing OCR technologies such as Google Tesseract and ABBYY for high-speed character recognition), AI-based context analysis and voice-to-speech optimization (not simply reading text, but providing more natural voice output that takes meaning into account), and cloud-based data storage and sharing (allowing OCR results to be stored in the cloud for later access).

[0133] [4-2. Multilingual Translation Device] This embodiment is intended to be used as a multilingual translation device for travelers and language learners, eliminating language barriers abroad and difficulties in reading comprehension when learning a language, and providing a smoother communication and learning experience.

[0134] [4-2-1.Configuration] The scanner in this embodiment has a structure optimized for travelers and language learners. Specifically, it is lightweight and compact for portability when traveling, and can even be miniaturized to keychain size. It also has a robust structure that is shock-resistant and waterproof, anticipating use in a variety of environments, such as outdoors or in restaurants.

[0135] It also features a multilingual voice feedback function that instantly plays back the translation results after translating scanned text. By optimizing accents and pronunciations for each language (e.g., US and British English, Mandarin and Cantonese Chinese), it provides a more natural-sounding voice.

[0136] The LED lighting 12 has a function that automatically adjusts brightness according to the ambient light, making it easy to scan in dark places (restaurants, street signs at night, etc.). The LED lighting 12 automatically turns off in natural light to reduce battery consumption. Operation is possible not only with manual touch buttons, but also with voice commands such as "start translation" and "read again," and operation via an app is also possible in conjunction with a smartphone.

[0137] [4-2-2. Software configuration] The scanner of this embodiment has the following software configuration.

[0138] (a) OCR (Optical Character Recognition) and Multilingual Support: Images scanned with a scanner are converted into text data using OCR technology. It supports major multilingual fonts, including Japanese, English, Chinese, Korean, French, and German, and is also equipped with a function to distinguish between vertical and horizontal writing, allowing accurate recognition of Japanese books and newspapers.

[0139] (i) Real-time translation: In cooperation with cloud translation APIs (Google Translate, DeepL, Microsoft Translator, etc.), scanned text is instantly translated. It also has an offline translation mode that saves dictionary data in advance on the device 1 so that it can be used overseas without an internet connection. It also allows users to select a translation style according to their needs, such as travel mode or learning mode.

[0140] (c) Text-to-Speech (TTS): The translation results are read aloud using text-to-speech synthesis. By applying the appropriate intonation for each language, the translation content is presented in a natural pronunciation. For language learners, the service also offers setting change functions to assist learning, such as "slow pronunciation," "native speed," and "pronounce each word separately."

[0141] (d) Scan Area Guide: This function has the ability to notify the user through on-screen or audio feedback during scanning, such as "The scan position is off" or "The characters are not being properly recognized." It utilizes the scanner's gyro sensor and position sensor to guide the user to the appropriate scan direction and speed, thereby assisting in accurate scanning.

[0142] (e) Smartphone / Tablet Integration: Connect to a smartphone or tablet via Bluetooth or USB and save OCR results to the cloud or share them with others. Translated text can be saved in a memo or dictionary app to save your learning history. It also seamlessly connects to voice translation apps such as Google Translate, Papago, and iTranslate.

[0143] [4-2-3. Application functions] This embodiment can also have the following application functions. (a) Offline translation mode: Multilingual dictionary data is stored in advance in the scanner itself1 so that translation can be performed even in environments without a cloud connection. This allows travelers to use it safely even in places without Wi-Fi. Major languages ​​(English, Chinese, Spanish, French, etc.) are prioritized.

[0144] (i) AI-based context analysis: When translating, rather than translating words literally, the system takes the context into account and translates them in a more natural and appropriate way. For example, it translates "Can I get the bill?" as "Please explain the bill." For language learners, it also provides an option to display the meaning of each word and grammar explanation.

[0145] (c) Additional features for travelers: Provides translation modes optimized for specific situations, such as "restaurant menu translation mode," "traffic sign translation mode," and "airport guide translation mode." It is also possible to add a "travel notes" function that saves translation results on the smartphone for later reference, and a "conversation assistance" function that suggests useful phrases for the traveler.

[0146] [4-2-4. Specific usage scenarios] The scanner of this embodiment is useful in the following specific usage scenarios. When traveling abroad, for example, translating restaurant menus: By scanning the menu, the translation is displayed in real time and you can also hear the translation aloud, making ordering easier. It is also possible to automatically identify the type of menu item (meat dish, fish dish, etc.) and provide related information.

[0147] Directions and sign translation: Scan signs at bus stops, train stations, airports, etc., and the appropriate translation will be displayed. For example, directions will be provided in expressions appropriate to the situation, such as "Ticket Gate" or "Exit A."

[0148] Language learners can translate books and newspapers: Scan English books with a scanner and display the translation word by word. It can also be integrated with language learning apps to save and review translation history, and has a learning support mode that automatically displays explanations of words and grammar.

[0149] [4-2-5. Technical elements for realization] To realize this embodiment, the following technical elements are important. Hardware technology: High-precision image sensor (a small sensor capable of high-resolution OCR), low-power LED lighting12 (enables accurate scanning even in dark places), integrated position sensor and gyro (corrects misalignment during scanning), flash memory for offline data storage (storing translation history and dictionary data).

[0150] Software technology: Real-time OCR engine (utilizing OCR technologies such as Tesseract OCR and ABBYY OCR), seamless integration with translation APIs (Google Translate, DeepL, Microsoft Translator, etc.), AI-based free translation and context analysis (providing more natural translations rather than literal translations), cloud integration and offline mode (storage of translation history and offline dictionary function).

[0151] [4-3. Document copying support for courts, parliaments, and public institutions] In this embodiment, the present invention is utilized as a document copying support device in courts, parliaments, and other public institutions, with the aim of efficiently digitizing highly confidential documents, ensuring information security, and improving document management.

[0152] [4-3-1.Configuration] The scanner of this embodiment has the following configuration, which is optimized for an environment in which confidential information is handled. (a) Confidentiality function: To scan highly confidential documents, the system implements a setting that allows data to be saved only on the local device (smartphone or tablet) without being sent to the cloud. Wireless communication connections such as Bluetooth and Wi-Fi are restricted, allowing offline use. The acquired image data is automatically encrypted to prevent unauthorized access.

[0153] (i) Tamper-proof function: Scanned data is digitally signed to ensure it has not been tampered with. Furthermore, a timestamp is added to ensure that the data remains valid as evidence. (c) Security authentication function: The device will be equipped with an option to require fingerprint or passcode authentication to view or output scanned data. It will also include a function to record detailed user authentication logs, in anticipation of use by public institutions and courts.

[0154] (d) High-definition image sensor and improved character recognition accuracy: The device is equipped with a high-resolution sensor that can clearly scan small characters and seals, such as those on contracts and judgments. Even when scanning at an angle, the automatic correction function ensures accurate text capture. (e) Energy-saving configuration for long-term use: A low-power configuration will be adopted, assuming long-term use in meetings, courts, etc.

[0155] [4-3-2. Software configuration] The scanner of this embodiment has the following software configuration. (a) OCR (Optical Character Recognition) and format analysis: Accurately processes official documents such as court documents and parliamentary minutes with OCR to convert them into text data. Equipped with format recognition technology, it can scan contracts, litigation documents, resolutions, and other documents while preserving their complex layout. It also comes with an OCR dictionary that supports special fonts (such as fonts specific to court decisions, old characters, and official document fonts).

[0156] (i) Automatic classification and tagging: Scanned documents are automatically classified and tagged with tags such as "contract," "judgment," "evidence," and "minutes." Specific keywords within the documents (such as "plaintiff," "defendant," and "evidence number") are automatically extracted, improving searchability.

[0157] (c) Document digitization and management: Scanned data is automatically saved in a specified folder, enabling integration with public document management systems (DMS). File formats can be selected, including PDF, DOCX, and TXT, and a mode is included that links the OCR data with the original image after scanning, allowing the original and text to be displayed simultaneously.

[0158] (d) Access restrictions and log management: Scan data viewing and editing history is recorded in detail to prevent unauthorized access. When shared, the data is encrypted to prevent unauthorized third parties from viewing it. Scan data can also be set to be automatically deleted after a certain period of time as a measure to protect information.

[0159] [4-3-3. Application functions] This embodiment may also have the following special application functions. (a) Document review by voice reading: The system will be equipped with a function that reads the contents of scanned documents aloud using text-to-speech (TTS). This will allow judges and lawyers to review document contents even while on the move. Options such as "reading out only important keywords" and "playing back only the parts of a specific person's speech" will also be available. (i) Highlight and memo function: After scanning, you can mark specific parts of the text data and add comments. For example, you can record annotations such as "This part will be used as evidence" or "This is the plaintiff's argument." By connecting to a smartphone or tablet, you can streamline the organization of evidence in meetings and court cases.

[0160] (c) AI-based summarization and related information suggestions: AI will summarize lengthy minutes and court documents, allowing users to quickly understand the content. Implementing functions such as "searching for similar past cases to this judgment" and "presenting related legal provisions" will make information searches more efficient. (d) Original document verification and anti-counterfeiting: The system compares the electronic data of scanned documents with the original paper copy to automatically detect tampering or discrepancies. This system can be used by public institutions and courts as an authentication function for important documents.

[0161] [4-3-4. Specific usage scenarios] The scanner of this embodiment is useful in the following specific usage scenarios. Scanning evidence in court: Lawyers scan evidence documents with a scanner and digitize them. They save them in a searchable format (PDF + OCR text) and add tags and notes to quickly organize the necessary information. Parliamentary minutes management: Scan minutes, extract and organize key points, color-code and categorize them by speaker, visualize the flow of discussion, and use the text-to-speech function to check the contents even when you're on the go. Storage and management of official documents: Approval documents from government agencies are scanned and stored electronically. To prevent document tampering, electronic signatures and encryption are applied, retention periods are managed in accordance with the law, and unnecessary documents are automatically deleted.

[0162] [4-3-5. Technical elements for realization] To realize this embodiment, the following technical elements are important. Hardware technology: High-definition image sensor (accurately scans fine print and seals on official documents), anti-tampering technology (module that can apply electronic signatures and timestamps), enhanced security features (module that supports fingerprint authentication and passcode lock). Software technology: Advanced OCR engine (dictionaries and algorithms that can handle legal and technical terms), AI-based document analysis and summarization (efficiently shortening long minutes and court decisions), integration with electronic document management systems (DMS) (API integration with existing systems of public institutions).

[0163] [4-4] Barcode and QR Code (registered trademark) scanning in store and warehouse management As a fourth embodiment of the fingertip-operated scanner (mouse scanner) according to the present invention, we will explain its use as a barcode / QR code (registered trademark) scanning device in store and warehouse management. This embodiment aims to improve the efficiency of inventory management, prevent erroneous orders, and reduce work time.

[0164] [4-4-1.Configuration] The scanner in this embodiment has a configuration suitable for use in warehouses and stores. (a) Small and lightweight design: The pen-shaped and ring-shaped design makes it easy to carry, and the ergonomic design reduces strain on workers even when used for long periods of time. (i) Image sensor for high-speed scanning: Equipped with a highly sensitive image sensor capable of scanning 30 to 60 times per second. Equipped with an advanced image correction function that can accurately read even dirty or rubbed barcodes, improving reading accuracy.

[0165] (c) LED lighting for dark places: Equipped with LED lighting 12 with automatic dimming function to enable accurate reading even in warehouses and dark places. It is possible to switch between lighting for close range (diffused light) and long range (directional light) to accommodate various situations. (d) Vibration feedback function: The system is equipped with a function that notifies the user by voice and vibration when a barcode or QR code (registered trademark) is successfully read. This allows the user to know whether the scan was successful or not without visual confirmation, improving efficiency when scanning large quantities of data. (e) Robustness and waterproofing: The device will have a durable structure that can withstand drops and impacts. By meeting IP65 or higher waterproof and dustproof specifications, it will be able to withstand harsh environments such as outdoor warehouses and refrigerated warehouses.

[0166] [4-4-2. Software configuration] The scanner of this embodiment has the following software configuration. (a) Real-time barcode and QR code (registered trademark) recognition: Supports major barcode standards (JAN, EAN, UPC, Code128, Code39, etc.). It can also read two-dimensional codes such as QR code (registered trademark), DataMatrix, and PDF417, and even scans at an angle can be accurately analyzed using the image correction function. Even for damaged or dirty barcodes, the error check function complements the information, improving the reading rate.

[0167] (a) Linkage with inventory and sales management systems: The scanned data is immediately sent to the inventory management system (WMS: Warehouse Management System) and POS (Point of Sale) system, enabling real-time updates of inventory and linking with functions such as price checks, product description displays, and order processing in stores. (c) Local data storage and offline mode: The offline mode allows users to work in warehouses or remote locations without a network connection. Temporary data is stored on the device and automatically synchronized with the system when a network connection is restored.

[0168] (d) Voice feedback and notification function: Provides a function to notify product information by voice (e.g., "Product A, 5 units remaining in stock"). In addition, it will add an alert function for "products that need to be ordered" or "products with an approaching expiration date," and will also be able to send notifications to smartphones or wearable devices after scanning a barcode. (e) Integrated management of multiple scanner devices: When multiple scanners are used in the same warehouse, a system that allows for centralized management will be introduced. This system will share the scan data of each worker in real time, prevent duplicate updates to inventory data, and also include a history management function for who scanned which product and when. [4-4-3. Application functions] This embodiment can also have the following application functions. (a) Integration with voice assistants: Warehouse workers can give instructions using voice commands (e.g., "Scan the item on the next shelf"). Detailed information about the scanned item can also be provided via voice. (i) Hands-free mode: Scanning can be done hands-free while wearing the fingertip scanner. It also features a "continuous mode" that automatically scans at regular intervals, enabling smooth inventory taking.

[0169] (c) Collaboration with AR (Augmented Reality): Scanned product information is displayed in AR, allowing workers to check it in real time through smart glasses, etc. For example, information such as "stock quantity," "order status," and "storage location" can be visually displayed. (d) Geofencing (location information sharing): Records the area in the warehouse where the product was scanned and tracks its storage location. Using GPS and beacons, it guides workers to efficiently pick up products.

[0170] [4-4-4. Specific usage scenarios] The scanner of this embodiment is useful in the following specific usage scenarios. In-store price check and ordering: When an employee scans a product's barcode, the system automatically displays an ordering screen if the product is low in stock, simplifying the ordering process. When checking prices, the system also displays current sale prices and discount information. Inventory count in warehouse: Scan the barcode of each product using a scanner and update the inventory count in real time. Automatically aggregate the scanned data within a certain period of time to generate an accurate inventory count report. Product traceability management: Scan barcodes containing lot numbers, production dates, and expiration dates to record traceability information. This is useful in industries where expiration dates and expiry dates are important, such as food and pharmaceuticals, and automatically identifies products that need to be disposed of, streamlining disposal processes.

[0171] [4-4-5. Technical elements for realization] To realize this embodiment, the following technical elements are important. Hardware technology: High-sensitivity barcode scanner (1D / 2D compatible, recognizes even scratches and dirt), dustproof and waterproof configuration (IP65) (intended for use in outdoor warehouses and food warehouses), vibration feedback function (instant notification of scan success / failure), power-saving configuration and fast charging support (for long-term work). Software technology: Real-time OCR and barcode analysis (compatible with JAN, EAN, QR (registered trademark), and DataMatrix), correction of unknown barcodes using AI (can read even damaged or dirty barcodes), integration with inventory management and POS systems (API integration with WMS and ERP), AR display and smart glass integration (improving warehouse work efficiency).

[0172] [4-5] Learning support in a smartphone-free environment at elementary schools and exam venues This embodiment is intended to be used as a learning support device in environments where smartphones are prohibited, such as elementary schools and examination halls, and aims to allow students to maintain concentration and prevent cheating while enjoying the convenience of digital tools.

[0173] [4-5-1.Configuration] The scanner in this embodiment has the following configuration, which is optimized for use in an environment where smartphones are prohibited. (a) Offline operation: Equipped with a standalone mode that can be used without a smartphone or tablet. Scan data is temporarily saved in the main unit 1 and can be later transferred to an external device such as a PC 30. (b) Highly visible display: For environments where smartphones cannot be connected, the device is equipped with a small E-Ink display or LCD display to display the scan results directly. It also offers functions for enlarging the display and changing fonts, making it suitable for younger children and students with visual challenges.

[0174] (c) Simple button operation: Functions such as "scan," "read aloud," and "dictionary search" are intuitively controlled using physical buttons, making it easy for even younger children to use. (d) Durability and safety: The device will have a shock-resistant and waterproof structure (equivalent to IP54) that is resistant to drops and water. It will also be designed with safety in mind, such as rounded corners, in anticipation of use by elementary school students. (e) Long-lasting battery life: The device will have a low-power consumption configuration that allows it to be used for several days on a single charge. It will be compatible with both AAA batteries and rechargeable batteries, making it easy to manage in schools.

[0175] [4-5-2. Software configuration] The scanner of this embodiment has the following software configuration. (a) OCR (Optical Character Recognition) and dictionary search function: Scanned characters are recognized by OCR and search results are instantly displayed from the dictionary built into the main unit 1 or from a linked dictionary database. It links with an easy-to-understand Japanese dictionary for elementary school students, an English-Japanese / Japanese-English dictionary, and a Japanese dictionary, allowing users to select the appropriate dictionary level for each grade level. (i) Text-to-speech (TTS): Text obtained by OCR is read aloud using text-to-speech (TTS). A "slow reading" mode is provided for lower grades, and native pronunciation can be selected for English learning.

[0176] (c) Exam mode (anti-cheating): The device will be equipped with an "exam mode" to prevent cheating at schools and exam venues. The dictionary and translation functions will be automatically disabled during the exam, and the scan history will be recorded so that the exam supervisor can check it. It will also be possible to set a time limit for use during the exam. (d) Saving learning history: The history of words and sentences looked up by students can be saved in the scanner body 1 so that they can be reviewed later. In the teacher management mode, it is possible to check the usage history of each student and understand their learning progress. (e) Hiragana conversion and ruby ​​display: When difficult kanji characters are scanned, furigana (ruby) is automatically added. This makes it a learning tool that is easy to use even for younger children.

[0177] [4-5-3. Application functions] This embodiment may also have the following special application functions. (a) Collaboration with voice assistants: When a child asks, "What does this word mean?", the system will respond with a dictionary search result. When learning English, if a child says, "Tell me how to pronounce this word," the system will play back the native pronunciation. (i) Kanji and English word writing support: Provides a function that allows users to trace scanned words on the touch panel of the device 1 and check the correct stroke order. For English learning, it is equipped with a spell check function and is also capable of handwriting recognition. (c) Digital note function: Scanned words and sentences are automatically saved and can be reviewed later. Teachers can manage the data and analyze students' understanding. (d) Special Needs Education Features: A mode that provides both audio and visual support for children with reading and writing difficulties. The screen background color and font size can be changed to accommodate learning disabilities such as dyslexia.

[0178] [4-5-4. Specific usage scenarios] The scanner of this embodiment is useful in the following specific usage scenarios. Elementary school students use it as a dictionary: When they scan a word they don't understand during Japanese class, the meaning is displayed with furigana and an audio explanation is also provided. The history is also saved so that they can review it later. Scan English textbooks and check pronunciation: Scan English sentences and check the pronunciation with a native speaker. Search the meaning of English words in the dictionary and see example sentences. Use in exam halls: Exam mode restricts unnecessary functions to prevent fraudulent use. Scanned content is recorded in history, allowing the supervisor to review it later. Use in special needs education: Children with reading disabilities can use the scanner to scan text and listen to it aloud, making it easier for students who have difficulty reading to understand the learning content.

[0179] [4-5-5. Technical elements for realization] To realize this embodiment, the following technical elements are important. Hardware technology: Low power consumption (long-term use), shock-resistant and waterproof (robust enough to withstand use by children), small display (visibility that allows results to be checked without the need for a smartphone). Software technology: Real-time OCR (high-precision recognition of handwritten and printed characters), offline dictionary search (can be used without an internet connection), speech synthesis (gentle reading voice for children), exam mode (anti-cheating function). [Explanation of symbols]

[0180] 1...Main unit 2...Finger insertion area 3...Sliding surface 4…Lid 5...Chain 6. Ring 7...Operation buttons 8...Holder 9...Opening 11...Area sensor 11a...area-type image sensor 11b...Thin macro lens 11c...Transparent plate 11d...Spacer layer 12...LED lighting 13...Angle sensor 14...Optical mouse sensor 15...Main control unit 16...Image capture processing unit 17...Sensor data integration processing unit 18...Image synthesis processing unit 19...Image compression section 20...Communication module 21...Operation unit 30...PC 31...Image development section 32...OCR processing section 33...Input linking section

Claims

1. A fingertip-operated scanner having a main body attached to a fingertip, an area sensor provided in the main body that scans the surface of an object to acquire image information; a position sensor for detecting movement information of the main body; and an angle sensor for detecting angle information of the main body; an LED light source provided on the main body for illuminating a scan area; an operation unit of the scanner provided on the main body; an image synthesis unit that synthesizes the plurality of images acquired by the area sensor and information detected by the position sensor and the angle sensor to generate a synthesized image; a communication module that transmits the composite image or processing information based on the composite image to an external electronic device; the area sensor includes a high-density surface sensor, a thin macro lens laminated on a lower surface of the high-density surface sensor, and a transparent plate laminated on a lower surface of the macro lens; A fingertip-operated scanner.

2. The image synthesis unit a sensor data integration and motion tracking unit that synchronizes and integrates a plurality of image data successively acquired by the area sensor and real-time movement information and posture information of the main body detected by the position sensor and the angle sensor; an image alignment and combination unit that estimates a relative positional relationship between the plurality of images based on the integrated information and combines the images while correcting image distortion; an output pre-processing unit that performs output pre-processing including distortion correction, perspective correction, keystone correction, and image enhancement processing on the composite image combined by the image alignment and combination unit; 2. The fingertip scanner according to claim 1, further comprising:

3. an input linking unit that inputs the composite image to a user application of the external electronic device; 2. The fingertip-operated scanner according to claim 1, further comprising: an optical character recognition (OCR) processing unit provided in the external electronic device and configured to recognize character information included in image data input to the input linkage unit.

4. an OCR processing unit that recognizes character information included in the composite image; 2. The fingertip-operated scanner according to claim 1, further comprising an input linkage unit that directly inputs character information recognized by the OCR processing unit into a cursor position of a user application of the external electronic device.

5. a ring-shaped finger insertion portion to be worn on a user's finger on the underside of the main body; a sliding surface on the top surface of the main body that is movable in contact with the object to be read; 5. The fingertip-operated scanner according to claim 3, wherein the area sensor and the LED light source are provided facing the object to be read from the sliding surface.

6. 6. The fingertip-operated scanner according to claim 5, wherein the main body has a cover that opens and closes to cover the sliding surface, and the surface of the cover is decorated.

7. 5. The fingertip-operated scanner according to claim 3, further comprising a wired communication means using a USB cable in addition to the communication module.

8. The fingertip-operated scanner according to claim 3 or 4, characterized in that the image data or character information transmitted via the communication module is in a format that can be used by one or more of the following applications (a) to (d) running on an external electronic device: (a) A translation application that performs translation processing (b) A visual support application that converts recognized text information into audio and outputs it. (c) A code reading application that recognizes barcodes or QR codes (registered trademarks) (d) Input assistance applications that assist with filling out certain digital forms

9. a mouse operation unit provided on the main body; an optical mouse sensor that generates mouse pointing information based on the position sensor and the mouse operation unit; a communication module for transmitting mouse pointing information based on the optical mouse sensor to an external electronic device; 2. The fingertip scanner of claim 1, further comprising:

10. 2. A method for performing image synthesis using a fingertip-operated scanner according to claim 1, comprising: a step of synchronizing and integrating a plurality of pieces of image data sequentially acquired by the area sensor with real-time movement information and attitude information of the main body detected by the position sensor and the angle sensor; estimating a relative positional relationship between the plurality of images based on the integrated information, and combining the images while correcting image distortion; performing pre-output processing on the combined image, including distortion correction, perspective correction, keystone correction, and image enhancement; An image synthesis method comprising:

11. An information input method using the fingertip operation type scanner according to claim 3 or 4, An information input method comprising: scanning a predetermined area on the surface of an object; recognizing character information from the scanned image; and automatically inputting the character information at a cursor position of a user application in an external electronic device.

Images