Method for reading indicia from a display of a mobile device

Abstract

This application relates to methods for reading indicia from a display of a mobile device, wherein a method of scanning and reading indicia from a display is specifically disclosed. The method can utilize the light sensor of the mobile device to improve the quality of reading indicia such as barcodes from the display. The scanner illuminates the mobile device in order to increase the backlight intensity and obtain a high-contrast image on the display, and then stops the illumination in order to scan the high-contrast image without specular reflections caused by the scanner illumination. In one embodiment, the scanner illuminates the mobile device for a period of time to reach the maximum luminance level that the mobile device is capable of emitting on the display in the period of time before stopping the illumination. In another embodiment, the scanner detects whether the change in luminance before stopping the illumination is greater than a predefined level.

Description

Technical Field

[0001] This invention relates to a method for scanning and reading markings on a display. More specifically, the invention utilizes the light sensor of a mobile device to improve the quality of reading markings such as barcodes on a display. Background Technology

[0002] Generally, traditional point-of-sale systems have the ability to read barcodes from the screens of mobile devices such as smartphones. Use cases may include reading mobile coupons, membership cards, or other similar applications. However, reading barcodes from smartphone screens can be difficult with traditional 1D or 2D barcode scanners because these scanners are not optimized for these applications. For example, lighting from the scanner may cause reflections on the screen, which can make barcode reading difficult. Furthermore, the backlight intensity on smartphones can be dim, which can make reading difficult due to a lack of contrast.

[0003] Therefore, there is a need to improve the ability of traditional scanners to read tags from the screens of mobile devices such as smartphones. Summary of the Invention

[0004] Accordingly, in one aspect, the invention includes a method for scanning and reading markings on a display associated with a mobile device such as a smartphone. The markings may be barcodes.

[0005] In an exemplary embodiment, a method for scanning and reading markers on a scanner includes: activating a scanner's scanning mode to capture an image presented on a display associated with a mobile device, and illuminating the mobile device for a period of time to increase the backlight intensity of the display in terms of brightness. When the period ends, the illumination of the mobile device stops, and the scanner continues to capture images presented on the display. At this time, the scanner attempts to decode the captured image.

[0006] In one aspect of the exemplary embodiment, the mobile device is a smartphone, and the display is the smartphone's screen. Furthermore, the barcode can be an image presented on the display, and the captured image can include one or more data frames. Additionally, when the mobile device is illuminated by the scanner, this causes the mobile device's light sensor to increase the backlight intensity compared to the backlight intensity before the mobile device was illuminated.

[0007] In another aspect of the exemplary embodiment, a scanning mode is initiated in response to: receiving a command from a host application; decoding a programming barcode; activating hardware on the scanner; receiving signals from an inertial sensor on the scanner; and / or processing an image captured by the scanner; and wherein, when the scanning mode is enabled, the scanner performs the steps of illuminating, stopping illumination, capturing an image, and attempting to decode the captured image. Furthermore, the scanner performs image processing on the scanner's sensor video stream to decode the captured image.

[0008] In another exemplary embodiment, a method for scanning and reading markers from a scanner includes: activating a scanning mode of the scanner to capture images presented on a display associated with a mobile device, and attempting to decode a first set of captured images during a first time period while the scanner is not illuminating the mobile device. If decoding of the first set of captured images during the first time period is unsuccessful: the scanner takes the steps of: (1) illuminating the mobile device for a second time period, (2) attempting to decode a second set of captured images during a first portion of the second time period, (3) if attempting to decode the second set of captured images is unsuccessful, continuing to illuminate the mobile device to increase the backlight intensity of the display associated with the mobile device to a brightness level; (4) when the second time period ends, stopping the illumination of the mobile device, and then capturing a third set of captured images; and (5) attempting to decode the third set of captured images.

[0009] If decoding the third set of captured images is unsuccessful, the method repeats steps (1) through (5) disclosed in the previous paragraphs to another second time period.

[0010] In another aspect of this exemplary embodiment, if the images captured in the first, second, or third group are successfully decoded, the decoded images are transmitted to a computer. Otherwise, the scanning is terminated at the end of the timeout period or when prompted by the user.

[0011] In another aspect of this exemplary embodiment, the illumination of the mobile device causes the light sensor of the mobile device to increase the backlight intensity compared to the backlight intensity before the mobile device was illuminated.

[0012] In another aspect of this exemplary embodiment, a scanning mode is initiated in response to: receiving a command from a host application; decoding a programming barcode; activating hardware on the scanner; receiving a signal from an inertial sensor on the scanner; and / or processing an image captured by the scanner; wherein, when the scanning mode is enabled, the scanner performs the steps of illuminating, stopping illumination, capturing an image, and attempting to decode the captured image.

[0013] In yet another exemplary embodiment, a method for scanning and reading markers on a scanner includes: activating a scanning mode of the scanner to capture images presented on a display associated with a mobile device, and attempting to decode a first set of captured images for a first time period while the scanner is not illuminating the mobile device.

[0014] If decoding the first set of captured images fails in the first time period, the scanner takes the following actions: (1) illuminates the mobile device; (2) attempts to decode the second set of captured images; (3) if decoding the second set of captured images fails, detects whether the backlight intensity of the display has increased beyond a predetermined level compared to the previous measurement; (4) if the backlight intensity has increased beyond the predetermined level, stops illuminating the mobile device and captures the third set of captured images presented on the display; and (5) attempts to decode the third set of captured images.

[0015] If decoding the third set of captured images is unsuccessful, the method repeats steps (1) through (5) disclosed in the previous paragraphs.

[0016] In another aspect of this exemplary embodiment, if the images captured by the first, second, or third group are successfully decoded, the decoded images are transmitted to a computer.

[0017] In addition, the scan will terminate at the end of the timeout period or when prompted by the user.

[0018] In another aspect of this exemplary embodiment, the method further includes: continuing to illuminate the mobile device if the backlight intensity of the display has not increased beyond a predetermined level compared to another previous measurement.

[0019] In another aspect of this exemplary embodiment, the method further includes performing image processing on the sensor video stream of the scanner to detect changes in the backlight intensity of the display. The image processing determines the changes in backlight intensity. As an example, the changes in backlight intensity may be based on a measurement of the average value of pixels in each frame.

[0020] The foregoing illustrative invention and other exemplary objects and / or advantages of the invention, as well as the manner in which the illustrative invention, objects and / or advantages are achieved, are further explained in the following detailed description and accompanying drawings. Attached Figure Description

[0021] Figure 1 The image depicts scanners and mobile devices graphically, including a scenario where a scanner is scanning a barcode on a mobile device.

[0022] Figure 2 A flowchart depicting an exemplary embodiment of a method for scanning and reading markings on a display associated with a mobile device.

[0023] Figure 3 A flowchart depicting another exemplary embodiment of a method for scanning and reading markers on a display associated with a mobile device.

[0024] Figure 4A , 4B A flowchart depicting yet another exemplary embodiment of a method for scanning and reading markings on a display associated with a mobile device. Detailed Implementation

[0025] This invention includes a method for scanning and reading markings on a display. The invention utilizes the light sensor of a mobile device to improve the quality of reading markings on the display. In typical applications, the mobile device may be a smartphone, and the markings may be barcodes.

[0026] A barcode reader (or barcode scanner) is an electronic device that can read and output printed barcodes to a computer. It may consist of a light source, a lens, and a light sensor that converts optical pulses into electrical pulses. The barcode reader may include decoder circuitry that analyzes the image data of the barcode provided by the sensor and sends the barcode content to the scanner's output port.

[0027] Barcodes are used to encode information as machine-readable visual patterns. They are used for a variety of reasons, including tracking product, price, and inventory levels, and for centralized recording in computer software systems. There are two types of barcodes—linear and 2D.

[0028] Most barcode scanners consist of three distinct parts: an illumination system, sensors, and a decoder. Generally, a barcode scanner "scans" the black and white elements of a barcode by illuminating it with light; the code is then converted into matching text. More specifically, sensors in a barcode scanner detect reflected light from the illumination system and generate a sensor stream reflecting the captured image of the barcode. This sensor stream is sent to the decoder. The decoder processes frames from the sensor stream, verifies the barcode, and converts it into text. This converted text can then be delivered by the scanner to a computer software system that holds a database of the manufacturers, costs, and quantities of products sold.

[0029] This invention utilizes a light sensor found on the front of many smartphones on the market. A conventional use of this sensor is to detect the amount of light in a room so that the smartphone's backlight intensity can be adjusted accordingly to save battery. In brightly lit environments, the backlight intensity can be at full brightness to maximize readability, and in darker environments, the backlight intensity can be dimmed to save battery. A further example of this use is typically an automatic brightness setting on smartphones that allow these operations. Automatic brightness can be turned on by default, and this feature helps save battery life.

[0030] This invention describes a scanning mode that, when enabled, turns on the scanner's illumination for a period of time, e.g., 500 milliseconds. During this period, the scanner can fill the smartphone's light sensor with light, thus tricking the smartphone into thinking it is in a well-lit environment. This action causes the smartphone to increase the backlight intensity on its display to the maximum brightness level that the mobile device can emit during this time, thereby maximizing the contrast of the barcode on the screen. At this time, the scanner can turn off the illumination and read the high-contrast barcode on the smartphone screen.

[0031] The scanning mode can be enabled in various ways, such as via commands from a host application, programmed barcodes, dual-trigger pulls, hardware buttons on the scanner, gestures recognized by inertial sensors (e.g., side-to-side swings), or by recognizing a smartphone in the field of view via image processing technology. Once the scanning mode is enabled, the scanner may attempt to decode a few frames without any illumination. If this decoding is unsuccessful, the scanner may turn on its illumination at full power for a set duration. This duration may be hard-coded to a predetermined duration, or it may be kept on until the software detects a change in screen brightness, at which point the illumination can be turned off. If no change in brightness is detected, it will also be turned off during the timeout period. The scanner may attempt to decode frames while the illumination is on, but it will most likely have a better chance of reading the code when the illumination is off and the backlight is fully illuminated. In this case, the screen may have high contrast, which can improve the scanner's ability to decode the captured frames (i.e., images). Another possible operating mode could be to keep the illumination on for the set duration and then briefly turn it off (for only a few milliseconds) while a few frames are attempted to be decoded. This cycle will continue until the code is decoded. The flashing is barely visible to the user and to the light sensor, but it allows us to continuously fill the light sensor with light while simultaneously attempting to decode the captured image. This is perhaps the optimal embodiment of the invention. The invention assumes that automatic brightness setting is enabled on the target device.

[0032] In summary, this invention includes at least two scenarios. In one scenario, while the smartphone is being illuminated, the scanner can actually perform image processing on the sensor's video stream to detect when there has been a change in backlight intensity. The scanner can achieve this by averaging the values ​​of pixels in the frames, and when the average value increases by more than n from one or more previous frames, the scanner will stop illuminating the smartphone and begin decoding. Another scenario might be: the scanner illuminates the smartphone for a period of time and then simply assumes it has tricked the smartphone into increasing its backlight intensity. At this point, it will stop illuminating, attempt to decode a few frames, and repeat the loop if unsuccessful.

[0033] Figure 1 The illustration shows a scanner 102 and a mobile device 104, where the scanner 102 is scanning a barcode on the mobile device 104. The scanner 102 includes illumination, sensor, and decoder functions. Figure 1 The illustration also depicts mobile device 106 in a fully backlit environment, resulting in a high-contrast display. Alternatively, mobile device 108 is illustrated in a dimmed backlit environment, resulting in a low-contrast display. Compared to the low-contrast display shown in mobile device 108, scanner 102 may have improved scanning performance for high-contrast displays such as those shown in mobile device 106. Mobile devices 104, 106, and 108 determine the contrast level based on the amount of light received by a light sensor. The sensor is depicted on mobile device 108. Light sensors may be located on many smartphones.

[0034] In the exemplary embodiment, Figure 2 The diagram illustrates a flowchart of a method for scanning and reading markings on a display associated with a mobile device. It begins with a scanner (step 202), initiating a scanning mode to capture an image on the mobile device's display. (Step 204) The scanner then illuminates the mobile device for a period of time. (Step 210) As previously discussed, when the mobile phone's light sensor is filled with light, the backlight intensity of the display can increase in brightness compared to the backlight intensity before the mobile device is illuminated, eventually increasing to the maximum brightness level that the mobile device can emit during that time period. (Step 212) When that time period ends, during the period of high-contrast display as illustrated on mobile device 106, the scanner stops illuminating the mobile device and then captures an image from the display. The scanner then attempts to decode the captured image. (Step 214) By scanning the display image immediately after the transition from a high-brightness environment to a low-brightness environment and capturing an image from a high-contrast display, the probability of successfully decoding the captured image is increased.

[0035] In another exemplary embodiment, Figure 3The diagram illustrates a flowchart of a method for scanning and reading markings on a display associated with a mobile device. It begins with the scanner (step 302), initiating a scanning mode to capture images from the mobile device's display. (Step 304) The scanner then attempts to capture a first set of images within a first time period without illuminating the mobile device. (Step 306) During these attempts, the handheld scanner may be in motion, so the scanner can scan a few frames to optimize the likelihood of obtaining a focused image.

[0036] If the captured image is successfully decoded before the end of the first time period, the decoded image is sent to the computer. If the captured image is not successfully decoded before the end of the first time period, the scanner illuminates the mobile device for a second time period. (Step 310) Next, while the mobile device is illuminated, the scanner attempts to decode a second set of captured images scanned during the first part of the second time period. (Step 311) If decoding is successful, the decoded image is sent to the computer. If decoding is unsuccessful, the method continues.

[0037] Since the scanner is illuminating the mobile device (including its light sensor), the backlight intensity can continue to increase until it reaches a level comparable to the backlight intensity before the mobile device was illuminated, and can reach the maximum brightness level that the mobile device can emit during the second time period. (Step 312) When the second time period ends, the scanner stops illuminating the mobile device. In the absence of illumination, the scanner captures a third set of images from the display and then attempts to decode the third set of captured images. (Step 316) If the third set of captured images is successfully decoded, the decoded images are sent to the computer.

[0038] If decoding of the third set of captured images fails, the scanner illuminates the mobile device for another second time period. The method then repeats the steps of the second time period, including steps 310, 311, 312, and 316.

[0039] In yet another exemplary embodiment, Figure 4A and Figure 4B The diagram illustrates a flowchart of a method for scanning and reading markings on a display associated with a mobile device. Starting with the scanner (step 402), a scanning mode is initiated to capture an image on the mobile device's display (step 404).

[0040] The scanner then attempts to decode the first set of captured images in the first time period without illuminating the mobile device. (Step 406) During these attempts, the handheld scanner may be in motion, so the scanner can scan a few frames to optimize the probability of obtaining a focused image.

[0041] If the first set of captured images is successfully decoded before the end of the first time period, the decoded images are sent to the computer. If the first set of captured images is not successfully decoded before the end of the first time period, the scanner illuminates the mobile device for a second time period. (Step 410) Next, while the mobile device is illuminated, the scanner attempts to decode the second set of captured images scanned after the mobile device is illuminated. If decoding is successful, the decoded images are sent to the computer (Step 411).

[0042] If decoding fails, the scanner monitors changes in the backlight intensity level of the mobile device's display. The scanner determines whether the backlight intensity has increased to a level greater than a predetermined level compared to previous measurements. (Step 415) If the predetermined level is reached, the scanner stops illuminating the mobile device. The scanner then captures images from the display and subsequently attempts to decode a third set of captured images. (Step 416) If decoding is successful, the decoded images are sent to the computer.

[0043] If decoding fails, repeat steps 411, 414, 415, and 416.

[0044] If the backlight intensity of the display has not increased beyond a predetermined level compared to another previous measurement in step 415, the scanner continues to illuminate the mobile device until the backlight intensity is greater than the predetermined level compared to the previous measurement (step 412).

[0045] Figure 3 A method is disclosed in which the decision to attempt to decode a captured image is based on certain time periods. Figure 4A and 4B A method is disclosed in which the decision to attempt to decode a captured image is based on changes in the brightness level of the display. Those skilled in the art will recognize that it is composable. Figure 3 as well as Figure 4A and 4B The method described in [the document / article].

[0046] Typical embodiments of the invention have been disclosed in the specification and / or drawings. The invention is not limited to such exemplary embodiments. The use of the term "and / or" includes any and all combinations of one or more of the associated listed items. The drawings are schematic representations and therefore not necessarily drawn to scale. Unless otherwise stated, specific terms have been used in a general and descriptive sense and not for limiting purposes.

Claims

1. A method for a scanner, comprising: The scanner is activated in scanning mode to capture the first set of images that are first presented on the display associated with the mobile device without illuminating the mobile device. Attempt to decode the first set of images captured without illuminating the mobile device; If the first set of images is not successfully decoded, the scanner illuminates the mobile device until the scanner detects a change in brightness on the display associated with the mobile device. Stop illuminating the mobile device, and then... After the mobile device is de-illuminated, the scanner captures further images of the display; and Attempt to decode the further image.

2. The method according to claim 1, wherein: When the mobile device is illuminated, the scanner attempts to read and decode the second set of images.

3. The method according to claim 1, wherein illuminating the mobile device is stopped when the given time period for illuminating the mobile device ends.

4. The method according to claim 1, wherein: The captured images consist of one or more data frames; and The method includes: Determine whether the average pixel value in a frame captured while the mobile device is illuminated increases by more than a given amount n compared to the average pixel value in the previous frame captured while the mobile device was illuminated; and Stop illuminating the mobile device when the average value has risen above a given amount n.

5. The method according to claim 1, wherein: The captured images consist of one or more data frames; and This method involves repeating the following loop: Illuminate the mobile device; and then Stop illuminating the mobile device while attempting to decode a frame.

6. The method of claim 5, comprising continuing the loop until decoding is successful.

7. The method according to claim 1, wherein, The scanning mode is initiated in response to the following: Receive commands from the host application; Decoding programming barcodes; Activate the hardware on the scanner; Receive signals from the inertial sensor on the scanner; and / or Process the images captured by the scanner; When the scanning mode is enabled, the scanner performs the steps of illuminating, stopping illumination, capturing an image, and attempting to decode the captured image.

8. A barcode scanner, comprising: An illumination system configured to illuminate the scanner's field of view; Sensor, wherein the sensor is configured to capture frames of image data; and The decoder is configured to decode the barcode into text, wherein the scanner is further configured to: The scanner is activated in scanning mode to capture a first set of images that will be displayed on the screen associated with the mobile device without illuminating the mobile device. Attempt to decode the first set of images captured without illuminating the mobile device; If the first set of images is not successfully decoded, the scanner illuminates the mobile device until the scanner detects a change in brightness on the display associated with the mobile device. Stop illuminating the mobile device, and then... After the mobile device is de-illuminated, the scanner captures further images of the display; and Attempt to decode the further image.

9. The scanner according to claim 8, wherein, The scanner is also configured to: Image processing is performed on the first set of images and the further images to detect whether the brightness of the display has increased.

10. The scanner according to claim 8, wherein, The increase in the brightness of the display is detected by measuring the average value of pixels in each frame of the first set of images and in each frame of the further images.

11. The scanner according to claim 10, wherein, The scanner is configured to: Determine whether the average pixel value in each frame of the first set of frames captured when the mobile device is illuminated has increased by a given amount compared to the average value of the previous frame captured when the mobile device is illuminated; and When it is determined that the average value of the pixels has risen above a given amount, the illumination of the mobile device is stopped.

12. The scanner according to claim 8, wherein, When the mobile device is illuminated, the scanner attempts to capture and decode a second set of images.

13. The scanner according to claim 8, wherein, After a given period of time has elicited an illumination of the mobile device, the illumination of the mobile device is stopped.

14. The scanner according to claim 8, wherein, The scanning mode is activated in response to the following: Receive commands from the host application; Decoding programming barcodes; Activate the hardware on the scanner; and / or Receive signals from the inertial sensor on the scanner; and / or Process the images captured by the scanner; When the scanning mode is enabled, the scanner performs the steps of illuminating, stopping illumination, capturing an image, and attempting to decode the captured image.

Images