Reading device and reading method
The reading device with a main and external terminal setup ensures consistent product recognition by controlling image transfer and recognition processing, addressing inconsistent notifications and improving accuracy.
Patent Information
- Application Number
- JP2021040233
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-03-12
- Publication Date
- 2024-11-20
- Estimated Expiration
- 2041-03-12
AI Technical Summary
Existing reading devices struggle with inconsistent product recognition when using both code symbol and general object recognition methods, leading to incorrect audio notifications and difficulty in determining which method failed.
A reading device configuration with a main terminal and an external terminal connected via a connector, where the determination output unit controls image transfer and recognition processing to ensure exclusive control, preventing out-of-order recognition by detecting connection status and waiting for completion of one recognition method before proceeding with the other.
Ensures consistent and accurate product recognition by preventing out-of-order processing, allowing the device to clearly notify the operator of successful product identifications and avoid confusion.
Smart Images

Figure 0007587447000001 
Figure 0007587447000002 
Figure 0007587447000003
Abstract
Description
[Technical field]
[0001] FIELD OF THE DISCLOSURE The present invention relates to a reading device and a reading method. [Background technology]
[0002] Conventionally, in a sales data registration process for registering information on products to be sold, multiple methods are sometimes used in combination to identify products (for example, Patent Document 1). The product identification methods used in combination are, for example, product identification by code symbol recognition such as a barcode, and product identification by general object recognition (object recognition). Product identification by code symbol recognition is performed by decoding a code symbol included in an image (captured image) output by an imaging unit. Product identification by general object recognition is performed by calculating a feature amount of the product's appearance based on the captured image, and querying a database for this calculated feature amount. When code symbol recognition or general object recognition is completed, the reading device notifies the user of successful recognition by a sound such as a beep.
[0003] When the above two methods are used together, basically, products with code symbols are identified by code symbol recognition, and products without code symbols, such as fruits and vegetables, are identified by general object recognition. However, at the time of capturing an image, the reading device does not know whether the captured product is a product that will be identified by code recognition or general object recognition. For this reason, the reading device performs recognition processing using both methods using the captured image, and uses the output of the successful one.
[0004] Here, decoding of a code symbol (e.g., a barcode) is completed in, for example, about 30 msec, whereas product identification by generic object recognition takes about 100 msec. Therefore, if a second product recognized by a code symbol is imaged after a first product identified by generic object recognition is imaged, the latter second product may be recognized first unless special processing is performed. In this case, if recognition of the former first product fails, there will be no audio notification for the first product, but there will be an audio notification for the second product, but it is difficult for the operator to tell from the audio which product recognition failed. For this reason, various efforts (exclusion control) have been made to prevent product recognition from being out of order, but further improvements are desired. Summary of the Invention [Problem to be solved by the invention]
[0005] The problem to be solved by the present invention is to provide a reading device and a reading method that can easily implement exclusive control when generic object recognition is also used. [Means for solving the problem]
[0006] The reading device of the embodiment includes an imaging unit, a first device having a first recognition unit that recognizes a product from an image output by the imaging unit, and a second device separate from the first device and having a second recognition unit that recognizes a product from the image using a method different from that of the first recognition unit, and is equipped with a connection unit provided in the first device that connects to the second device so as to be able to communicate with the second device and outputs whether or not the connection is present, a transfer unit that transmits the image to the first recognition unit and also transmits the image to the second recognition unit via the connection unit, and a determination unit that, when connection of the second device by the connection unit is detected, instructs the transfer unit to stop transferring the image after the transfer unit transfers the image until a predetermined signal is received from the second recognition unit. [Brief description of the drawings]
[0007] [Figure 1]FIG. 1 is a block diagram showing an example of a basic configuration of a checkout device according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing characteristic configurations and data flows of the main terminal and external terminal according to the embodiment. [Diagram 3] FIG. 3 is a flowchart illustrating an example of a process flow performed by the decision output unit according to the embodiment. [Figure 4] FIG. 4 is a flowchart illustrating an example of the flow of an exclusive control process performed by the decision output unit according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0008] An embodiment of a reading device will be described in detail below with reference to the accompanying drawings. Fig. 1 is a block diagram showing an example of a basic configuration of a checkout device 1 according to this embodiment. The checkout device 1 includes a POS (Point Of Sales) terminal 100, a main terminal 101, and an external terminal 10.
[0009] The POS terminal 100 is used for registering sales data, for example, in a supermarket or a convenience store. The main terminal 101 is an example of a first device. The main terminal 101 is realized, for example, as a so-called vertical scanner, and is used by connecting it to the POS terminal 100. The external terminal 10 is an example of a second device. The external terminal 10 is realized, for example, as a PC (Personal Computer), and is used by connecting it to the main terminal 101. In this embodiment, the main terminal 101 and the external terminal 10 constitute an example of a reading device.
[0010] The main terminal 101 is a device that accepts input of information about products (product information), recognizes products by reading information from images obtained by imaging (captured images), and accepts product information manually input by an operator.
[0011] The main terminal 101 is connected to the POS terminal 100 and the external terminal 10 via connection I / Fs (Interfaces) 175 and 176 so as to be able to transmit and receive data.
[0012] The main terminal 101 includes an operation display unit 104 for receiving operations from a store clerk, a second display unit 109 for providing information to customers, a lighting unit 112, an imaging unit 113, and an audio output unit 114. The operation display unit 104 includes a first display unit 106 having a touch panel 105 laminated on its surface, a keyboard 107, a card reader 111, and the like.
[0013] The main terminal 101 also includes a microcomputer (control unit 160) that executes information processing. The control unit 160 is configured by connecting a CPU (Central Processing Unit) 161, which executes various arithmetic processing and controls each unit, a ROM (Read Only Memory) 162, and a RAM (Random Access Memory) 163 via a bus. The ROM 162 stores a program 164 that the CPU 161 executes.
[0014] The lighting unit 112, the imaging unit 113, the audio output unit 114, etc. are connected to the CPU 161 via various buses and I / Fs. The CPU 161 controls the operation of each of these units (the lighting unit 112, the imaging unit 113, and the audio output unit 114).
[0015] In addition, an operation display unit 104 for store clerks and a second display unit 109 for customers are connected to the CPU 161 via the connection I / Fs 175 and 176. The operation display unit 104 and the second display unit 109 are connected to the CPU 61 of the external terminal 10 via the connection I / F 176 and the connection I / F 65 of the external terminal 10.
[0016] The first display 106 is, for example, a liquid crystal display, and mainly displays various information for the operator. The touch panel 105 accepts various operations according to the display contents of the first display 106. The second display 109 is, for example, a liquid crystal display, and is mainly used to display various information for customers.
[0017] The audio output unit 114 is an audio circuit or a speaker for generating a preset sound such as a beep. The lighting unit 112 is configured, for example, by arranging a plurality of LEDs (Light Emitting Diodes) in a line. The lighting unit 112 emits pulsed light in response to a light emission control signal input from the control unit 160, and illuminates the imaging area of the imaging unit 113. Note that the lighting unit 112 may be configured with other light emitting elements.
[0018] The imaging unit 113 is an imaging device including an image sensor (also called an imaging element) such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor) and optical elements such as a lens. The imaging unit 113 sequentially reads frame images into the RAM 163 at a frame rate (the number of frames read per second) based on an imaging control signal.
[0019] Moreover, the imaging unit 113 outputs data of the image obtained by imaging (that is, the captured image) to the control unit 160 of the main terminal 101 and the control unit 60 of the external terminal 10 in sequence.
[0020] An operator who operates the main terminal 101 holds a commodity over the main terminal 101. At this time, the imaging unit 113 captures an image of the commodity. The image data obtained by imaging is used for code symbol recognition in the main terminal 101, and is also output to the external terminal 10 and used for general object recognition.
[0021] The external terminal 10 performs generic object recognition (object recognition) based on captured image data input from the main terminal 101. Note that, in the present embodiment, as an example, a case will be described in which a target article (target article) for generic object recognition is a commodity, but the target article may be an article other than a commodity.
[0022] The external terminal 10 includes a microcomputer (control unit 60) that executes information processing. The control unit 60 includes a CPU 61 that executes various arithmetic processing and controls each unit, a ROM 62, and a RAM 63 that are connected via a bus.
[0023] A hard disk drive (HDD) 64 is connected to the CPU 61 of the external terminal 10. The HDD 64 stores a program 641 and various files. The program 641 stored in the HDD 64 is executed in whole or in part by the CPU 61 when the external terminal 10 is started up. One example of the program 641 stored in the HDD 64 is a program 641 for processing product sales data.
[0024] Examples of files stored in the HDD 64 are a product master file 642 and a dictionary file 643. The product master file 642 and the dictionary file 643 are distributed from the store computer SC at a predetermined timing, such as before the store opens, and are stored in the HDD 64. The HDD 64 also stores a sales master file and the like. The HDD 64 may be any storage device as long as it is a storage device configured with other storage media, such as an SSD (Solid State Drive).
[0025] The product master file 642 is a file that stores information used for sales registration for each product displayed and sold in a store, and stores product information such as product name and price in association with a product code that enables the product to be identified. The dictionary file 643 stores reference values of feature quantities of multiple products in association with the product code.
[0026] A communication I / F (Interface) 25 for executing data communication with a store computer or the like installed in the back yard of the store is connected to the CPU 61 of the external terminal 10 via an input / output circuit. The HDD of the store computer stores data for the product master file and data for the dictionary file to be distributed to the external terminal 10.
[0027] Furthermore, a connection I / F 65 that enables data transmission and reception between the CPU 61 of the external terminal 10 and the main terminal 101 is connected to the connection I / F 65. The main terminal 101 is connected to the connection I / F 65.
[0028] The POS terminal 100 is operated by an operator (a store clerk) and registers products and settles accounts for one transaction based on the output of the main terminal 101.
[0029] 2 is a diagram showing characteristic configurations and data flows of the main terminal 101 and the external terminal 10. The main terminal 101 further includes a connector 171. The control unit 160 of the main terminal 101 functions as an imaging control unit 611, a barcode recognition unit 612 (an example of a first recognition unit), and a determination output unit 613. The control unit 60 of the external terminal 10 functions as an object recognition unit 601 (an example of a second recognition unit).
[0030] The connector 171 is an example of a connection unit that communicatively connects to the second device and outputs the presence or absence of the connection, and communicatively connects the main terminal 101 and the external terminal 10. More specifically, the connector 171 is a terminal into which a cable is inserted to connect the connection I / Fs 175 and 176 to the connection I / F 65 in a wired manner. The inserted / unplugged state of the cable to the connector 171 is provided to the determination output unit 613.
[0031] The imaging control unit 611 is an example of a transfer unit that transmits an image output by the imaging unit 113 to the barcode recognition unit 612 (first recognition unit) and also transmits the image to the object recognition unit 601 (second recognition unit) via the connector 171 (connection unit). The imaging control unit 611 controls the imaging unit 113 and outputs data of an image (captured image) captured by the imaging unit 113 to the barcode recognition unit 612. Furthermore, when the object recognition unit 601 is connected via the connector 171, the imaging control unit 611 also outputs the same captured image data as that output to the barcode recognition unit 612 to the object recognition unit 601.
[0032] The barcode recognition unit 612 is an example of a first recognition unit, and decodes, for example, a barcode included in the captured image to obtain a product code. Note that in this embodiment, the barcode recognition unit 612 is described as an example of the first recognition unit, but in practice, the first recognition unit may decode not only barcodes but also other code symbols such as two-dimensional codes.
[0033] The object recognition unit 601 is an example of a second recognition unit, and performs a general object recognition process (so-called object recognition process). In one example of the general object recognition process, the object recognition unit 601 first extracts a feature amount indicating an external feature of the product 300 from a captured image of the product 300 captured by the imaging unit 113. In extracting the feature amount, the object recognition unit 601 first acquires the captured image of the imaging unit 113 via the connector 171, and detects an image area representing the product 300 included in the captured image.
[0034] In detecting the image area, the object recognition unit 601 extracts contour lines and the like from an image obtained by binarizing the captured image. Alternatively, the object recognition unit 601 detects the commodity 300 based on the difference between the contour lines extracted from a background image prepared in advance and the contour lines extracted from the captured image. Here, the background image is an image captured by the imaging unit 113 of a background that does not include the commodity 300. In this way, the object recognition unit 601 detects the image area in which the commodity 300 is depicted by extracting contour lines that are not included in the contour lines of the background image from the captured image.
[0035] When the image area is detected, the object recognition unit 601 extracts feature amounts of the product 300 from the image area of the product 300 detected from the captured image. The feature amounts are information that parameterizes external features of the product 300, such as its shape, surface color, pattern, and unevenness.
[0036] When the feature amounts are extracted from the captured image of the product, the object recognition unit 601 compares the extracted feature amounts with reference values of the feature amounts stored in the dictionary file 643, and calculates the similarity between each reference value and the feature amount of the product 300. Then, the object recognition unit 601 determines, for example, that the product code associated with the reference value having the highest similarity to the feature amount of the product 300 is the product code of the product 300, and ends the generic object recognition process.
[0037] The barcode recognition unit 612 outputs the product code obtained by decoding the barcode to the judgment output unit 613. Furthermore, the object recognition unit 601 outputs the product code obtained by comparing the feature amount with a reference value stored in the dictionary file 643 to the judgment output unit 613. Here, the product code may be one that can identify a product, that is, one that is unique for each product of the same type or size.
[0038] If the dictionary file 643 does not contain a reference value that corresponds to the feature amount, the object recognition unit 601 outputs a code indicating that there is no match.
[0039] The judgment output unit 613 is an example of a judgment unit that, when a connection of an external terminal 10 (second device) via the connector 171 (connection unit) is detected, instructs the imaging control unit 611 to stop transferring images after the imaging control unit 611 (transfer unit) transfers an image until a predetermined signal is received from the object recognition unit 601 (second recognition unit).
[0040] Specifically, the determination output unit 613 refers to the product master file 642 to acquire information (product information) on a product corresponding to the product code obtained from the code symbol. The determination output unit 613 also refers to the product master file 642 to acquire information (product information) on a product corresponding to the product code obtained by the generic object recognition process. The main terminal 101 then outputs the product information acquired by the determination output unit 613 to the POS terminal 100. The POS terminal 100 registers the product information input from the main terminal 101 as sales data.
[0041] As described above, the connector 171 provides the judgment output unit 613 with the plugged-in / unplugged state of the cable. More specifically, the judgment output unit 613 can detect the plugged-in / unplugged state of the cable to the connector 171 by transmitting signals between the connector 171 and the judgment output unit 613. Furthermore, the judgment output unit 613 judges whether the external terminal 10 is connected to the main terminal 101 by this plugged-in / unplugged detection.
[0042] When the external terminal 10 is connected to the connector 171, the determination output unit 613 determines that the barcode recognition by the main terminal 101 and the general object recognition process by the external terminal 10 are used in combination. In this case, the determination output unit 613 performs an exclusive process. The exclusive process is a process in which the recognition process of a certain captured image is not performed until the recognition process (barcode recognition process and general object recognition process) for another captured image is completed. In this exclusive process, the determination output unit 613 may act on the imaging control unit 611 to stop imaging by the imaging unit 113 until the recognition process of a certain captured image is completed, or may act on the imaging control unit 611 not to output the captured image input from the imaging unit 113 to others (the barcode recognition unit 612 and the object recognition unit 601).
[0043] 3 is a flowchart showing an example of a process flow performed by the determination output unit 613 according to the embodiment. The determination output unit 613 waits for detection of insertion / removal of a cable into the connector 171 (No in step S1 and No in step S2), and when a cable is inserted into the connector 171 (Yes in step S1), it determines that the main terminal 101 and the external terminal 10 are connected (step S2) and starts an exclusive control process (see FIG. 4, described later) (step S3). Moreover, when a cable is not inserted into the connector 171 (No in step S1) but is pulled out from the connector 171 (Yes in step S4), the determination output unit 613 determines that the external terminal 10 is not connected to the main terminal 101 (step S5) and ends the exclusive control process (step S6).
[0044] 4 is a flowchart showing an example of the flow of the exclusive control process performed by the decision output unit 613 according to the embodiment. The decision output unit 613 instructs the imaging control unit 611 to wait without outputting the next captured image data until a "cancel instruction" is received each time captured image data is output (step S11), and the process proceeds to step S12.
[0045] In step S12, if a code is input from the barcode recognition unit 612 (Yes in step S12), the decision output unit 613 waits for the code to be returned from the object recognition unit 601 (No in step S13). If a code is returned in step S13 (Yes in step S13), the decision output unit 613 outputs a signal indicating a release instruction to the imaging control unit 611 (step S14), and the process returns to step S12. Upon receiving the release instruction in step S14, the imaging control unit 611 resumes the process, i.e., outputs the next captured image data to the barcode recognition unit 612 and the object recognition unit 601.
[0046] In step S12, if there is no input from the barcode recognition unit 612 (No in step S12), the process waits for input of a code from the object recognition unit 601 (step S15). If there is no input of a code from the object recognition unit 601 in step S15 (No in step S15), the decision output unit 613 returns the process to step S12.
[0047] In step S15, if a code is input from the object recognition unit 601 (Yes in step S15), the decision output unit 613 advances the process to step S14.
[0048] Here, an additional explanation will be given regarding the end of the exclusive control process in step S6 in Fig. 3. Specifically, in step S6, the instruction from the determination output unit 613 to the imaging control unit 611 in step S11 is cancelled. That is, upon receiving this instruction, the imaging control unit 611 outputs the captured image data to the barcode recognition unit 612 every time the captured image data is input from the imaging unit 113. This causes the main terminal 101 to enter an operating mode in which product information is read only by barcode recognition.
[0049] In this way, according to this embodiment, when a product code is obtained by the barcode recognition unit 612, the next captured image can be recognized by waiting for the completion of processing in the object recognition unit 601 of the captured image data captured at the same timing. Therefore, according to this embodiment, it is possible to avoid a problem of losing consistency in the order, such as the barcode recognition unit 612 outputting the recognition result of the next captured image data while the object recognition unit 601 is recognizing the captured image data captured earlier.
[0050] In this embodiment, the above-mentioned exclusive control process can be performed in response to detection of cable insertion / removal. Therefore, according to this embodiment, it is possible to provide a reading device and a reading method that can easily perform exclusive control when using both code symbol recognition and general object recognition (object recognition).
[0051] In this embodiment, an example in which a reading device is configured by a main terminal 101 and an external terminal 10 has been described, but the implementation is not limited to a reading device configured in this way, and can be applied to any reading device that performs code symbol recognition and general object recognition using separate terminals.
[0052] In addition, in this embodiment, an example has been described in which the external terminal 10 is an information processing device with a general computer configuration such as a personal computer, but this is not limited to this in implementation, and in implementation, the external terminal 10 may also be the POS terminal 100.
[0053] Although several embodiments of the present invention have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]
[0054] 1. Check-out device 10 ... External terminal (second device, constitutes the reading device) 25 ...Communication I / F 60...control unit, 61...CPU, 62...ROM, 63...RAM 64...HDD 65 …Connection I / F 100…POS terminal 101...Main terminal (first device. Constituting a reading device) 104: operation display unit, 105: touch panel, 106: first display, 107: keyboard 109: 2nd display, 111: card reader 112: lighting section, 113: imaging section, 114: audio output section 160: control unit, 161: CPU, 162: ROM, 163: RAM 171...Connector (connection part) 175,176...Connection I / F 300…Products (goods) 601...Object recognition unit (second recognition unit) 611: Imaging control unit; 612: Barcode recognition unit (first recognition unit); 641... program, 642... product master file, 643... dictionary file [Prior art documents] [Patent documents]
[0055] [Patent Document 1] JP 2016-091360 A
Claims
1. A reading device including: a first device including an imaging unit; a first recognition unit that recognizes a commodity from an image output by the imaging unit; and a second device that is separate from the first device and includes a second recognition unit that recognizes a commodity from the image by a method different from that of the first recognition unit, a connection unit provided in the first device, configured to connect to the second device so as to be capable of communicating with the first device and to output a presence or absence of the connection; a transfer unit that transmits the image to the first recognition unit and also transmits the image to the second recognition unit via the connection unit; a determination unit that instructs the transfer unit to stop transferring the image after the transfer unit transfers the image, when the connection of the second device by the connection unit is detected, until a predetermined signal is received from the second recognition unit; A reading device comprising:
2. The first recognition unit is a code recognition unit that obtains information by decoding a code symbol included in an image output by the imaging unit, The second recognition unit is an object recognition unit that obtains, from the image output by the imaging unit, a feature amount of an object appearing in the image.
2. The reading device according to claim 1.
3. When the determination unit receives a predetermined signal from the second recognition unit after the transfer unit transfers the image, the determination unit instructs the transfer unit to resume the transfer of the image.
3. A reading device according to claim 1 or 2.
4. A reading method executed by a reading device including: a first device including an imaging unit and a first recognition unit that recognizes a commodity from an image output by the imaging unit; a second device that is separate from the first device and includes a second recognition unit that recognizes a commodity from the image by a method different from that of the first recognition unit; and a connection unit that is provided in the first device, connects to the second device so as to be able to communicate with the second device, and outputs whether or not the connection is present; a transfer step of transmitting the image to the first recognition unit and also to the second recognition unit via the connection unit; a determination step of stopping the transfer of the image in the transfer step until a predetermined signal is received from the second recognition unit after the image is transferred in the transfer step when the connection of the second device by the connection unit is detected; A reading method including:
Citation Information
Patent Citations
Information processor and program
JP2016038792A
Commodity sales data processing apparatus, reader, and program
JP2016091360A
Information processing device
JP2020030701A
