Reading system
The system addresses the inefficiencies in conventional barcode reading by using a directional and focus-adjustable camera to ensure accurate decoding of multiple codes, enhancing inventory efficiency.
Patent Information
- Application Number
- JP2024047658
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-04-26
- Filing Date
- 2024-03-25
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-03-25
AI Technical Summary
Conventional barcode reading systems become complicated and prone to errors when processing a large number of objects, leading to issues like skipped or duplicate readings, especially when objects are piled up.
A reading system with a camera that can change its imaging direction using a drive mechanism, controlled by a device that captures multiple objects, selects and decodes reading codes sequentially, and adjusts focus based on distance measurements, allowing precise targeting and decoding of codes.
The system enables efficient and accurate reading of multiple codes by ensuring each code is properly aligned and focused, reducing errors and simplifying the inventory process.
Smart Images

Figure 2024159530000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a reading system for reading encoded codes attached to products and parts. [Background technology]
[0002] When inventorying products and parts, the barcodes or QR codes (trademarks) attached to the products and parts are read and decoded to identify the products and parts. This allows for efficient inventorying of products and parts, but the work becomes complicated when there are a large number of products and parts.
[0003] To solve this problem, Patent Document 1 discloses a system that captures images of two barcodes at once and decodes them simultaneously. This system can reduce processing time and perform reading processing efficiently. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] U.S. Patent Publication No. 2021 / 312150 Summary of the Invention [Problem to be solved by the invention]
[0005] However, in the conventional technology such as Patent Document 1, when a large number of objects such as products or parts are piled up, the barcode reader must approach the object as close as possible to a place where it can be read, and then read two barcodes in sequence to process the object. This causes a problem that the process is complicated.
[0006] There is also a possibility of skipping or duplicate readings.
[0007] SUMMARY OF THE PRESENT EMBODIMENT An object of the present invention is to solve any of the above problems and provide an efficient reading system. [Means for solving the problem]
[0008] Independently applicable features of the present invention are listed below.
[0009] (1)(2)(3) A reading system according to the present invention is a reading system comprising a camera whose imaging direction can be changed by a drive mechanism, and a control device which controls the camera and the drive mechanism, wherein the control device acquires a first captured image from a camera which has captured an image of a plurality of objects having read codes attached thereto, and when a read code is selected in the acquired first captured image, controls the drive mechanism so that the imaging direction of the camera is in the direction of the read code, acquires a second captured image by increasing the imaging magnification of the camera facing the read code, and decodes the read code based on the second captured image.
[0010] Therefore, the read codes can be read sequentially and accurately.
[0011] (4) In the reading system of the present invention, the control of changing the imaging direction of the camera is performed based on the distance measured by an instrument that measures the distance between the camera and the object and the position of the selected reading code on the first captured image.
[0012] Therefore, the imaging direction of the camera can be changed to face the read code that has been selected with high accuracy.
[0013] (5) In the reading system of this invention, the control device changes the imaging direction of the camera using the driving mechanism, measures the distance between the camera and the object in at least two directions using a measuring instrument, and controls the driving mechanism based on the measured distances in the at least two directions so that the imaging direction of the camera approaches the normal to the imaging surface of the object.
[0014] Therefore, the imaging direction of the camera can be controlled with greater precision.
[0015] (6) In the reading system according to the present invention, the selection of the reading code in the first captured image is performed by an operator on a display showing the first captured image.
[0016] Therefore, the operator can select the reading code while checking it.
[0017] (7) In the reading system according to the present invention, the display is a touch panel display, and an operator operates the touch panel display to select a reading code displayed on the touch panel display.
[0018] Therefore, the operator can easily select the reading code.
[0019] (8) In the reading system according to the present invention, the selection of reading is performed in a state in which the first imaging screen is enlarged.
[0020] Therefore, the selection is easy.
[0021] (9) In the reading system according to the present invention, the selection of the read code in the first captured image is performed by the control device extracting the read code from the first captured image.
[0022] Therefore, the reading code can be automatically selected without the operator having to make a selection.
[0023] (10) In the reading system of the present invention, the control device is characterized in that it displays, in the first captured image, read codes that have already been decoded so as to be distinguishable from read codes that have not yet been decoded.
[0024] Therefore, it is possible to easily determine whether the decoding process has been completed.
[0025] (11) In the reading system of the present invention, the control device extracts characteristic points of the object in the first captured image, compares them with characteristic points of objects already captured and recorded, and displays that the object is one that has already been captured.
[0026] Therefore, it is possible to easily determine whether or not the object has already been processed.
[0027] (12) In the reading system of the present invention, the control device compares the information obtained by decoding the reading code with pre-recorded basic information, and if a mismatch occurs, displays an image containing the reading code.
[0028] Therefore, the read code in which a mismatch occurs can be easily identified.
[0029] (13) In the reading system according to the present invention, the drive mechanism is provided with a gimbal for maintaining a direction set in the initial state.
[0030] Therefore, even if the camera is unsteadily supported, reading can be performed with high accuracy.
[0031] (14) The reading system according to the present invention is characterized in that the camera and the control device are mounted on a moving body.
[0032] Therefore, reading can be performed anywhere the mobile object can move.
[0033] (15)(16)(17) A reading system according to the present invention is a reading system comprising a camera and a control device for controlling the camera, wherein the control device acquires an image from a camera which images a plurality of objects having reading codes attached thereto, decodes the reading codes, and displays in the image the reading codes which have already been decoded in such a manner that they can be distinguished from reading codes which have not yet been decoded.
[0034] Therefore, it is possible to easily determine whether the decoding process has been completed.
[0035] The term "apparatus" is a concept that includes not only one consisting of one computer, but also one consisting of multiple computers connected via a network, etc. Therefore, when the means of the present invention (or a part of the means) is distributed among multiple computers, these multiple computers correspond to the apparatus.
[0036] The term "program" is a concept that includes not only programs that can be executed directly by a CPU, but also programs in source format, compressed programs, encrypted programs, programs that work in conjunction with an operating system to perform their functions, and the like. [Brief description of the drawings]
[0037] [Figure 1] 1 illustrates an overall configuration of a reading system according to an embodiment of the present invention. [Diagram 2] FIG. 1 is an external view of the system when in use. [Diagram 3] FIG. 2 is a diagram showing details of the robot arm 24. [Figure 4] 2 shows the hardware configuration of the smartphone 20. [Diagram 5] 2 shows the hardware configuration of the robot arm 24. [Figure 6a] 1 is a flowchart of a reading program (control program) 48. [Figure 6b] 13 is a flowchart of a reading program (control program) 48 according to another example. [Figure 7] 1 is an example of inventory data. [Figure 8] 13 is an example of an entire image (first captured image). [Figure 9] 1 is an example of a label L on which a two-dimensional barcode M is printed. [Figure 10] 13 is a diagram showing the distance between an object B2 and the smartphone 20. FIG. [Figure 11] FIG. 13 is a diagram showing the position of a selected two-dimensional barcode M2 in the entire image. [Figure 12] 11 is a diagram for explaining calculation of the orientation of the smartphone 20. FIG. [Figure 13] This is an image when the orientation of the smartphone 20 is changed. [Figure 14] 13 is an image (second captured image) obtained when zooming is performed. [Figure 15] 11 is an example of read data. [Figure 16] 1 is an example of inventory data. [Figure 17] This is another example in which the smartphone 20 is supported by the support rod 23. [Figure 18] 1 is a diagram showing the relationship between the imaging direction of a camera and the normal direction of a surface of an object. [Figure 19a] 11 is a process flowchart for matching the imaging direction of the camera with the normal direction. [Figure 19b] 11 is a process flowchart for matching the imaging direction of the camera with the normal direction. [Figure 19c] 11 is a process flowchart for matching the imaging direction of the camera with the normal direction. [Figure 20] 1 is a diagram showing the relationship between the imaging direction of a camera and the normal direction of a surface of an object. [Figure 21] 1 shows the overall configuration of a reading system according to a second embodiment. [Figure 22] 4 is a flowchart of a reading program 48. [Diagram 23] FIG. 13 is a diagram showing an entire image with an end mark 96. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0038] 1. First embodiment 1.1 Functional configuration Fig. 1 shows the functional configuration of a reading system according to an embodiment of the present invention. n objects are each provided with a reading code M1, M2, M3, ... Mn, such as a QR code (trademark). Camera 2 captures images of all of these reading codes M1, M2, M3, ... Mn, and outputs a first image.
[0039] The control device 6 acquires this first image in a first image acquisition process 8. The operator selects one of the multiple read codes shown in this first image. For example, it is assumed that the read code M1 is selected. In a read code selection process 10, the read code M1 selected by the operator is identified.
[0040] In the direction control process 12, the driving mechanism 4 is controlled to change the direction of the camera 2 so that the camera 2 picks up an image in the direction of the selected read code M1.
[0041] In a second image acquisition process 14, the camera 2 facing the direction of the read code M1 is zoomed to acquire a second image with sufficient accuracy to decode the read code M1. In a decoding process 16, the read code M1 is decoded based on the second image.
[0042] In this manner, the read code M1 is decoded and used for inventory processing and stock management.
[0043] Subsequently, when the operator selects the read code M2 from the first image, the control device 6 directs the camera 2 to the read code M2, obtains a second image of the read code M2, and decodes it in the same manner as described above.
[0044] Thereafter, all the read codes displayed in the first image are sequentially decoded in the same manner.
[0045] As described above, according to this embodiment, reading and decoding are performed simply by the operator sequentially selecting the read codes displayed in the first image, so the work is simplified, errors are reduced, and it is efficient.
[0046] 1.2 Appearance and system configuration Figure 2 shows the external appearance of a reading system according to an embodiment of the present invention. Boxes B1, B2, ... containing products are stacked on a pallet 26. Each box is affixed with a two-dimensional barcode (QR Code (trademark)) indicating information about the product stored therein (product code, quantity, etc.).
[0047] The smartphone 20 is set at a position a predetermined distance away from the stacked boxes B1, B2, etc. The smartphone 20 is set at a position where the camera of the smartphone 20 can capture images of the entire boxes B1, B2, etc. The smartphone 20 is arranged so that the touch display is located on the side opposite to the side facing the boxes B1, B2, etc.
[0048] The smartphone 20 is fixed on a tripod 22 via a robot arm 24, which is a driving mechanism 4. In this embodiment, the smartphone 20 constitutes the camera 2 and the control device 6.
[0049] Fig. 3 shows the mechanism of the robot arm 24. The gripping portion is made up of a gripping piece 25 and a sliding piece 27. The sliding piece 27 is slidable in the A direction, and is biased in the closing direction by a spring or the like. Therefore, as shown in Fig. 3, the smartphone 20 can be clamped and held.
[0050] Grip piece 25 that holds smartphone 20 is configured to be rotatable in vertical direction B around shaft 29 by a motor (not shown). Similarly, grip piece 25 is configured to be rotatable in horizontal direction C by another motor (not shown). The drive of these motors is controlled by a control circuit (not shown) housed inside.
[0051] 1.3 Hardware Configuration 4 shows a hardware configuration of the smartphone 20. In this embodiment, the system includes the smartphone 20 and a robot arm 24.
[0052] The CPU 30 is connected to a memory 32, a touch display 34, a non-volatile memory 36, a camera 38, a communication circuit 40, a short-distance communication circuit 42, and a laser distance measuring device 44. The communication circuit 40 is a circuit for connecting to the Internet. The short-distance communication circuit 42 is a communication circuit such as Bluetooth (trademark). Through the short-distance communication circuit 42, the CPU 30 can control the robot arm 24. The laser distance measuring circuit 44 measures the distance to an object based on the time it takes to irradiate a laser such as LiDar and receive the reflected laser.
[0053] An operating system 46 and a reading program 48, which is a control program, are recorded in the non-volatile memory 36. The reading program 48 cooperates with the operating system 46 to fulfill its functions.
[0054] 5 shows the hardware configuration of the robot arm 24. A CPU 50 is connected to a memory 52, a short-range communication circuit 54, a non-volatile memory 56, and motors 58 and 60.
[0055] The short-range communication circuit 54 is a circuit for communicating with the smartphone 20 by Bluetooth (trademark) or the like. The motor 58 is for rotating the held smartphone 20 in the vertical direction B in Fig. 3. The motor 60 is for rotating the held smartphone 20 in the horizontal direction C in Fig. 3.
[0056] An operating system 62 and a control program 64 are recorded in the non-volatile memory 56. The control program 64 performs its functions in cooperation with the operating system 62. The CPU 50 controls the motors 58, 60 in accordance with the control program 64.
[0057] 1.4 Read Operations 6a shows a flowchart of the reading program 48. First, the CPU 30 acquires inventory data from the non-volatile memory 36 (step S1). The inventory data is calculated based on daily product arrival and shipment data. This inventory data can be recorded in the non-volatile memory 36 from the server device via the communication circuit 40. Alternatively, the inventory data can be recorded in the non-volatile memory 36 via a recording medium such as an SD card.
[0058] An example of inventory data is shown in Figure 7. The model number, lot, and quantity of the product stored in each box are recorded. The model number is an identifier that identifies the product, the lot is an identifier that identifies the manufacturing lot, and the quantity indicates the number of products stored in the box. For example, the fifth line shows that the target box contains 13,000 products manufactured in lots 241220B-0169 and 241220B-0170 with model number SQS-A03GF-PP.
[0059] Next, the CPU 30 acquires an overall captured image from the camera 38 and displays it on the touch display 34 (step S3). The displayed captured image is shown in Fig. 8 (step S3). The position of the tripod 22 is adjusted by the operator so that the entire stacked boxes B1 to B15 are captured.
[0060] 8, product labels L2, L3, . . . L15 are affixed to boxes B2, B3, . . . B15. The product labels are not captured on boxes such as B1 because the product labels are affixed to different sides of the boxes.
[0061] Details of the product label L2 are shown in Fig. 9. In addition to the model number 70, quantity 72, and lot number 74 being legibly displayed, a two-dimensional barcode M encoding these numbers is printed on the label.
[0062] In the overall captured image shown in Fig. 8, ten barcodes, namely two-dimensional barcodes M2, M3, ..., M15, are captured. However, the two-dimensional barcodes M2, M3, ..., M15 in the overall image are small and cannot be decoded as is. Therefore, in this embodiment, the two-dimensional barcodes M2, M3, ..., M15 are decoded as follows.
[0063] The operator taps with his / her finger on the two-dimensional barcode (e.g., M2) to be read in the entire image of Fig. 8 displayed on the touch display 34 (step S4). When the operator first taps on the entire image, the CPU 30 records the imaging direction and zoom ratio for capturing the entire image as initial settings. The CPU 30 calculates the direction of the tapped point (two-dimensional barcode M2) on the screen (step S5). The direction is calculated as follows.
[0064] First, CPU 30 calculates distance DS to the box, which is the target object, by laser distance measuring circuit 44. As shown in the plan view of Fig. 10, the vertical distance DS from smartphone 20 to the box is calculated. Next, CPU 30 calculates distance IX in the X-axis direction and distance IY in the Y-axis direction between center point CP of the screen (corresponding to the imaging direction of camera 38) and two-dimensional barcode M2 on the screen of touch display 34, as shown in Fig. 11. Note that calculation of distance IX and distance IY can be realized by using a library program for calculating coordinates prepared in smartphone 20.
[0065] The ratio between the actual distance DX in the X-axis direction of the box and the distance IX on the screen is predetermined by the vertical distance DS. Similarly, the ratio between the actual distance DY in the Y-axis direction and the distance IY on the screen is also predetermined by the vertical distance DS. Therefore, the CPU 30 can calculate the actual distances DX and DY based on the distances IX and IY on the screen. Note that the ratio between the actual distances DX and DY and the distances IX and IY on the screen varies depending on the vertical distance DS. Therefore, it is preferable to measure the relationship between the vertical distance DS and the ratio in advance and record it as a table or the like.
[0066] Alternatively, if the length of an object displayed on the screen (for example, the height and width of a cardboard box) is known, the length on the screen may be calculated and the ratio may be calculated.
[0067] 12, based on the vertical distances DS and DX, it is possible to calculate the angle θx at which the imaging direction of the camera 38 should be tilted in the X-axis direction in order to orient the imaging direction toward the two-dimensional barcode M2. Similarly, based on the vertical distances DS and DY, it is possible to calculate the angle θy at which the imaging direction of the camera 38 should be tilted in the Y-axis direction.
[0068] After calculating the angles θx and θy as described above, the CPU 30 controls the robot arm 24 based on the calculated angles so that the imaging direction of the camera 38 faces the two-dimensional barcode M2 (step S6). As a result, the image captured by the camera 38 is centered around the two-dimensional barcode M2, as shown in FIG.
[0069] When the imaging direction is directed toward the two-dimensional barcode M2 as described above, the CPU 30 controls the camera 38 to perform optical zoom (step S7). In this embodiment, zooming (increasing the imaging magnification) is performed to an extent that the two-dimensional barcode M2 can be decoded. For example, zooming is performed until the two-dimensional barcode M2 is represented by a predetermined number of pixels or more. This makes it possible to obtain a captured image such as that shown in FIG. 14. Note that the zooming may be optical or digital, as long as it is sufficient to make the two-dimensional barcode M2 decodable. However, optical zooming is used when digital zooming would cause image quality degradation that would make it impossible to decode the image.
[0070] The CPU 30 decodes the two-dimensional barcode based on the captured image (step S8), thereby obtaining read data such as that shown in FIG.
[0071] Next, the CPU 30 compares the calculated inventory data (FIG. 7) with the read data of the actually existing product (FIG. 15) (step S9). That is, it checks whether the read data in FIG. 15 exists in the inventory data in FIG. 7. Since the read data in FIG. 15 matches the data in the fifth line in FIG. 7, "OK" is recorded in the comparison column as shown in FIG. 16.
[0072] Next, the CPU 30 controls the robot arm 24 based on the recorded initial setting imaging direction, and sets the zoom ratio to the recorded initial setting, whereby the entire captured image is displayed on the touch display 34 again (step S3).
[0073] When the operator taps the next two-dimensional barcode (for example, M3) in the entire captured image, the CPU 30 reads the two-dimensional barcode M3 in the same manner as above and collates it with the inventory data (steps S4 to S9).
[0074] After that, when all the two-dimensional barcodes in the entire captured image have been read, the CPU 30 ends the process (steps S2 to S10).
[0075] The operator then sets the smartphone 20 in another orientation or at another location to read the two-dimensional barcode.
[0076] In this manner, the inventory data is collated with the read data, and it is possible to determine whether the inventory data matches the read data, whether the read data is not in the inventory data, whether there is no read data corresponding to the inventory data, etc., and to correctly correct the inventory data.
[0077] 1.5 Variations (Other) (1) In the above embodiment, the operator sequentially taps and specifies two-dimensional barcodes on the entire screen (FIG. 8) displayed on the touch display 34. However, the CPU 30 may sequentially find two-dimensional barcodes from the entire screen. For example, the position of the two-dimensional barcode can be estimated by a trained model trained to find the two-dimensional barcode from the entire image using the YOLO algorithm. In this case, the touch display 34 is not necessarily required.
[0078] In this way, the reading process can be carried out automatically without the need for the operator to select the two-dimensional bar code.
[0079] A flowchart for automatically determining the reading position as described above is shown in Fig. 6b. In this example, after capturing an entire image (step S3a), CPU 30 identifies the two-dimensional barcodes contained therein, and calculates and records the angle of each two-dimensional barcode based on the measured distance (step S3b). CPU 30 then zooms camera 38 based on the distance so that the two-dimensional barcodes have a predetermined size (step S7a). Thereafter, CPU 30 reads the two-dimensional barcodes sequentially using the two-dimensional barcodes and directions identified in step S3b (step S6).
[0080] It is preferable that the above reading order of the two-dimensional barcodes is set by CPU 30 so that the total angle control time of camera 38 is minimized (that is, so that the trajectory length for reading the two-dimensional barcodes is minimized).
[0081] Even in the case of automation as described above, the camera 38 may be temporarily returned to the angle and magnification for capturing the entire image before the next two-dimensional barcode is read, as shown in FIG. 6a.
[0082] Furthermore, in the flowchart of Fig. 6a, the operator returns to the whole image once and then specifies the next two-dimensional barcode. However, the operator may specify the position and reading order of the two-dimensional barcode when the whole image is displayed for the first time. In this way, it becomes unnecessary to perform the process of returning to the whole image each time a two-dimensional barcode is read.
[0083] (2) In the above embodiment, a two-dimensional barcode is used as the readable code to be read. However, one-dimensional barcodes or other encoded codes may be used as the readable code. Furthermore, characters or symbols that can be read by humans (and the CPU 30) may be used as the readable code.
[0084] (3) In the above embodiment, the inventory data is acquired by the smartphone 20, and the data read by the smartphone 20 is collated. However, the data read by the smartphone 20 may be transmitted to a server device, and the server device may collate the data against the inventory data.
[0085] (4) In the above embodiment, the smartphone 20 is used. However, a camera, a communication circuit, a control device, and the like may be provided separately.
[0086] (5) In the above embodiment, the tripod 22 is used so that the smartphone 20 can be stably fixed. However, as shown in FIG. 17, a support rod 23 may be used instead of the tripod 22. For example, the lower end 23A of the support rod 23 may be placed on the floor, and the operator may support the support rod 23 with his / her hand. Alternatively, the lower end 23A may not be placed on the floor, and the support rod 23 may be supported with his / her hand. In this way, reading can be performed even when there is no space to place the tripod 22.
[0087] However, supporting the smartphone 20 with the hand will result in blurring of the imaging direction of the camera 38. Therefore, in the example of Figure 17, a three-axis gimbal 90 is provided between the support rod 23 and the robot arm 24.
[0088] When using the device, the operator supports the support rod 23 and positions it so that the entire image is captured by the camera 38. In this state, the operator operates the three-axis gimbal 90 to lock the imaging direction. This keeps the captured image constant even if the support rod 23 is not securely fixed. Subsequent processing is the same as that described above.
[0089] Although a three-axis gimbal is used in the above, a two-axis gimbal or other multi-axis gimbal may also be used.
[0090] (6) In the above embodiment, one 2D barcode is selected and zoomed in to read it. However, multiple 2D barcodes may be selected and zoomed in to read multiple 2D barcodes simultaneously.
[0091] (7) In the above embodiment, the camera 38 is fixed and captures images. However, the camera 38 and the control device 6 may be mounted on a mobile object such as a drone (UAV), and a flight route (movement route) may be set in advance to capture images. In this case, it is preferable to set the device to fly (move) to the next location when capturing and reading images at one location is completed.
[0092] Similar to the above embodiment, the drone (mobile body) is stopped, an overall image is captured, and the angle is changed and zoomed in to read the two-dimensional barcode.
[0093] By using a drone, it is possible to read objects that are stacked high or in dangerous places, etc. Also, by using a mobile robot (one that moves on a flat surface or in space) as a moving body, it is possible to read low spaces where people cannot enter, places close to the floor, gaps, etc.
[0094] The flight route (movement route) may not be preset, and the moving object may move at its own discretion depending on the situation at the site, or may move according to the operation of the operator.
[0095] (8) In the above embodiment, the imaging direction of the smartphone 20 set by the worker is used as the initial state for control. Here, it is preferable that the imaging direction of the smartphone 20 coincides with the normal direction of the imaging surface (the surface on which the two-dimensional bar code is attached) of the target box. As shown in FIG. 18A, when the imaging direction coincides with the normal direction, the point where the imaging direction intersects with the surface of the box B coincides with the center of the imaging range. Therefore, the robot arm 24 is accurately controlled.
[0096] However, as shown in FIG. 18B, if the imaging direction of the camera differs from the normal direction, an error occurs in the control of the robot arm 24.
[0097] Therefore, the control program 64 may perform control so that the imaging direction coincides with the normal direction.
[0098] FIG. 19a shows a flowchart of the initial state setting of the imaging direction of the smartphone 20. The CPU 30 measures and records the distance DS1 to the object by the laser distance measuring circuit 44 (step S11). Next, the CPU 30 controls the robot arm 24 to change its orientation in the vertical direction by a predetermined angle (for example, 10 degrees upward) (step S12). The laser distance measuring circuit 44 measures and records the distance DS2 to the object at this time (step S13). Furthermore, the CPU 30 controls the robot arm 24 to change its orientation in the vertical direction by a predetermined angle (for example, another 10 degrees upward) (step S14). The laser distance measuring circuit 44 measures and records the distance DS3 to the object at this time (step S15).
[0099] The CPU 30 calculates an imaging direction that matches the normal direction based on the above-mentioned three distances DS1, DS2, and DS3, and controls the robot arm 24 to achieve this imaging direction (steps S16 and S17).
[0100] In the above, the imaging direction that matches the normal direction is calculated based on the measured distances at multiple locations (at least two locations), and the robot arm 24 is controlled based on this. However, the robot arm 24 may be controlled to change the angle in the vertical direction, and the point at which the measured distance is shortest may be found and determined as the imaging direction that matches the normal direction.
[0101] Flowcharts for this case are shown in Figures 19b and 19c. Steps S21 to S26 in Figure 19b determine whether the imaging direction of the camera will match the normal direction if it is directed upward, or whether it will match the normal direction if it is directed downward. Once it is determined whether the camera should be directed upward or downward, the camera orientation is changed by a unit angle accordingly. If the measured distance becomes shorter, it means that it is approaching the normal direction, so the orientation is changed by another unit angle (steps S34, S39). If this is repeated and the normal direction is passed, the measured distance will become longer.
[0102] When this is detected (steps S34 and S39), the camera angle is changed to face in the opposite direction. However, because the camera is approaching the normal direction, the unit angle for changing the direction is reduced to 1 / 2 (smaller). This allows the camera to approach the normal direction with greater accuracy.
[0103] The above process is repeated, and when the unit angle becomes sufficiently small (below the threshold), it is determined that the angle has come sufficiently close to the normal direction, and the camera is fixed in that direction (step S42).
[0104] Although the above describes angle adjustment in the up-down direction, the same applies to angle adjustment in the left-right direction.
[0105] (9) In the above embodiment, the relationship between the vertical distance DS and the ratio (the ratio between the actual distances DX, DY and the distances IX, IY on the screen) is measured in advance and recorded as a table, etc. However, the relationship between the vertical distance DS and the ratio may be calculated using the following formula.
[0106] Ratio = actual distance DX / distance on screen IX Actual distance DX = 1 element size x ((distance to object / focal length)-1) x number of pixels of object / camera magnification For example, since the width of the screen of the smartphone 20 is known, it is possible to calculate the ratio by taking this as the distance IX on the screen and calculating the corresponding actual distance DX.
[0107] The size of one element is known, and the focal length is also known. The number of pixels of the target, that is, the number of pixels in the horizontal direction of the screen of the smartphone 20, is also known. The camera magnification is the current camera magnification of the camera of the smartphone 20, and can be obtained from the camera app. The distance to the target can be measured by the laser distance measuring circuit 44.
[0108] (10) In the above embodiment, the user taps with a finger on the two-dimensional barcode (e.g., M2) that is to be read in the overall image of Fig. 8 displayed on the touch display 34. However, if the two-dimensional barcode is small, it is difficult to tap.
[0109] Therefore, the image may be enlarged by pinching out or the like, and the two-dimensional barcode may be enlarged and then tapped. In this case, the reading program 48 determines which two-dimensional barcode in the entire image the tapped two-dimensional barcode is based on which position in the original entire image the enlarged portion corresponds to. Thereafter, the enlarged image is returned to the entire image by the processing of the reading program 48 or the operation of the operator, and processing proceeds.
[0110] (11) In the above embodiment, the inventory data is compared with the read data, and if they match, OK is recorded in the comparison field. However, an image indicating the mismatch may be displayed on the entire image, etc., superimposed on the mismatched two-dimensional barcode.
[0111] (12) In the above embodiment, the entire captured image is displayed in step S3. At this time, for cardboard boxes that have already been read, the images of the cards that have already been read may be displayed superimposed on the image based on their positions on the image.
[0112] For example, when steps S8 and S9 in Fig. 6a are completed, the image feature points of the cardboard box on which the two-dimensional barcode is displayed are extracted and recorded. The feature points of the cardboard box imaged in step S3 are extracted, and if a cardboard box having the same feature points is recorded, a message indicating that reading has been completed is displayed. In this way, double reading can be prevented.
[0113] (13) The above-described modified examples may be implemented in combination with each other. Also, the above-described modified examples may be implemented in combination with other embodiments and modified examples.
[0114] 2. Second embodiment 2.1 Functional configuration Fig. 21 shows the functional configuration of a reading system according to the second embodiment. Each of n objects is provided with a reading code M1, M2, M3, ... Mn such as a QR code (trademark). Camera 2 captures images of all of these reading codes M1, M2, M3, ... Mn and outputs a first image.
[0115] The control device 6 acquires this first image in first image acquisition 8. Furthermore, one of the read codes M1, M2, M3, ... Mn in this first image is selected in read code selection 10. The selection of the read code may be performed by the operator as in the first embodiment, or may be performed by the control device 6.
[0116] The control device 6 decodes the selected read code in a decoding process 16. The decoding of the read code may be performed after zooming as in the first embodiment, or may be performed by enlarging the first image if the accuracy is sufficient.
[0117] In the first image display process 18, the control device 6 displays the first image so that the read code which has been completely decoded can be distinguished from the read code which has not been completely decoded.
[0118] As described above, according to this embodiment, by looking at the first image, it is possible to easily know for which read codes the decoding process has been completed.
[0119] 2.2 Appearance and system configuration The external appearance and system configuration are similar to those in the first embodiment shown in FIGS.
[0120] 2.3 Hardware Configuration The hardware configuration is the same as that in the first embodiment shown in FIGS.
[0121] 2.4 Reading and display processing 22 shows a flowchart of the reading program 48 in this embodiment. In the following, a case where the reading process of the two-dimensional barcode is performed in the same manner as in the first embodiment will be described. However, the reading process does not necessarily have to be the same as in the first embodiment.
[0122] In the flowchart of Fig. 22, steps S1, S2, and S4 to S10 are the same as those in the first embodiment. Step S31 is characteristically different.
[0123] In step S31, the CPU 30 displays the entire captured image by adding a mark to the two-dimensional barcode read in step S8. For example, an entire image as shown in Fig. 23 is displayed on the touch display 34. As shown in the figure, an end mark 96 is displayed near the two-dimensional barcodes M2 and M3 that have been read. This allows the operator to select the next two-dimensional barcode without overlapping or skipping.
[0124] 2.5 Variations (Other) (1) In the above embodiment, the 2D barcode selected by the operator is optically zoomed in on the entire screen and read based on the generated image. However, the 2D barcode may be read by enlarging the entire image (or without modifying it).
[0125] (2) In the above embodiment, the end mark 96 is displayed around the two-dimensional barcode that has been read. However, an incomplete mark may be displayed around the two-dimensional barcode that has not been read.
[0126] (3) In the above embodiment, the end mark is displayed around the two-dimensional barcode, which is the reading code. However, the end mark may be displayed on the entire object to which the two-dimensional barcode is attached.
[0127] (4) The above-described modified examples may be implemented in combination with each other. Also, the above-described modified examples may be implemented in combination with other embodiments and modified examples.
Claims
1. a camera capable of changing an imaging direction by a drive mechanism; A control device for controlling the camera and the driving mechanism; A reading system comprising: The control device includes: A first captured image is obtained from a camera that captures images of a plurality of objects having read codes attached thereto; When a read code is selected in the acquired first captured image, the driving mechanism is controlled so that the imaging direction of the camera is the direction of the read code; acquiring a second captured image by increasing the imaging magnification of a camera facing the read code; A reading system comprising: a reading unit that reads the code based on the second captured image.
2. At least one processor; a memory storing at least one control program configured to be executed by said at least one processor; A control device comprising: The at least one control program A first captured image is obtained from a camera that captures images of a plurality of objects having read codes attached thereto; When a read code is selected in the acquired first captured image, a drive mechanism is controlled to change an imaging direction of the camera so that the imaging direction of the camera is a direction of the read code; acquiring a second captured image by increasing the imaging magnification of a camera facing the read code; A control device having instructions for decoding the read code based on the second captured image.
3. at least one control program configured for execution by at least one processor, A first captured image is obtained from a camera that captures images of a plurality of objects having read codes attached thereto; When a read code is selected in the acquired first captured image, a drive mechanism is controlled to change an imaging direction of the camera so that the imaging direction of the camera is a direction of the read code; acquiring a second captured image by increasing the imaging magnification of a camera facing the read code; A control program comprising instructions for decoding the read code based on the second captured image.
4. In the system of claim 1, the device of claim 2, or the program of claim 3, A system, device, or program characterized in that control for changing the imaging direction of the camera is performed based on a distance measured by an instrument that measures the distance between the camera and an object and the position of the selected read code on the first captured image.
5. In the system of claim 1, the device of claim 2, or the program of claim 3, The control device includes: changing an imaging direction of the camera by the driving mechanism, and measuring distances between the camera and an object in at least two directions by a measuring device that measures the distances; A system, device, or program that controls the drive mechanism so that the imaging direction of the camera approaches the normal to the imaging surface of the object based on the measured distances in the at least two directions.
6. In the system of claim 1, the device of claim 2, or the program of claim 3, A system, device or program, characterized in that a selection of a read code in the first captured image is performed by an operator on a display showing the first captured image.
7. The system, device or program of claim 6, the display is a touch panel display, A system, device, or program, characterized in that an operator operates the read code displayed on the touch panel display to select the read code.
8. The system, device or program of claim 7, The system, device, or program, wherein the selection of the reading is performed while the first imaging screen is enlarged.
9. In the system of claim 1, the device of claim 2, or the program of claim 3, A system, device, or program, characterized in that the selection of a read code in the first captured image is performed by a control device extracting a read code from the first captured image.
10. In the system of claim 1, the device of claim 2, or the program of claim 3, The system, device, or program, wherein the control device displays, in the first captured image, read codes that have already been decoded so as to be distinguishable from read codes that have not yet been decoded.
11. In the system of claim 1, the device of claim 2, or the program of claim 3, The control device extracts characteristic points of an object in the first captured image, compares them with characteristic points of objects that have already been captured and recorded, and displays that the object is one that has already been captured.
12. In the system of claim 1, the device of claim 2, or the program of claim 3, The control device compares the information obtained by decoding the read code with pre-recorded basic information, and if a mismatch occurs, displays an image containing the read code.
13. In the system of claim 1, the device of claim 2, or the program of claim 3, A system, device, or program, characterized in that the drive mechanism is provided with a gimbal for maintaining a direction set in an initial state.
14. In the system of claim 1, the device of claim 2, or the program of claim 3, The camera and the control device are mounted on a moving body.
15. A camera and A control device for controlling the camera; A reading system comprising: The control device includes: Acquire captured images from a camera that captures images of a plurality of objects with read codes; Decoding the read code; A reading system characterized in that, in the captured image, already decoded read codes are displayed so as to be distinguishable from read codes that have not yet been decoded.
16. At least one processor; a memory storing at least one control program configured to be executed by said at least one processor; A control device comprising: The at least one control program Acquire captured images from a camera that captures images of a plurality of objects with read codes; Decoding the read code; A control device characterized in that, in the captured image, already decoded read codes are displayed so as to be distinguishable from read codes that have not yet been decoded.
17. at least one control program configured for execution by at least one processor, The at least one control program Acquire captured images from a camera that captures images of a plurality of objects with read codes; Decoding the read code; a control program for displaying, in the captured image, read codes that have already been decoded so as to be distinguishable from read codes that have not yet been decoded;
Citation Information
Patent Citations
Image processing apparatus and program
JP2013012226A
Information collection device, article management system using the same, and winding device
JP2017218325A
Inspection management system for storage state of article in warehouse
JP2019167177A
Image processing apparatus, image processing method and program
JP2019220829A
Moving body, server device, moving body control system, moving body control method, and program
JP2023054958A
