Data collection method, data collection system, data collection device, data provision method, and non-transitory storage medium

Abstract

A data collection method, a data collection system, a data collection device, a data provision method, and a non-transitory storage medium that automatically collect data at a sorting stage are provided. The data collection method includes processes of acquiring an identification code read by a reader that reads the identification code of an identified article and transports the article to a different sorting destination based on the identification code from a sorter that sorts the article; acquiring image data of the article identified by the acquired identification code from a camera installed in a manner of capturing the article transported by the sorter; and storing the acquired image data in association with the acquired identification code.

Description

Technical Field

[0001] This invention relates to a data collection method, a data collection system, a data collection device, a data provision method, and a non-temporary storage medium for collecting data used in sorting machines that sort incoming items according to their destination. Background Technology

[0002] Sorting machines, used in logistics centers that handle the dispatch of large quantities of goods, sort items according to their destination. Items are fed into the sorting machine by a reader that reads the item identification codes affixed to the items by the sorting operator. By reading the item identification codes, the sorting machine identifies the items placed on the pallet and discharges them to containers for bundling or to chutes where boxes are prepared.

[0003] In large-scale logistics centers, the number of items is enormous. Furthermore, regardless of scale, the variety of items is vast, leading to a heavy workload for sorting workers and a desire for automated item identification. Automatic item identification using deep learning models based on images of the item's appearance is becoming practical. Patent Document 1 discloses a method for providing learning data to image data to improve identification accuracy by identifying errors in annotations used to label the identification data.

[0004] Prior art literature

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2020-030692 Summary of the Invention

[0007] Summary of the invention

[0008] The problem that the invention aims to solve

[0009] As disclosed in Patent Document 1, annotations have traditionally been made by humans through visual observation. In visually-based annotation, there is a possibility that the time required to accumulate learning data to a level that ensures accuracy may not keep up with the product cycle, such as changes in the appearance of the product due to time and seasons.

[0010] The purpose of this invention is to provide a data collection method, data collection system, data collection device, data provision method, and non-temporary storage medium that automatically collects data during the sorting stage.

[0011] Solution for solving the problem

[0012] One embodiment of the data collection method disclosed herein includes the following processing: acquiring the identification code read by the reader from a sorting machine that includes a reader for reading identification codes of identified items and sorts the items to different sorting destinations based on the identification codes; acquiring image data of the items identified by the acquired identification codes from a camera installed in a manner to photograph the items transported by the sorting machine; and storing the acquired image data in a correspondence with the acquired identification codes.

[0013] One embodiment of the data collection system disclosed herein includes: a sorting machine comprising a reader for reading and identifying identification codes of items, and sorting the items by conveying them to different sorting destinations based on the identification codes; a camera mounted to photograph the items conveyed by the sorting machine; and a data collection device connected to the sorting machine and the camera to collect image data of the items, wherein the data collection device performs the following processing: acquiring the identification code read by the reader; acquiring image data of the items identified by the acquired identification code from the camera; and establishing a correspondence between the acquired image data and the acquired identification code and storing it in a storage unit.

[0014] One embodiment of the data collection apparatus disclosed herein includes: a mechanism connected to a sorting machine, the sorting machine including a reader for reading and identifying an identification code of an item and sorting the item to different sorting destinations based on the identification code; a mechanism for acquiring the identification code read by the reader; a mechanism for acquiring image data of the item identified by the acquired identification code from a camera mounted in a manner for photographing the item being transported by the sorting machine; and a mechanism for storing the acquired image data in correspondence with the acquired identification code.

[0015] In one embodiment of the present disclosure, a computer program is connected to a sorting machine, which includes a reader for reading and identifying identification codes of items and sorting the items to different sorting destinations based on the identification codes. The computer program causes the computer to perform the following processes: acquiring the identification code read by the reader; acquiring image data of the items identified by the acquired identification codes from a camera installed in a manner that captures images of the items transported by the sorting machine; and establishing a correspondence between the acquired image data and the acquired identification codes and storing them.

[0016] In the data collection method, data collection system, data collection device, data provision method, and computer program disclosed herein, an identification code obtained from a reader that reads the identification code of an item in a sorting machine is automatically associated with the image data of the item. This allows for accurate and sequential collection of labeled image data without hindering the operation of the sorting machine itself. The identification code is, for example, the globally used EAN (JAN) code. The sorting machine can handle any item that can be identified by the identification code, and this is a diverse range of items; therefore, it is not used for identifying items as simply "good" or "bad." Learning data for identification with multiple options can be collected from image data of a wide variety of items.

[0017] By using multiple cameras, a correspondence can be established between the same object and image data taken from different angles. When the recognition model learns using image data, an improvement in recognition accuracy can be expected.

[0018] The data collected by the data collection method disclosed herein includes: image data of the transported items captured by a camera installed on a sorting machine that sorts items to different sorting destinations; and identification codes for the items captured in the image data, read by a reader provided with the sorting machine. This data is used to enable a recognition model to learn in a manner that outputs data and accuracy for recognizing the items when image data of the items are input.

[0019] The data collection method of one embodiment of this disclosure may further include processing of establishing a correspondence between the acquired image data and the shooting date and time of the image data and storing them.

[0020] In the data collection method disclosed herein, a correspondence is also established with the date and time of image data capture. When using a sorting machine, data can be collected continuously without hindering its operation, thus allowing for the collection of items whose appearance may change with time and season.

[0021] In one embodiment of the data collection method disclosed herein, newly acquired image data from the camera is input into a recognition model. The recognition model learns to output data and accuracy for recognizing the object when image data is input, based on stored image data and identification codes. It determines whether the data output from the recognition model is consistent with the identification code read by the reader for the object photographed in the image data, and whether the accuracy is above a predetermined value. If the accuracy is determined to be below the predetermined value, a correspondence is established between the newly acquired image data and the identification code and the data is stored.

[0022] In the data collection method disclosed herein, a recognition model learned from collected image data and identification codes is used. If the recognition accuracy of the recognition model decreases, the image data is collected for relearning. This method can address situations where the appearance of an item changes beyond the learning range of the recognition model.

[0023] In one embodiment of the data providing method of this disclosure, the image data is provided by establishing a correspondence between the image data and the identification code of the item from a storage device storing data stored by any of the above-described data collection methods.

[0024] The collected image data is used not only in the model-based recognition of the reader that replaces the sorting machine, but also from the storage device to other communication devices. It can also be used for learning purposes in other communication devices.

[0025] Invention Effects

[0026] According to this disclosure, it is possible to collect image data that automatically establishes a correspondence with a highly universal identification code read by a sorting machine used in the sorting of items. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the data collection method of this embodiment.

[0028] Figure 2 This is a block diagram showing the structure of the data collection device in Embodiment 1.

[0029] Figure 3 This is a flowchart illustrating an example of the data collection and processing process based on the control unit.

[0030] Figure 4 This is a diagram illustrating an example of the content of data collected through data collection and processing.

[0031] Figure 5 This is a block diagram showing the structure of the data collection system in Implementation 2.

[0032] Figure 6 This is a flowchart illustrating an example of the data collection and processing procedure in Implementation Method 2.

[0033] Figure 7 This is a schematic diagram of a recognition model that learns based on collected data.

[0034] Figure 8 This is a block diagram showing the structure of the data collection system in Implementation 3.

[0035] Figure 9 This is a flowchart illustrating an example of the data collection and processing procedure in Implementation Method 3. Detailed Implementation

[0036] This disclosure will be described in detail with reference to the accompanying drawings illustrating embodiments thereof.

[0037] Figure 1 This is a schematic diagram of the data collection method of this embodiment. The data collection system 300 includes a sorting machine 100, a camera 101 mounted in such a way that the tray 121 of the sorting machine 100 is included in the field of view, and a data collection device 2 connected to the camera 101 and the control unit 10 of the sorting machine 100. The data collection device 2 collects and stores the image data captured by the camera 101 and the identification code of the item read by the sorting machine 100 by establishing a correspondence.

[0038] The sorting machine 100 is divided into an input section 11, a conveying section 12, and a chute section 13.

[0039] The input unit 11 includes a workbench 111 and a reader 112 that reads identification codes attached to items. The reader 112 can be a barcode reader, a QR code reader, or an RFID (Radio Frequency Identifier) ​​reader. The reader 112 may use short-range wireless communication. The identification code may be, for example, an EAN (JAN) code. Alternatively, the identification code may be a code for identifying books or magazines. Other possible identification codes include CODE128, NW-7, CODE39, and ITF. Figure 1 As shown, the input unit 11 may include a group of multiple workstations 111 and readers 112.

[0040] The conveying unit 12 includes a plurality of pallets 121 that move along a track 122 arranged in a ring shape, and a tilting mechanism for tilting the pallets 121. The track 122 on which the plurality of pallets 121 move is as follows: Figure 1 As shown, the multiple trays 121 can be configured to circulate parallel to the horizontal plane, or to move vertically and intermittently in a straight line, or to circulate in a spiral.

[0041] Multiple trays 121 are equipped with sensors for determining whether a tray 121 is empty. In one example, the sensors are weight sensors attached to each of the multiple trays 121. In another example, the sensors may be photoelectric sensors or displacement sensors to determine whether items are placed on the tray 121 and the size and structure of the items. In yet another example, the sensors are image sensors that capture images of the tray 121; by comparing these images with images of an empty tray 121, it can be determined whether the tray is empty.

[0042] Identification data is attached to each of the multiple trays 121. The transport unit 12 is equipped with a detection mechanism that detects the location of at least a specific tray 121 within the transport unit 12. The transport unit 12 is capable of detecting the position of each tray within the transport unit 12 according to the connection sequence of the trays 121. The detection mechanism may include, for example, an encoder mounted on a motor of a drive unit that moves the tray 121, and a detection unit that receives pulse signals output from the encoder to detect the position. In another example, the detection mechanism detects the position of at least a specific tray by image analysis of image data obtained from a camera that captures images of the tray 121. In yet another example, the detection mechanism may also use a reader that reads identification tags attached to the tray 121 at a specific location to detect the position.

[0043] The tilting mechanism for tilting the pallet 121 can be implemented, for example, by a structure in which a support portion supporting the pallet 121 on the track 122 can be bent. The tilting mechanism may also be a mechanism that lifts a portion of the lower surface of the pallet 121 to tilt it. The conveying unit 12 is capable of tilting the pallet 121, as indicated by the control unit 10, to the indicated position.

[0044] The chute section 13 has a receiving section 131 that is arranged parallel to a portion of the track 122 of the pallet 121 of the conveying section 12, and receives items released from the pallet 121 by tilting the pallet 121. In one example, the receiving section 131 is as follows: Figure 1 As shown, a small container or corrugated cardboard box, i.e., logistics material C, is placed on the container for transport. The receiving unit 131 may also be a divided workbench, on which items released by the bundling operator are bundled into logistics material C.

[0045] The sorting machine 100's input section 11, conveying section 12, and chute section 13 are connected to the control section 10 via signal lines and are controlled by the control section 10. When the sorting operator uses the reader 112 to read the identification code of an item at the input section 11, the control section 10 detects the reading of the identification code and the code itself. For the detected identification code, the control section 10 obtains data on the identification of the pallet 121 on which the item was recently input, through the output from the sensor 123 and the detection mechanism of the pallet 121. Thus, the control section 10 temporarily stores which pallet 121 contains which identification code. Based on pre-provided sorting plan data, the control section 10 determines which pallet 121 should be tilted at the chute section 13 and instructs the conveying section 12 on the identification data of the determined pallet 121. Based on the position of the pallet 121 detected by the conveying section 12, the conveying section 12 tilts the indicated pallet 121 using the tilting mechanism. The control unit 10 can output the number and type of items to be put into the chute section 13 and the logistics material C to a certain delivery destination. As described above, the sorting operator puts the items into the sorting machine 100 by having the reader 112 of the input section 11 read the identification code of the items, and the sorting machine 100 automatically performs sorting based on the sorting plan.

[0046] In the data collection method disclosed herein, the data collection device 2 collects image data of items obtained by the camera 101 from the tray 121 into which items are placed via the input section 11, establishing a correspondence between the image data and the identification code read by the input section 11. The camera 101, as shown... Figure 1 As shown, the device is configured to photograph the tray 121 from different angles. Figure 1 In this example, two cameras 101 are provided. The data collection device 2 collects image data captured from cameras 101 at different angles. Figure 1 There can be two, but it can also be one or more than three.

[0047] Figure 1 The sorting machine 100 shown is of the type that transports items via multiple pallets. The sorting machine 100 is not limited to this; the conveying section 12 may also be a structure that transports items to the chute section 13 via conveyors (rollers, slats, etc.).

[0048] In this data collection method, no additional work is required from sorting operators; that is, the sorting operation using the sorting machine 100 can be changed, and image data can be collected together with identification codes that reliably identify items. In this embodiment's data collection method, the data collection device 2 can collect image data of items based on the type, manufacturer, or producer of the item identified by the identification code. The identification code easily distinguishes operators using EAN codes, so image data can be easily collected not only based on the item identified by the identification code but also based on the manufacturer. Furthermore, because of the identification code, the data collection device 2 does not rely on two or three selection options such as good / bad or A / B / C, and can collect image data for identifying a wide variety of items. In this data collection method, the data collection device 2 can also collect image data according to period or season. The collected image data can be used in various ways. The collected image data can be used to eliminate the need to read the identification code using a reader at the input section 11 of the sorting machine 100. A learning model that utilizes collected image data can determine the identification code of an item based on an image of the item. The collected image data can then be used for item identification at the delivery destination store.

[0049] Hereinafter, several embodiments will be given to illustrate the structure of the data collection device 2 that implements the data collection method described above.

[0050] (Implementation Method 1)

[0051] Figure 2 This is a block diagram illustrating the structure of the data collection device 2 in Embodiment 1. The data collection device 2 includes a control unit 20, a storage unit 21, and an input / output unit 22. The data collection device 2 may also be a PLC (Programmable Logic Controller). The control unit 20 includes a CPU (Central Processing Unit) 200 and a non-volatile memory 201. The control unit 20 may also be a microcontroller. The control unit 20 collects data by executing a data collection program 2P stored in the memory 201 through the CPU 200.

[0052] Storage unit 21 is a non-volatile storage medium such as a hard disk or SSD (Solid State Drive). In storage unit 21, collected image data is stored by establishing a correspondence between the collected image data and the identification code of the object represented by the image data. Image data can also be stored by establishing a correspondence between the image data and the shooting period.

[0053] Storage unit 21 stores setting information for data collection, as described later. This setting information may include, for example, information for determining when the camera 101 can capture an item identified by the identification code within the field of view after the identification code is read by reader 112. The setting information may be time, or, as described later, pulse count.

[0054] The input / output unit 22 is an interface connected to the sorter 100 and the camera 101. The control unit 20 can obtain the identification code read by the reader 112 from the sorter 100 via the input / output unit 22. The control unit 20 can also obtain data from the sensor 123 used to determine the pallet 121 on which items have been placed via the input / output unit 22. When the conveyor section 12 of the sorter 100 is a conveyor type, the control unit 20 can obtain data indicating the range (hypothetical pallet) of the items (identified by the identification code) within the conveyor section 12 via the input / output unit 22. The range of items can be determined by the size of the items measured by the sensor 123. The control unit 20 can also obtain data indicating the position of the pallet 121 via the input / output unit 22. This position data is, for example, from the encoder 124 of the motor that moves the pallet 121; the control unit 20 can obtain the position of the pallet 121 by counting pulses from the encoder 124. The control unit 20 receives the image signal monitored and output from the camera 101 via the input / output unit 22, and can acquire image data of the captured object from the image signal at a determined time. Figure 2 As shown, each signal obtained from the sorting machine 100 is connected to the sorting machine 100 via a different signal line.

[0055] Figure 3 This is a flowchart illustrating an example of the data collection and processing procedure based on the control unit 20. During operation, the control unit 20 continuously performs the following processing based on the data collection procedure 2P.

[0056] The control unit 20 acquires the identification code read by the reader 112 (step S201). Therefore, whenever the control unit 10 of the sorting machine 100 receives an identification code from the reader 112 of the input unit 11, it outputs the identification code along with the data from the reader 112 to the data collection device 2. The input / output unit 22 can also receive the signal branched from the signal output from the reader 112 of the input unit 11 to the control unit 10, and the control unit 20 acquires the identification code without going through the control unit 10.

[0057] When an item identified by the acquired identification code enters the field of view of the camera 101, the control unit 20 acquires image data from the camera 101 (step S202). The timing of acquiring the image data is determined in advance, for example, by the layout of the sorting machine 100, the setting position of the camera 101, and the waiting time from the acquisition of the identification code, which is set according to the conveying speed of the conveyor unit 12. The waiting time is stored in advance as setting information in the storage unit 21 or non-volatile memory.

[0058] The timing for acquiring image data can also be determined by pulse counts corresponding to the movement distance of the tray 121 (position or range in the case of a conveyor type) output from the control unit 10 of the sorting machine 100. The pulse counts are stored in the storage unit 21 or a non-volatile memory as preset information. In the case of pulse count-based acquisition, the sorting machine 100 outputs the pulse counts of the tray 121 from the encoder 124. Furthermore, the timing for acquiring image data can also be determined based on an image sensor that separately reads the identification data of the tray 121 that is the target. Multiple readers 112 are provided in the input unit 11, and the timing is determined according to the reader 112 depending on the distance to the camera 101.

[0059] The control unit 20 establishes a correspondence between the image data obtained in step S202 and the identification code and shooting date and time obtained in step S201, and stores them in the storage unit 21 (step S203), thus ending one image data collection cycle. Storing the shooting time in step S203 is not mandatory.

[0060] Data collection device 2 continues to perform its functions during operation. Figure 3 The processing procedure is shown in the flowchart. The image data collected in the storage unit 21 of the data collection device 2 is periodically read from the storage unit 21 by the maintenance personnel of the sorting machine 100 and used.

[0061] Figure 4 This is a diagram illustrating the content of data collected through data collection and processing. For example... Figure 4 As shown, multiple image data obtained from camera 101 are stored by establishing a correspondence with the identification code read by reader 112. For example... Figure 4 As shown, a correspondence can be established between the shooting date and time and the image data. It should be noted that the identification code can be divided into upper and lower digits, differentiated according to the operator. A correspondence can be established between the image data ID and the image data ID, which is used to identify the image data separately, and then stored.

[0062] The identification code uses the EAN (JAN) code. By using a database that stores the correspondence between the EAN code and data such as item name, manufacturer name, product number, and price, it is possible to identify which product was produced by which business in which country. Image data is collected by establishing a correspondence with the identification code, allowing for the collection of image data for various items based on their type, manufacturer, and producer. Furthermore, image data is collected by establishing a correspondence with the shooting period, allowing for the collection of image data based on the period of sorting for delivery. For example, even if a bag uses a seasonally limited color or pattern, it can be reflected in the learning process or excluded.

[0063] (Implementation Method 2)

[0064] Figure 5 This is a block diagram illustrating the structure of the data collection system 300 in Embodiment 2. The data collection system 300 in Embodiment 2 also includes a storage device 3 that receives and stores data collected by the data collection device 2 via the network N. In addition to the control unit 20, storage unit 21, and input / output unit 22, the data collection device 2 in Embodiment 2 also includes a communication unit 23. Although the data collection device 2 stores setting information in the storage unit 21, it sequentially transmits image data to the storage device 3 via the communication unit 23. Multiple data collection devices 2 exist, and image data is transmitted according to each sorting machine 100.

[0065] The communication unit 23 enables the transmission and reception of image data with the storage device 3 via a network N, including the Internet. The communication unit 23 may be, for example, a network interface card (NIC) or a wireless communication module. The network N may include the Internet and a carrier network. The network N may also be a dedicated line.

[0066] Storage device 3 includes a control unit 30, a storage unit 31, and a communication unit 32. Storage device 3 is a server computer. Storage device 3 is managed, for example, by the manufacturer of sorting machine 100. Control unit 30 is a processor using a CPU and / or GPU (Graphics Processing Unit), and includes built-in volatile memory, clock, etc., to perform storage processing.

[0067] The storage unit 31 includes a non-volatile storage medium such as an SSD or hard disk. In the storage unit 31, the collected image data and the identification code of the object represented by the image data are stored separately. The image data can be stored by establishing a correspondence with the shooting period, or it can be stored by establishing a correspondence with the device identification data of the data collection device 2 indicating the sending source.

[0068] The communication unit 32 enables the transmission and reception of data with the data collection device 2 and the communication terminal device 4 via the network N. The communication unit 32 is, for example, a network interface card (NIC) or a wireless communication module.

[0069] Figure 6 This is a flowchart illustrating an example of the data collection process in Embodiment 2. During operation, the control unit 20 of the data collection device 2 continuously executes the following processes based on the data collection program 2P; similarly, the control unit 30 in the storage device 3 continuously executes the following processes. Regarding... Figure 6 The flowchart shows the processing steps and Figure 3 The flowchart shows a common process, and detailed explanations are omitted.

[0070] Whenever the control unit 20 of the data collection device 2 receives an identification code read by the reader 112 from the sorting machine 100, it acquires the identification code (S201) and acquires image data of the item identified by the acquired identification code from the camera 101 (S202).

[0071] The control unit 20 establishes a correspondence between the acquired image data and the identification code and shooting date and time obtained in step S101, and sends it to the storage device 3 via the communication unit 23 (step S213), and ends the processing corresponding to an item input.

[0072] The control unit 30 of the storage device 3 receives image data of an item that has been associated with an identification code and the date and time of the shooting, which is sent from the data collection device 2 (step S301), stores it in the storage unit 31 (step S302), and then ends.

[0073] In this way, image data of items identified by the identification code is accumulated in the storage section 31 of the storage device 3 by establishing a correspondence with the identification code. The storage device 3 can collect image data together with the identification code from multiple sorting points. The storage device 3 can directly store the accumulated image data, or it can generate an identification model that identifies items using the collected image data.

[0074] Figure 7 This is a schematic diagram of the 3M recognition model, which learns based on collected data. For example... Figure 7 As shown, the recognition model 3M includes convolutional layers, pooling layers, and fully connected layers. Recognition model 3M learns by outputting data that identifies the objects depicted in the image data based on the features of the input image data, along with scores representing its accuracy. The data for object recognition can be labels suitable for learning by recognition model 3M. Alternatively, the data for object recognition can be the identification code itself.

[0075] The learning data is image data collected in the storage unit 31 of the storage device 3. The recognition model 3M initially struggles to recognize all items. Therefore, the recognition model 3M can learn by pre-classifying image data based on identification codes, according to items that are sorted by the same sorting machine 100 at the same time during the sorting operation, items supplied by the same operator, and items that, although supplied by different operators, share a common classification. For example, for fresh foods such as vegetables that are difficult to attach identification codes to, in order to be able to identify them by producer (operator), the recognition model 3M can learn by extracting only image data that establishes a correspondence with the identification codes of the same type of vegetable. In this case, the original identification code is printed on a label affixed to the fresh food. For example, in order to be able to recognize items that are easily contained in logistics materials C, the recognition model 3M can reduce the learning to image data of items sorted by the same sorting machine 100 at the same time. Alternatively, the recognition model 3M can learn image data by period or season using the date and time on which the image data was taken.

[0076] The learned recognition model 3M can replace the reader 112 of the input unit 11 of the sorting machine 100. The input unit 11 has a recognition device that replaces the reader 112. This recognition device includes a camera, a storage unit storing the recognition model 3M, and a processing unit that performs recognition processing. The recognition device inputs the image data captured by the camera into the recognition model 3M, identifies the item based on the recognition data with the highest accuracy score output from the recognition model 3M, and outputs the identification code to the control unit 10. Thus, even if the sorting operator does not work through the sorting machine 100, the sorting machine 100 can automatically perform sorting.

[0077] Storage device 3 can communicate with, for example, a personal computer, a tablet computer, or a communication terminal device 4 acting as a POS (Point of Sales) terminal via network N. Storage device 3 can retrieve image data collected in its storage unit 31 regarding image data of the type or attribute permitted by the user (e.g., the manufacturer of the goods) using user identification data used in communication terminal device 4. Storage device 3 can accept learning requests and provide the learned recognition model 3M to communication terminal device 4 based on those requests. Storage device 3 can provide desired data, for example, by extracting and sending image data of goods from specific manufacturers. Storage device 3 can also provide desired data by extracting and sending image data of specific types of goods to communication terminal device 4. Furthermore, storage device 3 can also provide data by extracting and sending image data of specific goods from specific periods.

[0078] For example, the communication terminal device 4 is a terminal installed in a convenience store, which can receive from the storage device 3 a recognition model 3M for automatically identifying the items for sale. The communication terminal device 4 can receive only the necessary image data of the items and the identification code from the storage device 3.

[0079] In this way, the image data collected by the sorting machine 100 using multiple parts can be used for various purposes. In the sorting machine 100 that identifies items by reading identification codes, new data continues to be collected. In addition, information based on the collected image data can be fed back to the convenience store that should identify the item.

[0080] (Implementation Method 3)

[0081] In implementation 3, data is collected using a recognition model 3M that has been learned from data collected by data collection device 2. Figure 8 This is a block diagram illustrating the structure of the data collection system 300 in Embodiment 3. The structure of the data collection system 300 in Embodiment 3 is the same as in Embodiment 1, except that the storage unit 21 of the data collection device 2 stores the definition data of the identification model 3M and the data collection method differs. Structures common to Embodiment 1 in the following description of the data collection system 300 in Embodiment 3 are marked with the same symbols, and detailed explanations are omitted.

[0082] The definition data of the recognition model 3M stored in the storage unit 21 is as used in Embodiment 2. Figure 7 As explained, the model's parameters and network definition data are learned from image data collected and associated with identification codes. When image data of a photographed object is input, the recognition model 3M outputs the object's identification code as reflected in the image data, along with a score indicating the accuracy of the representation. The recognition model 3M can differentiate based on the object's manufacturer or operator, etc., using different learning units.

[0083] Figure 9 This is a flowchart illustrating an example of the data collection and processing procedure in Embodiment 3. During operation, the control unit 20 of the data collection device 2 continuously executes the following processing based on the data collection program 2P. Regarding... Figure 9 The flowchart shows the processing steps and Figure 3 The flowchart shows a common process, and detailed explanations are omitted.

[0084] Whenever the control unit 20 receives an identification code read by the reader 112 from the sorting machine 100, it acquires the identification code (S201) and acquires image data of the item identified by the acquired identification code from the camera 101 (S202).

[0085] The control unit 20 provides the image data obtained in step S202 to the recognition model 3M (step S223), and obtains the accuracy score and recognition code output from the recognition model 3M (step S224). The control unit 20 determines the score output from the recognition model 3M, i.e., the recognition code with the highest accuracy (step S225).

[0086] The control unit 20 determines whether the identification code determined in step S225 is consistent with the identification code obtained in step S201 (step S226). If the identification code is inconsistent (S226: No), the control unit 20 establishes a correspondence between the image data obtained in step S202 and the identification code obtained in step S201 and stores it in the storage unit 21 (step S227). Thus, the image data is stored for relearning.

[0087] If the match is determined in step S226 (S226: Yes), the control unit 20 determines whether the accuracy score corresponding to the identification code obtained in step S224 is above a predetermined value (step S228). If the accuracy score is determined to be below the predetermined value (S228: No), the control unit 20 establishes a correspondence between the image data obtained in step S202 and the identification code obtained in step S201 and stores it in the storage unit 21 (S227). In step S227, the control unit 20 may also establish a correspondence with the shooting date and time to store the image data.

[0088] If, in step S228, it is determined that the score indicating accuracy is above a predetermined value (S228: Yes), the control unit 20 terminates the process. If, in step S228, the score indicating accuracy is above a predetermined value and the learned recognition model 3M can accurately identify the target, the image data may not be collected for relearning.

[0089] The image data for relearning stored in storage unit 21 is read from storage unit 21 by the maintenance personnel of sorting machine 100 and used for relearning recognition model 3M. In embodiment 3, instead of storing it in storage unit 21 in step S227, the image data can be associated with the recognition code and sent to storage device 3 via network N.

[0090] In the sorting machine 100, the corresponding identification code and image data can be obtained at all times, so the accuracy of the identification model 3M used in other parts can be confirmed. For example, if the identification accuracy decreases due to changes in the appearance of the item, it can be relearned.

[0091] Since the control unit 20 can obtain the identification code read by the reader 112, it can collect data when new data is needed, except in cases of reduced accuracy, such as when the shooting date and time corresponding to the already stored image data is more than a specified period.

[0092] The data collection system 300 shown in embodiments 1 to 3 is an example, and can be combined appropriately.

[0093] The embodiments disclosed above are illustrative in all respects and not intended to be limiting. The scope of the invention is shown in the technical solutions and includes all modifications within the same meaning and scope.

[0094] Symbol explanation:

[0095] 100 sorting machines

[0096] 10. Control Department

[0097] 11. Investment Department

[0098] 12 Conveying Department

[0099] 101 Camera

[0100] 300 Data Collection System

[0101] 2 Data collection device

[0102] 20 Control Department

[0103] 21 Storage Department

[0104] 2P Data Collection Program

[0105] 3. Storage device

[0106] 31 Storage Department

[0107] 3M recognition model

Claims

1. A data collection method, wherein the data collection method is performed by a computer connected to a sorting machine that sorts items to different sorting destinations, wherein, The data collection method includes the following processing: The processor obtains the identification code read by the reader from the reader installed in the sorting machine that reads and identifies the identification code of the item; The processor acquires image data of the items identified by the identification code within the conveyor section of the sorting machine from various angles using two or more cameras installed in a manner that captures images of the items from different angles. The images are taken from the camera at the moment the item identified by the acquired identification code enters the camera's field of view. The timing of acquiring the image data is predetermined by the layout of the sorting machine, the camera's placement, and a waiting time set according to the conveyor speed of the conveyor section, or a pulse count corresponding to the distance the conveyor section has traveled. Multiple images captured from different angles are linked to the acquired identification codes and stored.

2. The data collection method according to claim 1, wherein, The identification code will be associated with the type of item, manufacturer, or producer to make it identifiable. For different items, multiple image data taken from different angles are stored according to type, manufacturer, or producer, and a corresponding relationship is established between each item and its respective identification code.

3. The data collection method according to claim 1, wherein, The data collection method further includes the following processing: establishing a correspondence between the multiple image data and the obtained identification code and the shooting date and time through the processor and storing them.

4. The data collection method according to any one of claims 1 to 3, wherein, The data collection method also includes the following processing: Accepts search requests that specify the type, manufacturer, or date and time of shooting; Extract image data corresponding to the identification code of an item of the type specified by the search request, or image data corresponding to the identification code of an item from a manufacturer specified by the search request, or image data taken on the date and time specified by the search request; and The extracted image data is sent to a terminal device with a different retrieval requirement source than the sorting machine.

5. The data collection method according to claim 1, wherein, The camera acquires new image data of the newly inserted item, and the reader acquires the identification code of the newly inserted item. The processor inputs the new image data into the recognition model, which, given the input image data of the object, outputs data on the accuracy of recognizing the object, and learns based on the stored image data and identification codes that establish mutual correspondence. The processor determines whether the data output from the recognition model corresponds to the identification code of the newly inserted item corresponding to the new image data. The processor determines whether the accuracy output from the recognition model is above a specified value. If the processor determines that the accuracy is less than a specified value, it establishes a correspondence between the new image data and the identification code of the newly introduced item and stores it.

6. A data collection system, comprising: A sorting machine includes a reader that reads and identifies the identification codes of items, and sorts the items by transporting them to different sorting destinations based on the identification codes. Two or more cameras are mounted in a manner that captures images from different angles of the area within the conveyor section of the sorting machine where the items, identified by the identification code and read by the reader, are placed in the conveyor section; and A data collection device, connected to the sorting machine and the camera, collects image data of the items. in, The data collection device performs the following processing: Obtain the identification code read by the reader; Image data of items identified by the acquired identification code are acquired from various field-of-view angles by the camera, wherein the image data is acquired from the camera at the moment when the item identified by the acquired identification code enters the field-of-view angle of the camera, and the timing of acquiring the image data is predetermined by the layout of the sorting machine, the position of the camera, and a waiting time or pulse count corresponding to the travel distance of the conveyor, set according to the conveying speed of the conveyor; and The acquired image data from multiple angles is associated with the acquired identification codes and stored in the storage unit.

7. The data collection system according to claim 6, wherein, The data collection system also includes a storage device, which stores the image data collected by the data collection device in a correspondence with the identification code. The storage device performs the following processing: Accepts retrieval requests based on the identification code of a specified item; Based on the identification code specified by the retrieval requirements, image data is extracted according to the image data stored in the established correspondence and the identification code; as well as The extracted image data is provided to other devices.

8. A data collection device, wherein, The data collection device includes: A mechanism connected to a sorting machine, the sorting machine including a reader that reads and identifies the identification codes of items, and sorts the items by transporting them to different sorting destinations based on the identification codes; An organization that obtains the identification code read by the reader; A mechanism that captures image data of items identified by the identification code within the conveyor section of the sorting machine from various angles, based on the range of the items carried in the conveyor section after being read by the reader and identified by the identification code, wherein the image data is acquired from the camera at the moment when the item identified by the identification code enters the field of view of the camera, the timing of acquiring the image data is predetermined by the layout of the sorting machine, the camera's placement position, and a waiting time set according to the conveyor speed of the conveyor section or a pulse count corresponding to the movement distance of the conveyor section; and An organization that establishes a correspondence between multiple image data captured from different angles and the acquired identification codes and stores them.

9. A data provision method, wherein, The image data is provided by establishing a correspondence between the image data and the identification code of the item from a storage device storing data stored by the data collection method of claim 1 or 2.

10. A non-transitory storage medium storing a program product that causes a computer connected to a sorting machine sorting items to different sorting destinations to perform data collection processing, wherein, The program product causes the computer to perform the following processes: The identification code read by the reader is obtained from the reader installed on the sorting machine that reads and identifies the identification code of the item; Two or more cameras are installed to capture image data of the items identified by the identification code and placed within the conveyor section of the sorting machine from different angles. The images are taken from various angles of the items identified by the identification code. The image data is acquired from the camera at the moment the item identified by the acquired identification code enters the camera's field of view. The timing of acquiring the image data is predetermined by the layout of the sorting machine, the camera's placement, and a waiting time set according to the conveyor speed of the conveyor section, or a pulse count corresponding to the distance the conveyor section has traveled. Multiple images captured from different angles are linked to the acquired identification codes and stored.

Images